Jun 11, 2009

Slashdotted; Post-mortem

About a day and a half ago, Reddit user stderr posted a link to the PHP documentation showing that GOTO will be a part of the language as of version 5.3. Somewhere in the reddit comments for this post, a link was pasted to an entry on this blog where I announced the feature being added years earlier. This naturally brought out the usual flame wars that circle around something like GOTO and drove some new traffic to my blog. No big deal, my server is pretty low-traffic, it can handle a few extra hits.


Within a few hours, burghler had browsed through other entries on my blog, finding what was at the time, the most recent entry about my friend's experience burying her mother. Just like the first reddit post, which had made the front page, this one also had a somewhat incendiary title.


Okay, more than a little incendiary, but I'll get to that in a moment...

The lesson

You would think, given that I'm the Architect for Yahoo WebSearch Front-end Engineering, that I would know something about configuring a server to not fall over under load. And in fact, I spend a good portion of my time on making sure that unexpected traffic spikes aren't enough to make a server get overloaded and trigger a chain-reaction of front-ends falling over. I really have no excuse for not preparing my server for what happened.


When I woke up Wed morning, I found that my server just didn't seem to respond to SSH or HTTP attempts. A reboot request didn't help, and the colo folks insisted that the server was simply running slow, but it was running. So I left my ssh connection attempt running and eventually I did get a login prompt. Several pained minutes later I managed to get an iptables rule in place to block off the flood of traffic (quicker than trying to stop the webserver). Suddenly, the CPU load was gone! Turns out I had my MaxClients setting much too high, and after a certain degree of concurrency, enough web-server children had spawned off to use up the available memory, which triggered disk swap, and made the CPU load 10x worse. Again, I know this effect exists, I really should have set up this server better.


A few tweaks to the config later and I got my server running smoothly. I also took the time to re-run my access log statistics. Of the aproximately 3 years of stats I've got, a full 2% of my hits were logged yesterday. Impressive reddit.... you win this round...

Bright squares indicate heavy traffic, dark squares indicate low traffic


Prior to yesterday's traffic

Jan






























Feb






























Mar






























Apr






























May






























Jun






























Jul






























Aug






























Sep






























Oct






























Nov






























Dec































01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31


Current stats

Jan






























Feb






























Mar






























Apr






























May






























Jun






























Jul






























Aug






























Sep






























Oct






























Nov






























Dec































01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

My flickr views...



Clarifications

Despite the title of the reddit entry, I have nothing against Christians. Further, I count myself as one. So if you really want to take a piss at people who are calling Christians evil, don't aim at me. Thanks.


Moreover, I don't think those people were evil, or even horrible (though I did use that word in the heat of the moment). A loved one had just died, and everyone was in pain. Death SUCKS, and it was a bad situation no matter how you slice it. I don't hate those people for trying to erase my friend though, I pity them. I pity the fact that they turned down the chance to mourn the passing of their loved one by crying on another loved one's shoulder. I pity the fact that they didn't get nearly the closure that my friend did. I pity them for willingly becoming victims of their own grief.


The rest is between them and God.

Jun 5, 2009

Wait, did you seriously just say that?

I grew up in California, and my state is pretty well true to it's reputation of being....less conservative than average. Granted, I've been harassed, shunned, even outright beaten over being queer, but I always thought that when it comes to certain things, especially family, there's a line that just isn't crossed. This is why the events of the past weekend so thoroughly amazed me.

Several months ago, when she told me that her mother had Cancer, I promised my kinda-ex-nevermind-it's-complicated-girlfriend that when the time came, I'd jump on a plane and fly out to the Midwest to help her bury her mom. So it was, that when she called me from Toronto (she was in the middle of a business trip), I booked a one-way flight (we weren't sure how long things would take), and flew off to Cleveland to meet her and offer my sympathies.

Death is hard, no matter how prepared for the end you are, but for my friend it was doubly difficult. For the past four years, she'd only rarely been able to see her mom, due to both distance and conflict between her and other people in her mother's family. We'd hoped that in the darkness of losing a loved one, those hard feelings and prejudices could be set aside, that family could come together for just one day to share grief and say good-bye. We were unbelievably wrong...

We landed together at CLE on friday night and headed over to her dad's house (her parents had been long divorced). He greeted us, filled her in on the past few months and her mother's last days, and we left feeling the weight of the occasion, but ready to grieve. On the way out, her sister called saying, "Some of the family have asked that you don't show up to the viewing. They say you can come into the funeral home for a few minutes afterward, but not during the general viewing."

I know, take a minute to absorb that statement. Who are these family members? Vague mumbles always answered this question, but they were certainly not going to tolerate my friend's presence. We resolved to call around the next day...

It was Saturday and the obituary had come out. Anachronistic newsprint mourned the loss of a sainted mother to my friend's sister, "special mother" to my friend's ex-spouse, but no mention of my friend. They had simply erased her, with a nod to her ex as a special kind of insult.

My friend called her sister back, wanting to understand how so much animosity could survive and flourish like that. She wasn't going to be erased, so she told her sister that the family could just deal with the fact that she would be present at the viewing. It was her mother that'd died, and she intended to say good-bye. A few hours later, news had filtered through to her step-father who called and delivered a matter-of-fact edict. "Here's your options, and I'm only going to say this once. You can come at 7, after everyone else is done, and take your 15 minutes to say good-bye, or you can not come at all. I'll have a cop at the door to make sure of that. *click*"

Yes, you heard that right, he was hiring an off-duty police officer to keep her out of her own mother's services.

And before you ask, yes, her step-father had that legal right, as the funeral home was private property contracted by him. So it was, that the next day, we (My friend, another friend of her's, and myself), drove to a shopping center near the funeral home and parked where our cars couldn't be identified and vandalized, and we walked to the funeral home at the official start time. Sure enough, a uniformed police officer stood guard at the entrace.


Arriving

We sent in the third member of our party, as she comes off much less threatening than I, and hadn't been explicitly told not to show up. She went inside, and formally asked the funeral home's director (with the step-father standing nearby), if we could be allowed inside. "Oh my God! They're here! They're here!", I wasn't present, of course, but I'm told the step-father reacted like we'd just stormed the beach at Normandy. "Those people are not to be allowed inside under any circumstances!", the funeral director could only agree with his client's request, so our other friend walked back to the sidewalk, where we sat down and initiated project Ghandi.

Okay, we didn't have any clever name for it at the time, but the premise was simple enough. We were in a labor state where the right to strike is sacrosanct. Being a public easement, we knew we were in our legal rights to remain there. This was the crux moment. Although we had legal rights, the police officer could have trumped up a reason to remove us for the duration of the ceremony, and we'd have lost our gamble completely, but he just stood at the door, showing no hint of emotion of sympathies for either side. A credit to his uniform.

I'm told, from those who were inside, that the step-father was yelling at the cop to have us removed and/or arrested. The cop answered that we were on public grounds and not causing a disturbance, so there was nothing he could do. I'm told he would get up every 15 minutes to look out the window and see if we were still there. We were. I'm told he called the police department to have additional officers sent around to "round us up", each new arrival told him the same story, no laws were being broken.


The challenge

About half an hour into the services, we were approached by the ex, who calmly flipped through the binder full of memories my friend had brought with her. They hugged, but didn't say much... What can be said at that point? Then my friend's ex-mother-in-law came over and just as I thought we were going to see another calm, but sad episode of sharing... "You know, you brought this on yourself."

I felt numb for a moment. I couldn't possibly have heard that come out of her mouth, not with my friend's mother lying in a casket 20 feet away... My friend managed to utter, "No", then turned away and began crying. "Yes, you did. You brought this on yourself." Our other friend interjected, "That's really not necessary..." "Yes it is", the mother in law replied, "this situation is because of her actions..."


She didn't get to finish that sentence. Something inside me snapped and a wellspring of rage came over me. I stood directly in front of the MiL, and found the most even temped voice I could manage, "Ma'am, I understand that you've just lost someone dear to you. I understand that you're in pain, and I understand that you think you're doing good, so I'm going to try to say this in the nicest way possible.", from here, my decorum ran out of rehearsed speeches and my rage took over, "You are an unchristian, bad...bad... Horrible person."

"No she isn't.", I heard from my firend's ex, but I ignored that. I stared at, and through her MiL, almost wanting a physical confrontation. "Sara..." The plaintive caution of our other friend snapped me out of my anger and I stepped away, andrenaline thundering through my blood. The MiL gave up trying to berate my friend and began to walk away. "I'm sorry", I offered, "That was unnecessary and uncalled for, I apologize." I don't remember much of the next several minutes, as my stress level slowly calmed down.

"Thank you", my friend told me. "I needed some strength right there and that's exactly what you gave me." After a moment she smiled, "You did unload on her a bit though...". "Yeah", I smiled back, "I was killing a few of my own demons there at the same time, might've over-estimated the firepower...

Soon afterward, her step-grandmother came around. "Why don't you just leave? You've making a spectacle of your mother's viewing!" She continued and I held myself back as long as I could. My friend held her own against this onslaught much better, continually drawing the conversation back to her mother, and trying to remember the best of her mother and focus on the mourning of her passing. The step-grandmother continued to catalouge all the (made-up) offenses my friend had allegedly committed against the family (all of which were thinly veiled reproachments for her queerness), until I spoke up, asking her if Jesus taught forgiveness, "Yes, BUT... we someone, when you have, when the person that has brought you into this world..." Honestly, even though I captured video of the entire event, I couldn't tell you for certain what she said as her argument descended and crashed into incoherency. Eventually she threw her hands up and walked away. Our other friend looked at me a little dumb-founded... "That... Was almost a thing of beauty the way she fell apart..." Yeah, I felt a little guilty at the efficacy of it too....


Redemption

Next up, was my friend's cousin. "What are y'all doing out here? Aren't you coming inside?" We told him what the step-father had said. His curious smile fell into disbelief, then anger rose in his face until his aura throbbed a dark crimson red. "That's ridiculous, I'll be right back." He stormed inside, and I'm told he raised holy hell on our behalf. Shortly thereafter, several more family members came out until there was all but a line queuing up around my friend to console her and share grief over her mother's death. For myself, I began to worry that we'd start to qualify as an unruly mob and give the police something to raise a fuss about. Nothing happened though, I think I even saw the cop smile.

Vindication

Eventually her mother's nurse, who was no part of the family, pulled my friend aside. "I only knew your mother for a few months, but she told me things about you in that time. Good things.", and she proceeded to list off things that she couldn't have made up. Things that proved to my friend that her mother really had loved her at the end, but was too afraid to show it.

My friend visibly brightened after that. As though she'd let go of something sour and offal. A few more hours passed in the pleasant late-spring weather. People came out and went back in, arrived and departed, noone else said an unkind word after her step-grandmother. Finally, when the funeral home was nearly empty, the cop approached us, "The funeral director's told me it's okay for you to come in now. I'm sorry for your loss." We thanked him and shook his hand and made our way inside. The step-father and other naysayers were nowhere to be seen, but there at the end of an array of fold-out chairs, was my friend's mother.


A poetic ending

I won't go into the details of her weeping, but she cried out the last of her grief. After hours of sharing, but not having access to the body, my friend had excised the worst of her grief already. She had prepared herself and found the peace she needed to see her mother's still form. For all her step-father's machinations and attempts to rob her of her mother and her place in her family, my friend was able to grieve her mother's passing in a way that couldn't possibly have been any better or more complete. If we had been inside the whole time, the shock of seeing her with so many unresolved things would have been devastating. If we had simply taken orders and not shown up until the last few minutes, there would not have been nearly enough time, and there would have been no family to lean on. By standing on the sidewalk, we took away that power, we reclaimed it as our own, and we amplified it even further. We said good-bye.

And the naysayers? They just looked like monsters. Love: 1, Hate: 0; Schadenfreude, I haz it.

Jan 8, 2009

Yahoo!, doing something right!

I love it when Yahoo! manages to just get something right. Checkout the awesome sauce of a media player widget. By doing nothing more than linking to an external script. All three of these P3 podcasts become magically playable directly from my blog.

Click the play buttons, but don't forget to notice the little slide-out player in the lower-left corner. Swanky, n'est-ce pas?