The SysAdmin Network

No more hiding in the server room

woe unto me becuase I'm working the weekend. How about you?

I run the IT department at an art museum. Here is a picture of one of the galleries about an hour ago:

Notice something? It is empty. Do you know why it is empty? Because we are CLOSED. Here is a shot of a few offices:

See how all the offices are dark? Again, because we are CLOSED. Perhaps you would like to come and see some of our art? Well TOO BAD!!! We are closed and, due to the holiday, we will stay closed until Monday.


Here is a picture of me:

Looks like I’m at my work doesn’t it? That’s because I am at my work!  It so happens that this is one of the rare occasions where the company will be closed to the public for 48 hours straight and therefore it’s my big opportunity for fun and excitement! Yes, I’ll be enjoying such thrilling pastimes as upgrading the IOS on various routers and switches, redoing all the cables in the wiring closet (no way that would be monotonous), re-racking our network gear, and upgrading the firmware on our SAN arrays! Oh, I’ll also be dealing with issues related to this previous post.


Fun, fun, fun! I can’t imagine a better way to spend a weekend. Most of the time this kind of thing can be done remotely but as I’m primarily working with our network I really need to be onsite. I’ve worked most of today and I’ll be up and at it at the abhorrent, abominable hour of 5:00AM for another long day. Incidentally I hate mornings; it’s verging on obscene that hours before 10:00AM-ish even exist. I wouldn’t mind this so much if this wasn’t the way almost all of our closures went. Guess where I was Christmas Eve last year? Hint: not at home.


Oh, my wife totally loves this too, btw. Earlier conversation, paraphrased:


her: so you’ll be working all weekend?!

me: umm, yeah. Hey, I know! You can come in and hang out with me at my work!

her (unleashing the icy stare of death) I’d just sit in your office while you’re on your computer?

me: Well, umm … I could show you how to update the BIOS on desktops! It’s really easy and …

her: icy stare is cranking up to 11

me: .. and, umm, never mind.

So this is my whining, self-pitying, tale of woe. It could be longer but I’ve got work to do. As an aside; the security officers on duty right now are playing the world’s saddest song on the world’s smallest violin for me. We’ve had security officers on duty 24 hours a day, every day, since the first day the museum opened. So, for some unfathomable reason, they’re not really sympathetic.


So are you working this weekend? Care to share any tales of crazy hours?

Views: 12

Reply to This

Replies to This Discussion

Do what I would do: 2 gallons of fresh water, 14 pounds of skittles and one boombox to crank up the Kittie ("Oracle" and "Until the End" would do nicely). I'd also mix some Nickel Creek and Kate Rusby in. Seriously, take this time to revel in your ability to crank some tunes up to vision impairing decibel levels. Never had your vision blurred because music was so loud? Try it. =)

Loud music makes time fly... as well as a cochlea or two. =)

And yes, I know a bit about rewiring network closets. Have a look at this thread over at the Daniel Petri forums where some decent advice (some by me) was given to someone who was recabling a rack. Bring some wet wipes. Stay away from the hard liquor. (Yes, that Petri thread quickly spirals into silliness. Mea Culpa.)
Your right, music helps, no doubt. I tend to bounce between techno and classical but I wasn’t able to crank it this time as the guards get all upset if they can’t hear each other :(

I read that thread … that’s a good idea about indicating port speed. I’m in the same boat about wishing I had spent more time planning and (especially) measuring. I object to your anti-scotch comments though!

How much of that Scotch was in you just befor you took that photo?=)

Were you able to get all of the updating done successfully? Has your wife's Icy Glary of Death warmed up to at least a Chilly Glance of Harm? Calling a florist and begging them to hand deliver a bouquet on Sunday might have helped (hint for next time). Or you could buy it a day in advance and have a friend hand deliver it. Or you could save up a stockpile of hand made poetry and letters, hide them in the house in various places and text message her once every few hours with a new location to look in. She may even come to look forward to your planned downtime weekends.

Or get her her own bottle of scotch? It's been rumored that innebriation kinda dulls eyeball daggers. =)
Sigh!, the hardest part of the job is making that phone call home at 6pm;

"Hi Honey, I know I said I wouldn't be late again but something has cropped up... `click!`, hello? hello?"

"innebriation kinda dulls eyeball daggers" - Sound relationship advice for any SysAdmin !
Yeah, I hate that one. I also I hate waking her up at 3:00AM to say I'm going into work to handle an outage.

Fortunately I've dumped a ton of time and money into our network over the last couple of years so those are quite rare now, whereas they were common events in the past.
Well I had been sipping on dram prior to that picture ;) That is an awesome hint to hide objects in the house!!!! I totally using that!!

My updates didn't go as well as I'd hoped; I got a lot done but I ran into some delays (big surprise) and wasn't able to get everything done, got the primary stuff done though. Upgrading to ESX4 vSphere was a breeze but I had issues with the code on our Cisco 2851 boxes. I think it was the SRST feature set that where causing issues. Anyway TAC got it worked out.

Scotch is isn't really her thing ... she refers to as that "vile stuff" that smells like "burnt wood".
I winced when you mentioned upgrading the firmware in your SAN. That always makes me queasy. Having issues updating the Ciscos kinda surprised me. Oh well... that's what Thanksgiving day is for! =)

Crank up the August Burns Red, pop your collar, snap your fingers at the security guys and commence the haxoring.
You're right, updating SAN firmware is always makes me a bit ... uneasy. No reason to think things will go wrong ... but if they did... Unfortunately, while the SAN fabric is redundant, I don't have the money for a redundant storage array so that array is a big fat single point of failure.

More worryingly is that a couple of years ago when I setup this array I made some config choices that seemed good at the time but now have boxed us into a corner. As a result I need to copy all the data off somewhere, blow all the RAID groups (moving from RAID10 to RAID6) and copy the data back. I only have around 10TB of data so this isn't a huge undertaking but I'm projecting this taking around 36 hours of down time.

It needs to be done sooner rather then later because I'm upgrading our VMware infrastructure to vSphere4 from VI3. I had considered saving the SAN rebuild until after the VMware upgrade but we can't do that because I configured the new servers without internal disks as I'm going to do FC boot. Can't install\boot those servers until the array rebuild.
heh, I work every weekend, not to mention crazy hours.

the work week here is sunday-thursday, while, having to support the rest of the world, I end up working mon-fri, sometimes having to stick around for US hours, while I'm in GMT+2 area.
Sorry that sucks. I think you win; a schedule that's out of sync for both time and day for your location totally beats out my story. I was just whining about working on a holiday ... but that's all time for you.
LOL, there are positive sides to this as well - I get Sundays off, which is a normal working day here. Just imagine not having to work on Mondays :)
Hey, what do you use there to manage the collection, out of interest, if you have a computer based collection management system?

RSS

© 2012   Created by Elizabeth Ayer and Michael Francis.   Powered by

Badges  |  Report an Issue  |  Terms of Service