Today I'm removing a series of servers from our racks. These are really old servers. As in, I wouldn't be surprised if one of them is 11 years old kind of old. Why they're still in our racks has a variety of reasons.
This is looking down the inside side of the rack's vertical post. Look closely at that clip-nut. Notice anything weird?
The clips on that nut are on the wrong side. They're facing away from the rack. Attempting to unscrew this post caused the nut to spin around and around, and required me to hold the nut with my fingers.
What's more, the order is wrong. It should be screw, server rail, rack post, nut. Instead, it was screw, rack post, server rail, nut. Happily the screw heads were large enough they didn't fit through the rack post holes.
There were a few other rails mounted like that one. One memorable rail was screw, rack post, nut, server rail. That only worked because the rail was threaded, but at least the nut was mounted correctly. Nuts that spin freely are not good.
As near as I can figure, this is what happens when someone moves from racks with round holes to one with square holes and doesn't have a manual. This was several years before my time so I have no idea who did this. I have suspicions, but I'm not going to bring this up. It was 10 years ago, whoever it was has learned better since then.
More broadly, we need new racks. The ones we have don't have nearly enough back-of-rack space for everything that needs to cram in back there. Unfortunately, our large surplus of racks means that convincing the powers that be that we need new ones is very, very hard. Also, the power-strips we have for these racks are sooo 1990's. These racks were not designed for modern densities of 1-3U servers, they're designed for high densities of 5-9U servers. Because of their lack of back-of-rack space, we can't use those nifty modern power-strips that give you a display of the load on that strip.
When I went to put the rails in for the new tape library (YAY!) I found that those rails and our racks aren't really compatible. For whatever reason, if I put a rack nut in the hole, the rail's holes won't align with the nut. Since the rail's holes are threaded there is very little tolerance for that 2mm difference of opinion for where the center of the rack-nut needs to be. I managed to hack something together that'll keep the rails in, but it's still just wrong. You work with what you got.
- We have a hard time stopping use of old servers. There is a constant need for development and testing servers. Old crap serves this need well.
- Due to our blade consolidation project several years ago, we had more rack-space than servers, so we just didn't remove stuff.
- Due to our VMWare consolidation project (ongoing) we have more rack-space than we know what to do with, so old servers can sit in the racks for years before we need the space.
This is looking down the inside side of the rack's vertical post. Look closely at that clip-nut. Notice anything weird? The clips on that nut are on the wrong side. They're facing away from the rack. Attempting to unscrew this post caused the nut to spin around and around, and required me to hold the nut with my fingers.
What's more, the order is wrong. It should be screw, server rail, rack post, nut. Instead, it was screw, rack post, server rail, nut. Happily the screw heads were large enough they didn't fit through the rack post holes.
There were a few other rails mounted like that one. One memorable rail was screw, rack post, nut, server rail. That only worked because the rail was threaded, but at least the nut was mounted correctly. Nuts that spin freely are not good.
As near as I can figure, this is what happens when someone moves from racks with round holes to one with square holes and doesn't have a manual. This was several years before my time so I have no idea who did this. I have suspicions, but I'm not going to bring this up. It was 10 years ago, whoever it was has learned better since then.
More broadly, we need new racks. The ones we have don't have nearly enough back-of-rack space for everything that needs to cram in back there. Unfortunately, our large surplus of racks means that convincing the powers that be that we need new ones is very, very hard. Also, the power-strips we have for these racks are sooo 1990's. These racks were not designed for modern densities of 1-3U servers, they're designed for high densities of 5-9U servers. Because of their lack of back-of-rack space, we can't use those nifty modern power-strips that give you a display of the load on that strip.
When I went to put the rails in for the new tape library (YAY!) I found that those rails and our racks aren't really compatible. For whatever reason, if I put a rack nut in the hole, the rail's holes won't align with the nut. Since the rail's holes are threaded there is very little tolerance for that 2mm difference of opinion for where the center of the rack-nut needs to be. I managed to hack something together that'll keep the rails in, but it's still just wrong. You work with what you got.
Sadly, it might not be as you say. Especially in an educational environment.
I, yesterday as a matter of fact, found the same configuration in one of our racks while moving an older server from "here" to "there". I too groused about the lack of common-sense, professionalism, handy-manism and a few other "isms" under my breath as I "correctly" installed this set of rails into our spiffy new rack. Thank god they have me here to straighten out their mistakes, these poor unwashed heathens.
(*THUNK*)
Huh?
(*THUNK*)(*THUNK*)(*THUNK*)
WTF?
It doesn't fit! The server (a 1U unit) is EXACTLY the width of the inside of the rack's vertical rails. You can't get the machine rails to "fit around the corner" for a proper installation, ergo, the installation style you saw in your racks. So, I had to take the darned rails out and resize them and pull the nuts and all and reinstall them. Cursing the entire time I was doing it.
Today's lesson (for *all* of us, myself included) is:
Don't jump to judgements.
Not everyone's an idiot.
Sometimes, against everyone's better judgement, to get the job done you have to violate "common-sense".
As long as you don't endanger people's lives, fingers, feet or the machine's integrity, this is acceptable.
It's even better if you took the time to document it!
BTW, you DID document what you found and how you had to make it work, in some place other than your blog right??
Frickin' humble pie is NASTY this early in the morning but we have to learn from our mistakes and experiences or we're lousy sysadmins.