Printer praccounting02 no longer prints anything.After half an hour the helpdesk has added a few notes to the ticket:
Has been happening for about a week. Seemed to happen to a few people at a time. Now no one can print.Now it's up to you.
In the course of your investigation you discover the following:
- The office received an upgrade to MS Office 2010 in stages over the last three weeks.
- That upgrade project completed two days ago.
- The user who reported it was on vacation until yesterday.
- The department has three of the same kind of printer, but only one seems to be experiencing issues.
- The print-server shows received and printed jobs for all three, but that one provably is not ironing paper despite what the print-logs show.
That pretty clearly points a finger at the Office upgrade as being somehow involved, but the other two printers not being affected are, shall we say, confounding variables. What's up with that one printer?
You dig deeper:
- That one printer has some upgrades the other two don't:
- It has a 4th paper-tray that can hold 2000 sheets.
- It has had a memory upgrade
- The drivers to all three are the same, since that model of printer gets the same driver enterprise-wide.
- That printer seems to only print whacking huge Excel spreadsheets.
- The other two printers have the more normal mix of email, Word, and printed off web-sites but very few Excel jobs.
Ahah! Whatever is going on is related to very big Excel jobs. You relay this to the helpdesk and they're able to reproduce it with the printer by them (same model, by chance). Big Excel files, usually more than 10 pages, with at least one Hidden column. Jobs hit the printer and nothing happens, but are recorded as 'printed' by the print-server. At least it beats vomiting paper...
Suspecting that Office 2010 may have added something printer-drivers don't like, you hunt up an upgraded driver for that particular printer model. The changelog for the driver doesn't make much sense, but it is newer than what's in use by a good 18 months. You give it to the helpdesk, and it fixes their problem neatly; jobs enter and get printed as you'd expect. Horoay! A solution.
Since this is the first in the field driver install, the helpdesk invites you out to make sure they're doing it right. No problem, we like consistency around here. So you go with them to the affected office. Doing the trouble-reporter first since it'll give them resolution.
You two talk up and announce that you need to upgrade a printer-driver to make that printer work again.
They fail to purchase.
What? No! I asked you to fix the ****** printer not my *** ****** computer. Fix that!It takes a while, but eventually the two of you convince this departmental dragon that it wasn't his fault the printer was broken, it was the standard driver, that everyone was experiencing it, and it was a failure of the Office 2010 Deployment Project to catch the very-large-Excel problem back in testing. He's still grumbling, but at least he lets you into his computer.
And it works the first time.
He grumbles a thankyou, and you two move on to the next station.
Just another day in the office for the Helpdesk tech; they are front-line customer-service professionals after all, but not so much for the SA who got dragged into it. No one likes to have their good work thrown in their face and rejected for provably wrong reasons. It gets to you after a while. And if you're like a LOT of sysadmins out there you cultivate a fine sense of sarcasm, because so very few people out there are able to appreciate the finely reasoned research that led you to this particular conclusion.
Eventually the helpdesk will stop asking you out into the field (all that sarcasm makes their job harder, something they really don't need). Which is all to the good! Fewer end users to deal with. Except for the few who get your direct phone number or email, but sarcasm is good for that too, so that goes down to a trickle.
This is how sysadmins earn their reputation for being unapproachable grumpasaruses.
This is a defense mechanism, pure and simple.
However, other people are key parts of our jobs no matter how much some of us would wish otherwise. There is almost guaranteed to be a boss of some kind somewhere. Unless you do all of your own part-sourcing and replacement, vendors are going to touch some parts of your infrastructure. Peers in other departments. Other SA's in IRC while you troubeshoot a problem. Or end users noticing a problem not covered by the monitoring infrastructure and passing the word on.
That last bit is perhaps the most important. While the human layer of the monitoring environment is the most error-prone, it can notice errors that the rest of the automation doesn't. So it pays to be sure such error reports get to you. Which means being at least somewhat approachable.