October 2019 Archives

A few of you may have noticed the link to 2005 in the last entry was dead. It took me a while to notice that too, and lead me to notice a lot of the older stuff wasn't resolving. It took more work than I wanted to get everything republished, but it's there now.

Along the way I discovered this gem from 2009, celebrating the 999th post on the blog.

It took a bit over 5 years to get there. And yeah, the first entries here back in 2004 look a lot like how folk use twitter now.

From 2009 to 2019 I've posted another 550 or so posts. So, my rate has slowed to a quarter of what it used to be. I blame a lot of things:

• No longer working somewhere that a Freedom of Information Act filing could get everything I was working on. I now have secrets I need to keep, and what I'm working on can count as a secret.
• The death of Google Reader, and a large number of my readers along with it.
• Twitter.

I'll probably reach 20 years on this blog, which is kind of scary.

If you've been in the tech industry for longer than 7 years, chances are good you've lived through some paradigm change in how things work. I've been here since 1996, so I've seen a few. And I'm looking down the barrel of another one. So I figured I would go over the renewal cycles I've personally lived through.

Ordered by dates impacting my career.

The Rise and Fall of Novell NetWare (Predates me, until about 2007)

Office automation was Novell NetWare for years. It was the first server in most offices, until they were replaced by either Windows NT, Windows 2000, or Windows 2003 machines. It was honestly the first distributed platform I ever worked with, since Novell Directory Services was a distributed database. It doesn't qualify as 'distributed' under today's definition, but in 1996 when NetWare 4.0 came out? It was cutting edge.

It was a NetWare migration that got me into Systems Administration in the first place. I did all the Sysadminny stuff like backups, user migrations, server patching and updates, and server consolidations. Servers were well pampered pets with cutesy names.

For me, NetWare died when my dayjob decided to get rid of it. I'm not at all surprised at the choice in retrospect. The upgrade at the time was from NetWare 6.5 (Novell kernel, 32-bit) to Open Enterprise Server (Linux kernel, 64-bit), and the effort to upgrade to that versus migrating everyone to Windows was equal enough we went to Windows instead.

Fun fact for newcomers: This very blog was hosted on NetWare for the first few years of its existence! It ran Apache! And Tomcat! Have a flash-back to that time.

This was the first operating system to die underneath me.

The Rise and Fall of File-Servers (Predates me through 2011)

File-serving is what NetWare did. The Windows migrations done were all to replace that functionality, and the NetWare admins involved were pissed because it was a step back in functionality. In my opinion, Windows didn't catch up to the NetWare feature-set until Server 2008.

But by 2008, the dedicated file-server was on the decline. At first it was NetApps, dedicated NAS systems.

Then it was things like SharePoint, that moved storage off of the direct-mapped file-share to application-specific application shares.

The university I was working for was one of the last market-segments to still operate large, clustered file-servers. When I left the university, it was in part because my dayjob had me working on a dead technology as a majority thing I was working on.

I'm not familiar with any new deployments of true file-servers. The orgs running them all started running them during the file-server era and haven't gotten rid of them yet.

The Rise and Fall of Actual Datacenters (Predates me to 2015)

The cloud mostly won. You need to be a certain size of company to make the physical plant of a datacenter make sense financially. Rack-n-stack sysadmins like me have moved on to cloudier things.

The Rise and Fall of Microsoft Windows (1997 through 2013)

I picked up Windows concurrent with NetWare, and grew my fluency with both at about the same rate. That made me marketable, which was nice. In the beginning (1997 through 2003) servers were still pampered pets with cutesy names. Once I moved to the university, the cutesy names mostly went away since our fleet was big enough we couldn't keep track of what Oberon did, but could figure out what AD-DC-3 did (2003-2011).

Then I moved to a startup that was using Windows as part of a SaaS product. That was an interesting time, but I learned lots doing it (2011-2013). By that time, servers were mere VMs with auto-generated names.

It stopped being relevant to my career in 2013, but I do have a soft spot for Windows.

The Rise and Fall of VMWare (2008 through 2015)

This was a big sea-change in how IT systems were managed. Once the Intel ISA support was there, ESX changed everything in IT. At the university I was working at, the rack with the ESX servers in it ate a sizable portion of the power-budget for the entire computer-room, at a fraction of the space that power-budget took in 2003. It was amazing.

Used it again at my first startup to manage an internal cloud of machines.

The big-corp after that was all ESX again with whole datacenters and a Managed Services platform on it. As I was leaving they were in the process of a lift-and-shift to AWS so they could close down their datacenter colos.

My current job does nothing with it because we have no physical datacenters.

The Rise of Linux (2011 to current)

I've been a Linux user since 1995, but wasn't allowed to administer it professionally until I moved to my first startup. This was one of several reasons why I left the University, by the way. I'm not going to spend much text on it, because Linux is central to so many things right now.

The Rise of Amazon AWS (2011 to current)

Using someone else's computers to do things! This was like VMware, but you didn't have to manage datacenters, or hardware, or power budgets, or rack-space, or cooling density, or cable-lengths, or server warranties, or out-of-band networks, or...

You paid someone else to handle all that, so you could focus on what you needed to focus on, which was provisioning operating systems and applications. And maybe not even that, if you end up using the managed-services like Relational Database Services, or go 'serverless' with Lambdas, ECS, and other techniques.

The Rise and Fall of VM-Oriented Configuration Management Systems (2011 through 2020)

Puppet was the first big-bang application in this space (there were others before it, like cfengine), with follow-ups in Chef, Salt, and Ansible. But Puppet was where I started, and still am. I spend a big chunk of my day making changes to our puppet-configs in order to bring change to what's running on our stuff. Infrastructure as code!

But Docker.

Puppet/Chef and company were all about building a VM image, or maintaining a pre-defined config on a running box. It does a great job of that, and auditors understand it. This paradigm doesn't work in Docker, which uses overlays to achieve similar things to how Puppet used modules. It sure can produce a VM-sized Docker image. But that image is not stylish.

The future has Puppet relegated to managing the state-maintaining systems that don't Docker well, and can't be run by your Cloud provider for some reason (be it compliance restrictions or technical).

From NetWare on a cutely named server in a closet, to a completely unnamed Docker instance in the Cloud, my career has been through a few paradigm changes. Change is constant. Just when I feel like I really know something well, a new thing comes along and I start anew.

First Language: BASIC, or Pascal depending on how you count.

Basic came with some Commodore business machine that definitely wasn't the 64, 65, or 128. It was the BASIC that came with MS-DOS. It was where I learned that programming needs a big book with all the commands, a paradigm I carried with me until the dawn of Google. The first language I programmed in using mostly online documentation was Perl.

Pascal came with my first computer-class in high-school. Didn't do much with it other than learn basic programming things like functions, types, and datastructures.

Had Difficulties: Coffeescript/Javascript

This is a recent one. We have a Hubot-based chatbot at work, which is written in Coffeescript because we built the chatbot in the 9 or so months when Coffeescript was seen as a useful abstraction for Javascript. Javascript works differently enough from every other language I use that I had real trouble reasoning it. Kind of like someone who learned English, French, Spanish and German suddenly trying to pick up Mandarin; your assumptions are wrong in ways you can't predict.

Before Coffeescript took the crown, it was C++. Object-Orientedness blew my brains for years. It actually was Powershell that got me thinking about OOP in ways that made sense.

Most Used: Puppet, Bash, Python

Puppet is how I code my infrastructure these days (Terraform is in there too, but is less used since it changes less often). I'm pretty advanced in this, having gone so far as to write both custom types and facts in order to solve problems. The Docker-based future we're all heading towards has me sighing about having yet another foundational piece of my understanding rotting out from under me due to legacy. I'm very good at managing fleets of VMs. Container-ships will be the next platform, and puppet doesn't really play there.

Python and Bash are how I do complex things on my infrastructure. In spite of our Docker future, I expect this part to continue. Utility scripting is all about automating the Docker bits (if not custom monitoring scripts here and there inside ye olde container).

Ruby used to be there, but default-system-support for python is more complete, so it won out.

Totally Hate: C

My CompSci degree was in it, but I haven't written any C in over two decades at this point, and that's OK. I can barely read it anymore, but I keep at it since digging through some open-source utility's code is sometimes the only documentation offered.

Most Loved: Ruby, Powershell

The advanced bits of Puppet are still written in Ruby (the JRuby variant). My first startup was a Ruby startup which is where I learned it. And golly, that community is nifty. I just like it.

Powershell is on this list because it was the first sign of the new Microsoft I ever saw. A Microsoft that seemed to be getting it regarding how modern systems administration was done. It was the language that got me thinking about how everything-is-an-object actually works in modern languages, which set me up for learning Ruby later.

DayJob doesn't have me using Windows in anything, so I haven't touched PS in years. But it still holds a special place in my heard.