I've been managing email systems for darn near all of my career. I first started in any capacity in late 1997 and it has only had periodic interruptions. I'm no longer maintaining email for my business users, but I am making sure the email we send to our customers actually gets there. I'm not the person in charge of it, but I am recognized as the person who has worked on this stuff the longest.
Way back in the day, this blog used to be managed through Blogger, back when they allowed FTPing to remote sites. When Google launched Gmail, they invited their bloggers to join and talk about it. I was one of them. Nearly 18 years later, I'm still there, and Google has fundamentally rewritten what email means for the Internet. You can see some of that fight from my deep archives:
- December 21, 2006: Where I talk about the realities of 2GB Exchange inboxes. This post goes into the quota limits we had at the time.
- March 3, 2007: The first in a series about an ongoing effort to outsource WWU's student email to something Campus IT didn't maintain. This was the right decision, but it took a while to bring it across the finish line.
- March 3, 2007 (more than one post a day! Those were the good old days): Where I respond to a Slashdot thread, also on the topic of outsourcing student email. I ended up describing a lot of the challenges of self-maintaining a large email system.
- March 5, 2007: Where I go into why we won't be using Novell GroupWise for student email. Going into the economics again, and touching on the spam problem as the driver forcing us into the SaaS space.
- October 13, 2007: More student email, but also a note about Gmail increasing their quota again. The other free email providers (old names like hotmail) had exceeded Gmail for storage quota as a way to compete. This was ultimately doomed, but they didn't know that yet.
- August 1, 2008: The first hints of what will become Microsoft Live and Office365. I called it "Exchange in the cloud" which was kind of what that offering was.
- April 30, 2009: The student email migration actually happens. The student body picked Microsoft over Google, because Microsoft had an answer to the "what are you doing with all of that data you're mining from our email" and Google had platitudes about trust. This article goes into more detail about the economics of why this was the right choice.
So, that was 13ish years ago. Now adays I'm in the outbound email business and all that implies. The other day I took a look at the logging for our mail sends to see what mail-servers we were talking to and ran some statistics. They're a weeny bit eye-opening. Here are the mail receivers that got over 1% of our mail:
- 55% Google (includes Google Apps and Gmail)
- 20% Microsoft
- 4.5% Yahoo
- 2.5% Point Proof Hosted
- 2% Minecast
- 1.5% Barracuda Networks
- 1% Sophos
- 13.5% Literally everyone else
Which means that two providers, Google and Microsoft, control about 75% of the email boxes we sent to that day. The rest over 1% are various email protection providers likely fronting self-hosted email systems.
This has profound effects on how email works as a whole. What Google says goes, and if Microsoft agrees everyone else has to deal with it or be left behind. Yahoo is the only other mail-provider to break the 1% line. If Google's spam algorithms suddenly mark you as suspicious, it can be weeks to dig out of that hole. Old standard techniques like DNS Reverse Blacklists are still used in part by the 25% non-GOOG/MSFT mailers, but getting blacklisted on those is something we can go a few days before noticing. As I wrote in 2007:
First and foremost, SPAM. The native anti-spam inside GroupWise is a simple blacklist last time I looked, which is effectively worthless in the modern era of SPAM.
Yeah, blacklists were definitely not the first line of defense even 15 years ago. They're absolutely not in the modern era. They're useful inputs to the spam/ham decision, but you get far more leverage out of building an IP reputation database of bad actors sending you stuff. And that benefits greatly from scale. Google and Microsoft will see the whole internet at some point, probably more than once a day. Hard to compete with that.
Finally, Google is killing off the open source protocols that used to be standard for accessing email: POP and IMAP. They're just too prone to attack these days, and they're password based which we know is a weak defense. Hard to two-factor-authenticate those without forcing the user into a browser anyway.