May 2022 Archives

Centralization of email

I've been managing email systems for darn near all of my career. I first started in any capacity in late 1997 and it has only had periodic interruptions. I'm no longer maintaining email for my business users, but I am making sure the email we send to our customers actually gets there. I'm not the person in charge of it, but I am recognized as the person who has worked on this stuff the longest.

Way back in the day, this blog used to be managed through Blogger, back when they allowed FTPing to remote sites. When Google launched Gmail, they invited their bloggers to join and talk about it. I was one of them. Nearly 18 years later, I'm still there, and Google has fundamentally rewritten what email means for the Internet. You can see some of that fight from my deep archives:

So, that was 13ish years ago. Now adays I'm in the outbound email business and all that implies. The other day I took a look at the logging for our mail sends to see what mail-servers we were talking to and ran some statistics. They're a weeny bit eye-opening. Here are the mail receivers that got over 1% of our mail:

  • 55% Google (includes Google Apps and Gmail)
  • 20% Microsoft
  • 4.5% Yahoo
  • 2.5% Point Proof Hosted
  • 2% Minecast
  • 1.5% Barracuda Networks
  • 1% Sophos
  • 13.5% Literally everyone else

Which means that two providers, Google and Microsoft, control about 75% of the email boxes we sent to that day. The rest over 1% are various email protection providers likely fronting self-hosted email systems.

This has profound effects on how email works as a whole. What Google says goes, and if Microsoft agrees everyone else has to deal with it or be left behind. Yahoo is the only other mail-provider to break the 1% line. If Google's spam algorithms suddenly mark you as suspicious, it can be weeks to dig out of that hole. Old standard techniques like DNS Reverse Blacklists are still used in part by the 25% non-GOOG/MSFT mailers, but getting blacklisted on those is something we can go a few days before noticing. As I wrote in 2007:

First and foremost, SPAM. The native anti-spam inside GroupWise is a simple blacklist last time I looked, which is effectively worthless in the modern era of SPAM.

Yeah, blacklists were definitely not the first line of defense even 15 years ago. They're absolutely not in the modern era. They're useful inputs to the spam/ham decision, but you get far more leverage out of building an IP reputation database of bad actors sending you stuff. And that benefits greatly from scale. Google and Microsoft will see the whole internet at some point, probably more than once a day. Hard to compete with that.

Finally, Google is killing off the open source protocols that used to be standard for accessing email: POP and IMAP. They're just too prone to attack these days, and they're password based which we know is a weak defense. Hard to two-factor-authenticate those without forcing the user into a browser anyway.