Reputation services, a brief history

| No Comments

Or,

The Problem of Twitter, Hatemobs, and Denial of Service

The topic of shared blocklists is hot right now. For those who avoid the blue bird, a shared blocklist is much like a shared killfile from Ye Olde Usenet, or an RBL for spam. Subscribe to one, and get a curated feed of people you never want to hear from again. It's an idea that's been around for decades, applied to a new platform.

However, internet-scale has caught up with the technique.

Usenet

A Usenet killfile was a feature of NNTP clients where posts meeting a regex would not even get displayed. If you've ever wondered what the vengeful:

*Plonk!*

...was about? This is what it was referring to. It was a public way of saying:

I have put you into my killfile, and I am now telling you I have done so, you asshole.

This worked because in the Usenet days, the internet was a much smaller place. Once in a while you'd get waves of griefers swarming a newsgroup, but that was pretty rare. You legitimately could remove most content you didn't want to see from your notice. The *Plonk!* usage still exists today, and I'm seeing some twitter users use that to indicate a block is being deployed. I presume these are veterans of many a Usenet flame-war.

RBLs

The Realtime Blackhole Lists (RBL) were pioneered as an anti-spam technique. Mail administrators could subscribe to these, and all incoming mail-connections could be checked against it. If it was listed, the SMTP connection could be outright rejected. The assumption here was that spam comes from insecured or outright evil hosts, and that blocking them outright is better for everyone.

This was a true democratic solution in the spirit of free software: Anyone could run one.

That same sprit means that each RBL had a different criteria for listing. Some were zero tolerance, and even one Unsolicited Commercial Email was enough to get listed. Others, simply listed whole netblocks, so you could block all Cable ISPs, or entire countries.

Unlike killfiles, RBLs were designed to be a distributed system from the outset.

Like killfiles, RBLs are in effect a Book of Grudges. Subscribing to one, means subscribing to someone else's grudges. If you shared grudge-worthy viewpoints, this was incredibly labor saving. If you didn't, sometimes things got blocked that shouldn't have.

As a solution to the problem of spam, RBLs were not the silver bullet. That came with the advent of commercial providers deploying surveillance networks and offering IP reputation services as part of their paid service. The commercial providers were typically able to deploy far wider surveillance than the all-volunteer RBLs did, and as such saw a wider sample of overall email traffic. A wider sample means that they were less likely to ban a legitimate site for a single offense.

This is still the case today, though email-as-a-service providers like Google and Microsoft are now hosting the entire email stack themselves. Since Google handles a majority of all email on the planet, their surveillance is pretty good.

Compounding the problem for the volunteer-lead RBL efforts is IPv6. IPv4 was small enough you can legitimately tag the entire internet with spam/not-spam without undue resources. IPv6 is vastly larger and very hard to do comprehensively without resorting to netblock blocking. And even then, there are enough possible netblocks that scale is a real issue.

Twitter Blocklists

Which brings us to today, and twitter. Shared blocklists are in this tradition of killfiles and RBLs. However, there are a few structural barriers to this being the solution it was with Usenet:

  • No Netblocks. Which means each user has to be blocked individually, you can't block all of a network, or a country-of-origin
  • The number of accounts. Active-users is not the same as total-users. In 2013, the estimated registered-account number was around 810 million. Four years later, this is likely over a billion. It's rapidly approaching the size of the IPv4 address space.
  • Ease of setting up a new account. Changing your IP address Changing your username is very, very easy.

The lack of a summarization technique, the size of the problem space, and the ease of bypassing a block by changing your identifier mean that a shared-blocklist is a poor tool to fight off a determined hatemob. It's still good for casual griefing, where the parties aren't invested enough to break a blocklist.

The idea of the commercialized RBL, where a large company sells account-reputation services, is less possible. First of all, such an offering would likely be against the Twitter terms of service. Second, the target market is not mail-server-admins, but individual account-holders. A far harder market to monitize.

The true solution will have to come from Twitter itself. Either by liberalizing their ToS to allow those commercial services to develop, or developing their own reputation markets and content tools.

Leave a comment

Other Blogs

My Other Stuff

Monthly Archives