4 ms·

> Botnets appeared as a way to get around RBLs, and in response spam fighters mapped out the internet to create a "policy block list" - ranges of IPs that were

> Botnets appeared as a way to get around RBLs, and in response spam fighters mapped out the internet to create a "policy block list" - ranges of IPs that were assigned to residential connections and thus should not be sending any email at all. This is the wrong way to do it. It may make things easier for Google. It may save them time, money, and worries. This is still the wrong way to do it, for one simple reason: it is now impossible to send email from home. Instead, you have to have a relay or a VPN that does it from a fixed IP that'd better not belong to a subnet in disrepute. It keeps the email network centralised, an antithesis to its origins, where an email went straight to the machine of the final recipient, from the machine of the original sender —with no intermediates besides the DNS servers. I cannot help but notice this centralization is convenient for Google and other huge mail providers. Whatever their actual intentions when they did this I don't know, but they clearly have no incentive to change that back (their business is to read your email for profit after all —the fact this is all automated only makes it worse, not better).

jcranmer 10y ago
My suspicion is that DNS block lists have largely been replaced at the major providers by reputation analysis, although the end effect is likely to be the same (which is to say that residential connections have their reputation so shot to hell you effectively can't send email). That said: do you have a better solution? Note that doing nothing is not acceptable--the cost of spam has been such that consumers have opted for anti-spam solutions over the purity of the original email architecture.
- loup-vaillant 10y ago
  > That said: do you have a better solution? Local (each his own) Bayesian filtering. Works well for me, with both Evolution and Thunderbird mail clients, and I have my email out there in plaintext in the open web.
tedunangst 10y ago
> an antithesis to its origins, where an email went straight to the machine of the final recipient, from the machine of the original sender —with no intermediates besides the DNS servers. Wow, is that some historical revisionism. Emails used to look like thrash!cmu!ucb!vax!tedu which meant send it to thrash, who will send it to cmu, who will send it to ucb, who will send it to the vax, which will deliver it to tedu. If you had a direct line, you could skip a few hops, but very few people were talking directly to the machine of the final recipient.
- drfuchs 10y ago
  No, you're the one with incorrect history. The uucp email scheme with all the exclamation points actually appeared the better part of a decade after ARPANET email with the simple, direct addressing and delivery was in wide use. Perhaps you are confused because UNIX came late to the nascent Internet, while the various 36-bit DEC machines and OSes were there from the very beginning. Many thousands of students and academics were sending email to foo@bar.edu for dozens of values of bar before anybody ever saw !ucbvax! in an address.
  - tedunangst 10y ago
    Yes, that works when the network is small enough to put literally every computer in the hosts file. But once your local sysadmin tires of updating the file every time some university across the world plugs in a computer, it starts falling apart. To the original point, if I somehow plugged my time warped laptop into arpanet, nobody would be able to send me email until I made a bunch of phone calls. (But thanks for the correction, I'd overlooked even earlier history.)
- dredmorbius 10y ago
  This misses the point. Bang paths were used (and I used them) because we didn't have DNS and MX records. Which meant that routing was done manually, by the mail sender. There was an end-to-end wired connection (usually -- batched and gathered mail, via UUCP was also a thing) between hosts. And pretty much any host, through intermediaries, could talk to any other. Traceroute will show you a fairly equivalent system today. The big difference was that the Internet (or its various ARPA / DARPA precursors) was a lot smaller. A few hundred hosts in the early 1980s, a few thousand by the end of the decade. And everyone knew each other. There was a book, literally, hardcopy print, with every site operator's name and phone number. Twice, for multiple indexing methods. In your example, tedu wouldn't have any cause to deny mail from thrash, or any other host, directly peered, or bang-pathed. At least not until Green Card (which was Usnet, and I remember that too).
honkhonkpants 10y ago
Nothing Google does prevents you from enjoying the end-to-end nature of email. Send it to whomever you want. However the end-to-end nature also allows the endpoint to reject your message.
- loup-vaillant 10y ago
  When the "endpoint" has millions of users, like Gmail, it is not an endpoint. It's a hub. I don't care it is technically and end point, in practice it's highly centralised. It's not like each user chose to reject such and such message. No, Google is rejecting incoming mail for them —in addition to individual spam boxes. That does prevent me from enjoying the end-to-end nature of email to an extent. Oh, and, constant man-in-the-middle spying, which is also not exactly end-to-end.
dredmorbius 10y ago
So, you're right and wrong (and tedungst is as well). Yes, the original Internet (and its precursors) had very little by way of limitations on what any one node could do. But the original Internet was tiny. I've been in modest-sized offices which had more networked compute nodes than the early 1980s DARPANet had. The Internet is as much a social phenomenon as a technical one, and one problem is that after we start hitting above about 300 of anything, behaviors start to change. At some point I'd really like to find a good behavior-by-scale reference, but quite simply, an Internet with billions of nodes cannot be run the same way one with a few hundred, or even tens of thousands, can. And yes, this frustrates me too. A huge problem is reputation, reputation management, and reputation sharing. I've watched much of the sordid mess of spam and anti-spam measures develop and evolve, and what does and doesn't work. I'd still like to see vastly better peer-based reputiations systems built directly into communications software (not just email). We'll see about that. One reason residential dialup, DLS, and cable are service no-go zones is that every small operator has their own system for carving off spam. And most of them are very, very crude. Lauren Weinstein, an Official Old Fart, announced some time back that he'd simply blackholed all of China for his personal email domain. Too much abuse, no valid content. Sucks if you're in China and want to reach Lauren. Oh well. Multiply Lauren out a few thousand or million times. There are a few possible solutions: 1. Buy business-grade Internet. It's expensive, but it's not residential, and you may be able to run services on it. Part of what you're paying for is for the service provider to call you up to bitch about shit flowing over your pipe that shouldn't be. 2. Set up specific understandings with the services you peer with. If it's just you and a few buddies, great. If you're trying to get email through to Google or a self-hosting F-50 company, good luck. They've got policies and shit and take ages. (Google may be fairly responsive, F-50 companies, in my experience less so.) 3. Build (or help build) something better. SMTP email has been with us a long time, has some useful features, and has served us well. But it's also got some tremendous shortcomings. An alternative with robust encryption, anti-spam features, poll-and-fetch (rather than unsolicited-send), better multiplatform support, and the ability to dispense with overly-formatted email (I still prefer console mailers), might be a good start. Being fully peer-to-peer even better. That gets complicated though.

jcranmer 10y ago

My suspicion is that DNS block lists have largely been replaced at the major providers by reputation analysis, although the end effect is likely to be the same (which is to say that residential connections have their reputation so shot to hell you effectively can't send email). That said: do you have a better solution? Note that doing nothing is not acceptable--the cost of spam has been such that consumers have opted for anti-spam solutions over the purity of the original email architecture.

loup-vaillant 10y ago
> That said: do you have a better solution? Local (each his own) Bayesian filtering. Works well for me, with both Evolution and Thunderbird mail clients, and I have my email out there in plaintext in the open web.

tedunangst 10y ago

> an antithesis to its origins, where an email went straight to the machine of the final recipient, from the machine of the original sender —with no intermediates besides the DNS servers. Wow, is that some historical revisionism. Emails used to look like thrash!cmu!ucb!vax!tedu which meant send it to thrash, who will send it to cmu, who will send it to ucb, who will send it to the vax, which will deliver it to tedu. If you had a direct line, you could skip a few hops, but very few people were talking directly to the machine of the final recipient.

drfuchs 10y ago
No, you're the one with incorrect history. The uucp email scheme with all the exclamation points actually appeared the better part of a decade after ARPANET email with the simple, direct addressing and delivery was in wide use. Perhaps you are confused because UNIX came late to the nascent Internet, while the various 36-bit DEC machines and OSes were there from the very beginning. Many thousands of students and academics were sending email to foo@bar.edu for dozens of values of bar before anybody ever saw !ucbvax! in an address.
- tedunangst 10y ago
  Yes, that works when the network is small enough to put literally every computer in the hosts file. But once your local sysadmin tires of updating the file every time some university across the world plugs in a computer, it starts falling apart. To the original point, if I somehow plugged my time warped laptop into arpanet, nobody would be able to send me email until I made a bunch of phone calls. (But thanks for the correction, I'd overlooked even earlier history.)
dredmorbius 10y ago
This misses the point. Bang paths were used (and I used them) because we didn't have DNS and MX records. Which meant that routing was done manually, by the mail sender. There was an end-to-end wired connection (usually -- batched and gathered mail, via UUCP was also a thing) between hosts. And pretty much any host, through intermediaries, could talk to any other. Traceroute will show you a fairly equivalent system today. The big difference was that the Internet (or its various ARPA / DARPA precursors) was a lot smaller. A few hundred hosts in the early 1980s, a few thousand by the end of the decade. And everyone knew each other. There was a book, literally, hardcopy print, with every site operator's name and phone number. Twice, for multiple indexing methods. In your example, tedu wouldn't have any cause to deny mail from thrash, or any other host, directly peered, or bang-pathed. At least not until Green Card (which was Usnet, and I remember that too).

honkhonkpants 10y ago

Nothing Google does prevents you from enjoying the end-to-end nature of email. Send it to whomever you want. However the end-to-end nature also allows the endpoint to reject your message.

loup-vaillant 10y ago
When the "endpoint" has millions of users, like Gmail, it is not an endpoint. It's a hub. I don't care it is technically and end point, in practice it's highly centralised. It's not like each user chose to reject such and such message. No, Google is rejecting incoming mail for them —in addition to individual spam boxes. That does prevent me from enjoying the end-to-end nature of email to an extent. Oh, and, constant man-in-the-middle spying, which is also not exactly end-to-end.

dredmorbius 10y ago

So, you're right and wrong (and tedungst is as well). Yes, the original Internet (and its precursors) had very little by way of limitations on what any one node could do. But the original Internet was tiny. I've been in modest-sized offices which had more networked compute nodes than the early 1980s DARPANet had. The Internet is as much a social phenomenon as a technical one, and one problem is that after we start hitting above about 300 of anything, behaviors start to change. At some point I'd really like to find a good behavior-by-scale reference, but quite simply, an Internet with billions of nodes cannot be run the same way one with a few hundred, or even tens of thousands, can. And yes, this frustrates me too. A huge problem is reputation, reputation management, and reputation sharing. I've watched much of the sordid mess of spam and anti-spam measures develop and evolve, and what does and doesn't work. I'd still like to see vastly better peer-based reputiations systems built directly into communications software (not just email). We'll see about that. One reason residential dialup, DLS, and cable are service no-go zones is that every small operator has their own system for carving off spam. And most of them are very, very crude. Lauren Weinstein, an Official Old Fart, announced some time back that he'd simply blackholed all of China for his personal email domain. Too much abuse, no valid content. Sucks if you're in China and want to reach Lauren. Oh well. Multiply Lauren out a few thousand or million times. There are a few possible solutions: 1. Buy business-grade Internet. It's expensive, but it's not residential, and you may be able to run services on it. Part of what you're paying for is for the service provider to call you up to bitch about shit flowing over your pipe that shouldn't be. 2. Set up specific understandings with the services you peer with. If it's just you and a few buddies, great. If you're trying to get email through to Google or a self-hosting F-50 company, good luck. They've got policies and shit and take ages. (Google may be fairly responsive, F-50 companies, in my experience less so.) 3. Build (or help build) something better. SMTP email has been with us a long time, has some useful features, and has served us well. But it's also got some tremendous shortcomings. An alternative with robust encryption, anti-spam features, poll-and-fetch (rather than unsolicited-send), better multiplatform support, and the ability to dispense with overly-formatted email (I still prefer console mailers), might be a good start. Being fully peer-to-peer even better. That gets complicated though.