5 ms·

Fetchmail grew to be a mail transport swiss army knife and encompassed half baked functionality of a domain-wide MTA rather focusing on single user async email

Fetchmail grew to be a mail transport swiss army knife and encompassed half baked functionality of a domain-wide MTA rather focusing on single user async email consolidation and collection for a logical single account, which is largely what this rant is about. Half of the half-bakedness is due to lack of MTA protocol support for MUA interactions on the receiving side. pop3 was never meant as a spool for MTA to MTA email transfers, but was instead meant for an MUA to access. It's not fetchmail's fault that the envelope TO isn't available via pop3. That, at the time, email service providers bolted on entire domain delivery to a single mailbox, and then users of those services put multiple recipients behind that single mailbox. Fetchmail made this work as best it could. Also missing at the time was any kind of serious connecting-client-as-MTA or MTA-to-MTA authentication (rather, relying on DNS and TCP!), which might have made async, receiver controlled MTA initiation more workable. This was also due to common SMTP MTAs not having good (or at least easy/obvious to configure) store-for-later-pickup functionality where an MTA could query for email destined for itself rather than be always-on connected to the internet waiting for another MTA to initiate delivery. Fetchmail filled a hole that existed in email delivery at the time, but it was really a hole that should have never existed. The quality of the fetchmail code is another, albeit serious, issue in itself. These days we have a better understanding of the differences between MTAs, MSAs, and MUAs.

ggm 6y ago
These days we have a better understanding of the differences between MTAs, MSAs, and MUAs. I don't want to be "that guy" but we've known about this distinction since the 1980s. I know, because I was fringing in the discussions in standards ml during the x.400 years.
- 0xbadcafebee 6y ago
  I actually learned the difference between MTAs, MUAs, etc because of fetchmail. Fetchmail kept randomly crashing on select spam messages and my bug reports went unheeded (and I wasn't what anyone would call a programmer) so I needed a new way to download my mail. So I wrote a crappy Perl script. All it does is use standard Perl modules to download the mail, and pipes it off to an MDA (postfix, maildrop, etc) to deal with them. After nearly two decades, no matter what system I plunk it on or mail provider I use, the same stupid script just keeps working.
- thwarted 6y ago
  You are "that guy". I didn't say we didn't know about this distinction, I said we have a better understanding. It used to be the case submitting originating email on port 25 was accepted. Now, if you have that configured, it's wildly considered that You're Doing It Wrong™. Many email clients, like Eudora and Thunderbird, were submitting email on port 25 for a long time, in the clear, where the only protection was very loose source IP address restrictions for outbound, otherwise you'd be an open relay. Using different servers for inbound vs outbound mail submission was a relatively rare thing too, because the routing can be a bear to set up. SMTPS (SMTP over SSL) on port 465 wasn't official and was locally used where it could be supported (by the client) and where an admin had bothered to configured it. Many email clients didn't have support for even using a non-standard port for any thing mail related, so even once the mail submission port 587 was standardized it was often a problem getting a client to use it or providing authentication on it. But even the OP is saying that fetchmail is a problem because it's trying to be an a full, transparent MTA when it's really a glorified MUA. The fact that fetchmail (and its ilk) talk MUA protocols to receive email (pop3, imap) and can then turn around and reinject that message into the mail stream using SMTP to implement local delivery is where a lot of the problems with fetchmail come from (such as munging the headers or creating email loops with mailing lists).
  - ggm 6y ago
    Quite a lot of the anti x400 vituperation was about complexity. People didn't want to think about the distinctions and where open 25 was possible, with 110, the barrier to entry was low. Setting up a proper mta/mua/msa respecting service was hard, and for many people remains hard. Exploring how to get utf8 clean across all the m* protocols for lhs@rhs reminds me how complex this can be.

ggm 6y ago

These days we have a better understanding of the differences between MTAs, MSAs, and MUAs. I don't want to be "that guy" but we've known about this distinction since the 1980s. I know, because I was fringing in the discussions in standards ml during the x.400 years.

0xbadcafebee 6y ago
I actually learned the difference between MTAs, MUAs, etc because of fetchmail. Fetchmail kept randomly crashing on select spam messages and my bug reports went unheeded (and I wasn't what anyone would call a programmer) so I needed a new way to download my mail. So I wrote a crappy Perl script. All it does is use standard Perl modules to download the mail, and pipes it off to an MDA (postfix, maildrop, etc) to deal with them. After nearly two decades, no matter what system I plunk it on or mail provider I use, the same stupid script just keeps working.
thwarted 6y ago
You are "that guy". I didn't say we didn't know about this distinction, I said we have a better understanding. It used to be the case submitting originating email on port 25 was accepted. Now, if you have that configured, it's wildly considered that You're Doing It Wrong™. Many email clients, like Eudora and Thunderbird, were submitting email on port 25 for a long time, in the clear, where the only protection was very loose source IP address restrictions for outbound, otherwise you'd be an open relay. Using different servers for inbound vs outbound mail submission was a relatively rare thing too, because the routing can be a bear to set up. SMTPS (SMTP over SSL) on port 465 wasn't official and was locally used where it could be supported (by the client) and where an admin had bothered to configured it. Many email clients didn't have support for even using a non-standard port for any thing mail related, so even once the mail submission port 587 was standardized it was often a problem getting a client to use it or providing authentication on it. But even the OP is saying that fetchmail is a problem because it's trying to be an a full, transparent MTA when it's really a glorified MUA. The fact that fetchmail (and its ilk) talk MUA protocols to receive email (pop3, imap) and can then turn around and reinject that message into the mail stream using SMTP to implement local delivery is where a lot of the problems with fetchmail come from (such as munging the headers or creating email loops with mailing lists).
- ggm 6y ago
  Quite a lot of the anti x400 vituperation was about complexity. People didn't want to think about the distinctions and where open 25 was possible, with 110, the barrier to entry was low. Setting up a proper mta/mua/msa respecting service was hard, and for many people remains hard. Exploring how to get utf8 clean across all the m* protocols for lhs@rhs reminds me how complex this can be.