4 ms·

Heh recently I wondered why we don't use SMTP to send for example telemetry data around. It has everything for reliable message delivery. Instead I went with MQ

Heh recently I wondered why we don't use SMTP to send for example telemetry data around. It has everything for reliable message delivery. Instead I went with MQTT for the latest project that collected telemetry data and shoved it into a database, out of fear the next admin would come to hunt me down for using SMTP for a thing like this.

messe 4y ago
> out of fear the next admin would come to hunt me down for using SMTP for a thing like this. Why would they need to hunt you down? If you’re actually implementing SMTP sensibly, they could reach you via the Reply-To address.
theamk 4y ago
SMTP will be fine if you just use a protocol and set up all new parallel infra. Overhead will be quite a bit higher but that's not a problem in modern world. But sharing the existing email infra is pretty dangerous. You never know when spam filters decide to filter your messages and turn your "reliable" message delivery into black hole.
st_goliath 4y ago
> Instead I went with MQTT for the latest project that collected telemetry data ... If you want broadcast-message-to-topic-subscribers style semantics, you could have [ab]used IRC for this, rather than SMTP :-P
- ReactiveJelly 4y ago
  Pretty unreliable [1] though. I'd sooner build some custom HTTP thing in front of Postgres or SQLite than try to do resending over IRC, a protocol which badly suffers from a fuzzy separation between payload and protocol. [1] If you lose connection, neither side is sure which messages they missed, and replaying is not part of the protocol. If you're gonna support a memo bot, why not build it in?
soneil 4y ago
I used to work with a platform that used smtp for telemetry (ish, event-based rather than stream-based). I'm not sure I'd ever do it again. It was essentially email alerting, but I consider it a close equivalent because the messages were sent machine-to-machine and (hopefully) never surfaced to a human, and they were sent realtime with no rate limiting (sent realtime, where reception is anything but). The single largest issue we had was that customers would use their own email address as the Sender as it was the easiest way to have it accepted by their own network. And then if anything went wrong with our MX, customers would get thousands of bounces back to their address. But the cause of a lot of issues is that mail does not have reliable delivery. The sending device cannot promise the email was delivered, only that its session with an smtp server ended without fault. It can't promise the message was forwarded or delivered (I have seen mailservers accept emails and disappear them), It can't interrogate the chain of custody from end to end. Receiving email is a near-unrelated second path that rarely exists in embedded applications, so the bounce/feedback mechanism is rarely a closed loop in these applications.
- RedShift1 4y ago
  The way I envisioned is that the entire thing would be separate from the regular public email system. The collecting device would first deliver it to a local mailserver, which then in turn delivers it to our mailserver which only accepts TLS connections with a client certificate signed by us.

messe 4y ago

> out of fear the next admin would come to hunt me down for using SMTP for a thing like this. Why would they need to hunt you down? If you’re actually implementing SMTP sensibly, they could reach you via the Reply-To address.

theamk 4y ago

SMTP will be fine if you just use a protocol and set up all new parallel infra. Overhead will be quite a bit higher but that's not a problem in modern world. But sharing the existing email infra is pretty dangerous. You never know when spam filters decide to filter your messages and turn your "reliable" message delivery into black hole.

st_goliath 4y ago

> Instead I went with MQTT for the latest project that collected telemetry data ... If you want broadcast-message-to-topic-subscribers style semantics, you could have [ab]used IRC for this, rather than SMTP :-P

ReactiveJelly 4y ago
Pretty unreliable [1] though. I'd sooner build some custom HTTP thing in front of Postgres or SQLite than try to do resending over IRC, a protocol which badly suffers from a fuzzy separation between payload and protocol. [1] If you lose connection, neither side is sure which messages they missed, and replaying is not part of the protocol. If you're gonna support a memo bot, why not build it in?

soneil 4y ago

I used to work with a platform that used smtp for telemetry (ish, event-based rather than stream-based). I'm not sure I'd ever do it again. It was essentially email alerting, but I consider it a close equivalent because the messages were sent machine-to-machine and (hopefully) never surfaced to a human, and they were sent realtime with no rate limiting (sent realtime, where reception is anything but). The single largest issue we had was that customers would use their own email address as the Sender as it was the easiest way to have it accepted by their own network. And then if anything went wrong with our MX, customers would get thousands of bounces back to their address. But the cause of a lot of issues is that mail does not have reliable delivery. The sending device cannot promise the email was delivered, only that its session with an smtp server ended without fault. It can't promise the message was forwarded or delivered (I have seen mailservers accept emails and disappear them), It can't interrogate the chain of custody from end to end. Receiving email is a near-unrelated second path that rarely exists in embedded applications, so the bounce/feedback mechanism is rarely a closed loop in these applications.

RedShift1 4y ago
The way I envisioned is that the entire thing would be separate from the regular public email system. The collecting device would first deliver it to a local mailserver, which then in turn delivers it to our mailserver which only accepts TLS connections with a client certificate signed by us.