4 ms·
I would advise reconsidering keeping the current format exactly. Something that would allow conventional email addresses to kept in a list that disambiguates th
by jerf 2mo ago
I would advise reconsidering keeping the current format exactly. Something that would allow conventional email addresses to kept in a list that disambiguates them unambiguously would probably be a good idea.
You don't want to embed the entire email in JSON. Too many parsers past, present, and future tend to want to decode the entire JSON document into memory before doing anything else, and so you'd be forcing entire email documents to be held in memory at scale, which causes you major problems. I'd recommend something more like either a JSON header that defines what parts are in the email, followed by a normal MIME document, which is perfectly normal in HTTP with other things that use the format as well, or having the top level continue to be a MIME document and specify that the first part must be a JSON document containing the headers. JSON replacing the headers is generally something that would be a good idea, though.
That improves the ability of more language environments to be able to handle the email as a stream or in chunks with "normal" libraries and practices. MIME parsers in languages that tend to favor pulling everything into RAM as a string are still more likely to have been forced to face this issue already.
- zahlman 2mo ago> You don't want to embed the entire email in JSON. Too many parsers past, present, and future tend to want to decode the entire JSON document into memory before doing anything else, and so you'd be forcing entire email documents to be held in memory at scale Why? Even if your system loads emails into memory a whole email at a time, that doesn't imply that it will load large numbers of emails simultaneously. The kind of processing that can be done by stream-processing large numbers of emails in parallel seems rather limited to me. > I'd recommend something more like either a JSON header that defines what parts are in the email, followed by a normal MIME document, which is perfectly normal in HTTP with other things that use the format as well, or having the top level continue to be a MIME document and specify that the first part must be a JSON document containing the headers. JSON replacing the headers is generally something that would be a good idea, though. If we're specifically talking about being able to short-circuit after headers, then why not a JSON header followed by JSON representation of the body?
- jerf 2mo ago"Even if your system loads emails into memory a whole email at a time, that doesn't imply that it will load large numbers of emails simultaneously. The" When thinking about a standard like this, thinking about "your" system is a fatal error. You have to think about all the systems. And, yes, there are systems out there in the world that load large numbers of emails simultaneously. Obviously. Many of them. Not only that, if a new standard can't capture some of those, it's going to be a worthless niche standard forever because those are precisely the nodes that you need to get to support the new standard. "If we're specifically talking about being able to short-circuit after headers, then why not a JSON header followed by JSON representation of the body?" Just because a JSON representation of the body is pretty worthless. MIME is an extremely well-known standard, already used in HTTP contexts, and it will be more efficient than JSON string encoding at a fairly small transition size.
- inigyou 2mo agoEmail already has the problem of complete buffering. That's why various services limit you to 10MB or 25MB.