5 ms·
> binary protocol Email was never a binary protocol. Notoriously so, it's why MIME types and MIME encodings get so complicated. Most of the "old internet" pro
by WorldMaker 1y ago
> binary protocol
Email was never a binary protocol. Notoriously so, it's why MIME types and MIME encodings get so complicated.
Most of the "old internet" protocols (email, FTP, even HTTP itself) were bootstrapped on top of built-mostly-for-plaintext Telnet. HTTP as the new telnet has a bunch of improvements when it comes to binary data, request/response-based data flows, and some other considerations. HTTP/3 is even inherently a binary protocol, it's lack of "telnet-compatibility" one of the concerns about switching the majority of the web to it.
vCard/vCal/iCard/iCal were also deeply "plaintext formats". JSON is an improvement because it is more structured, even more efficient, than those predecessors. JSON may not look efficient, but it compresses extremely well and can be quite efficient in gzip and Brotli streams.
I feel like "JSON over HTTP" is a subtle improvement over "custom text formats over telnet", even if it doesn't sound like "binary protocol efficiency" at first glance. Especially as HTTP/3 pushes HTTP more efficient and more "binary", and arguably "more fundamental/basic" with HTTP/3 even taking over more roles in the TCP/UDP layer of the internet stack. (Telnet would never try to replace TCP.) HTTP isn't the worst bootstrap layer the internet could use to build new protocols and apps on top of. Sure, it would be neat to see more variety and experiments outside of the HTTP stack, too, but HTTP is too useful at this point not to build a bunch of things on top of it instead of as their own from-scratch protocol.
- p_l 1y agoA lot of the textual nature of older IETF protocols, including the CR LF line endigns, can be probably traced to how easy it was to bang out a bad implementation full of subtle problems that could be debugged by sitting an undergrad student at a teletype instead of spending time on having some binary serializer (that telecom companies definitely had money for)
- JoshTriplett 1y agoYeah, a fair bit of email protocol reeks of "is this tolerant of `telnet mailserver 25` and whatever garbage that might produce".
- p_l 1y agoA lot of old RFCs explicitly mention running on top of TELNET. Additionally, as much people like to harp about "telcos focusing on connection-oriented protocols while we ran loops around them with packets", the reality is that NCP and later TCP pretty much focused on emulating serial lines around, and one of the earliest ways to access ARPAnet outside of machines directly on it was through calling into a TIP which set up bidirectional stream from your modem to a port on some host.
- tbrownaw 1y ago> Additionally, as much people like to harp about "telcos focusing on connection-oriented protocols while we ran loops around them with packets", the reality is that NCP and later TCP pretty much focused on emulating serial lines around, and The idea with packets is that you don't need to reserve N bit/s of each link along the route to whatever system you're talking to; instead you just repeatedly say "here's a chunk of data, send it to X". It's not really relevant that the typical thing to do with these packets is to build a reliable stream on top of them, what matters is that everything except the endpoints can be a lot dumber.
- miki123211 1y agoThere's something in-between traditional (IP style) packets and traditional (TDM / ISDN style) circuit switching, where you use packets with no bandwidth guarantees, but address them by connection number, not endpoint address. This still requires you to set up a connection beforehand, but doesn't require you to reserve resources you might not be using.
- p_l 1y agoAnd while certain telcos had packet network focused on that (Bell and Datakit, for example), and generally OSI cared about including QoS indicators, CLNS and its specific on-the-wire implementation CLNP is so similar to IP that IPv9 explicitly called it out in the proposal to replace IPv4 with it (because of ISO 160bit addressing)
- cyberax 1y agoIt's also a reflection of the state-of-the-art at the time. Binary protocols were an unmitigated disaster, with standards bodies thinking that applying to the ISO for your organizational ID is a perfectly fine step that anyone does anyway.
- p_l 1y agoThat's why there was an entire section for unassigned numbers. Binary protocols just meant you actually needed to implement serialiser/deserialiser and similar tooling instead of writing dumbest possible riff on strtok() and hoping your software won't be used anymore once DoD internet becomes mature
- cyberax 1y ago> That's why there was an entire section for unassigned numbers. That's also why the majority of OIDs in SNMP are rooted in the 1.3.6 hierarchy, which belongs to the DoD.
- p_l 1y agoSNMP OIDs are in 1.3.6 because TCP/IP stack was once more well known as "DoD Internet". And SNMP is explicitly a DoD Internet simplified alternative to CMIS
- zzo38computer 1y ago> with standards bodies thinking that applying to the ISO for your organizational ID is a perfectly fine step that anyone does anyway Fortunately there is the 2.25 OID arc now, which you can use without any registration with anyone. There are also other ways to register OIDs for free. (I think that it is better than using domain names, which can be reassigned, and also require registration anyways. IDN is an even more severe problem (it could have been designed better, but they made it worse instead).) I had idea (which would have to later be made standardized by ITU or ISO (preferably ITU)) of a new OID arc which allows you to combine an existing identifier (of many different types, such as: international telephone numbers, amateur radio call signs, internet domain names (encoded as bijective base 37), IP addresses, ICAO airport codes, etc) with a timestamp, and optional auto-delegation. (You can then add additional numbers like you can with other OIDs too) Binary protocols have other benefits as well, such as not requiring escaping, and allowing binary data to be transferred is a way that is not as messy, not causing problems with character sets, etc.
- p_l 1y agoAnother point is that the use of HTTP for everything, outside of the issue of middle boxes breaking protocols for everyone, is that it's essentially capitulation to the wisdom of OSI multi-layered protocols - we replicate their feature sets by reusing bits and pieces of HTTP spec all the time.
- frumplestlatz 1y agoHTTP is also a large, complex stack for any server/client that isn’t already a web server or a browser.
- zzo38computer 1y agoYes, this is the other issue with using HTTP.
- 9dev 1y agoIt’s not that I cannot appreciate the improvements in the space, I’m just wondering if there might be a big part of the design space for widely used protocols that ends up unexplored because the default for almost anything now is HTTP. It has basically become OSI layer 8 at this point.
- p_l 1y agoMIME/Content-Negotiation is essentially OSI Presentation Layer HTTP sorta acts as stump of ROSE with bit of ACSE. In addition it provides a bit of basic layer for passing some extra attributes that might be considered in-band or out (or side?) band to the actual exchange.
- WorldMaker 1y agoHTTP/3 replaces parts of OSI layer 4 (TCP/UDP). There's a perspective that HTTP has been trying to lower its layer for decades. Especially the way that TLS originated as HTTPS but now is almost a "universal" vertical chunk of Layers 4, 5, and 6. In that perspective HTTP is probably the new Layer 6.
- jeroenhd 1y agoHTTP/3 uses QUIC as a layer 4 equivalent (which is actually just UDP with extra implications in practice). Not many services leverage standards-compliant QUIC other than HTTP/3, but you don't need to do HTTP to get the same protocol working. If anything, HTTP/3 running on top of QUIC forced shitty middlebox vendors to de-ossify by permitting any QUIC-based protocol, as they cannot practically distinguish a new HTTP/3 connection from a QUIC connection.
- afiori 1y agoMy understanding is that they cannot even really recognise a QUIC connection (maybe the first time, but not after ip roaming)
- thequux 1y agoEhh, not really. OSI layer 5 was responsible for managing multiple non-overlapping sessions within a single transport stream, and routing those sessions to specific applications. This is precisely what HTTP/1.1 did (though the accept, content-type, accept-encoding, and transfer-encoding headers are really an implementation of layer 6); QUIC, on the other hand, covers the same layers that TCP does (3-5) plus the aforementioned layer 6. I recommend actually reading X.200 (the specification of the OSI model) at some point: it's quite approachable (especially for an ITU spec, which are notoriously dense reading), and will quickly make you realize how silly it is that we still use it as a reference for modern stacks.
- zzo38computer 1y ago> Notoriously so, it's why MIME types and MIME encodings get so complicated. I made up ULFI because I thought MIME has some problems. > JSON may not look efficient Efficiency is not the only issue; there is also the consideration of e.g. what data types you want to use. JSON does not have a proper integer type, does not have a proper binary data type (you must encode it as hex or base64 instead), and is limited about what character sets can be used. (Also, like other text formats, escaping will be needed.) > I feel like "JSON over HTTP" is a subtle improvement over "custom text formats over telnet" I think it can be, depending on the specific use; sometimes it isn't, and will make it worse. (HTTP does have the advantage of having URLs and virtual hosting, although I think it adds too much complexity more than should be needed.) However, I still think that DER is generally better than JSON. > HTTP isn't the worst bootstrap layer the internet could use to build new protocols and apps on top of. I think it depends on the specific application. However, even then, I think there are better ways than using HTTP with the complexity that it involves, most of which should not be necessary (even though a few parts are helpful, such as virtual hosting).
- dotancohen 1y ago> JSON does not have a proper integer type What are the drawbacks to using the JavaScript Number (really a double float I think) datatype as an integer in an object representation language such as JSON? I've never seen a use case where e.g. 42 (int) could be confused with 42.0 (float). If your application needs specifically an int or a float, then the ingesting application knows that. If the answer is monetary values, then those should never be floats, and should not be represented in JSON as such. E.g. a dollar and a half should be represented as 150 cents. This follows even for sub-cent precision.
- zzo38computer 1y agoI don't mean 32-bit integers, but rather, 64-bit (and sometimes bigger) integers. JSON already works OK for 32-bit integers (although it is not ideal).
- RedShift1 1y agoMuch easier would just be native decimal support in JS and we can represent a decimal using d as the decimal point, for example 15d0 (15.0) or 384d25 (384.25). Using cents instead of dollars sounds fine until you have to do math like VAT, you really need decimal math for that.
- 7bit 1y ago> Most of the "old internet" protocols (email, FTP, even HTTP itself) were bootstrapped on top of built-mostly-for-plaintext Telnet. That's just not true. Telnet and SMTP are built on top of TCP. They live on the same layer. They were originally both protocols that transmitted data with printable ascii, hence why they look similar. There are many other protocols like Telnet and SMTP that worked like that, auch as nntp, irc, and yes, even http.
- 20after4 1y agoI think that the point is this: You could literally use a telnet client to talk to all of those servers and it was considered a good thing, maybe even an essential feature. So protocol design was somehow influenced by the need to be fully plain text ASCII streams (and maybe that other options weren't adequately explored due to this restriction)
- dhosek 1y agoYou can also telnet into port 80 and communicate with an HTTP server.
- WorldMaker 1y agoUnless it is HTTP/3.