4 ms·
Having written trading applications with binary, JSON, and FIX protocols: this article is fucking terrible. 1) scanf on a floating point number works different
by CoolGuySteve 8y ago
Having written trading applications with binary, JSON, and FIX protocols: this article is fucking terrible.
1) scanf on a floating point number works differently on different platforms. If you explore the space of numbers that are expressible, you will find very different results in terms of floating point error depending on which implementation you use.
Therefore transmitting floating point in ascii is just asking for trouble. But IEE-754 is fairly universal and binary fixed point conversion will look the same everywhere.
2) The complaint about not being able to locate fields based on a text dump is okayish. But it's easily solvable by adding a 4 char code to specify the message type. For example, if you open a video file in a hex editor, you'll see headers like 'MPEG' all over.
3) In high performance or low memory applications, translating text to binary and back is expensive. But writing binary as text for debugging is just a printf away.
4) Auxillary to 3, allocating memory for streams becomes an O(logn) problem in terms of allocations since you don't know where the current message of arbitrary length will end. Meanwhile, binary messages over either UDP or TCP are fixed length and atomic, vastly simplifying your streaming and event code.
The worst case of this I ever saw was a text RPC protocol for video streaming, where a single message could be anywhere between 20 bytes and several GB. The guy that wrote that one made the same argument ESR makes, "It's so easy to read!".
- jcranmer 8y ago> The worst case of this I ever saw was a text RPC protocol for video streaming, where a single message could be anywhere between 20 bytes and several GB. The guy that wrote that one made the same argument ESR makes, "It's so easy to read!". That reminds me of the Mork file format. Apparently, the edict from management was that the database file format should be space-efficient and human readable. This is to store essentially an entity-attribute-value relationship. The resulting file looks like this: (^a0^23fca)(^a1^23fcb)(^a2^23fcc)... totally unreadable. And it's not particularly efficient as a file format either as a result.
- jcrawfordor 8y agoHe also repeats that binary protocols have the downside that you need to "have the spec in front of you" to interpret them... but network traffic is virtually never inspected by humans without the assistance of a protocol dissector that will add the descriptions, parse numbers, and so on for you. There is clearly an efficiency downside to packing this description into the messages themselves when the recipient software will necessarily have the protocol information necessary to decode it.
- basementcat 8y agoUnfortunately the binary representation of IEEE-754 may not look the same everywhere. https://en.wikipedia.org/wiki/Double-precision_floating-point_format#Endianness https://en.wikipedia.org/wiki/Double-precision_floating-poin... https://en.wikipedia.org/wiki/IEEE_754#Reproducibility https://en.wikipedia.org/wiki/IEEE_754#Reproducibility
- outworlder 8y agoNot terrible. I have learned some things about how the NTP protocol was designed. And it is a good summary of trade-offs involved. You may or may not agree with them, and they may or may not be important to you. However, I have learned even more things about how absolutely mind-bogglingly huge the egos of some posters here are.
- DonHopkins 8y agoFunny that you should mention mind-bogglingly huge egos in a discussion about ESR: > “Are you” she asked “the most famous programmer in the world?” > This was a question which I had, believe it or not, never thought about before. But it’s a reasonable one to ask... http://esr.ibiblio.org/?p=6839 http://esr.ibiblio.org/?p=6839
- leoc 8y agoWhat's the relevance of this? It's certainly not the case that the presence of one person with an embarrassingly obvious craving for adulation in the conversation means that other people with distended egos of their own will stay away. The opposite is nearer true.
- DonHopkins 8y agoIf you're so sure that the grandparent comment is relevant, then please tell us exactly which messages you agree are from people with absolutely mind-bogglingly huge egos even larger than Eric S Raymond's? I haven't noticed anyone making any appeals to their own authority as arrogant as claiming to be the most famous programmer in the world. Are you saying that it takes a huge ego to point out ESR's long track record of making flawed arguments, and quote the many patently false, self-aggrandizing, racist and sexist things he's said in the past, or that there's something wrong with pointing out racism instead of silently condoning it?
- AgentME 8y agoYeah I'm really surprised this article is getting as much shit as it is. There's details it misses, but it seems like a good enough intro to some of the trade-offs between fixed-size messages and self-describing ones, and designing for extensibility.