4 ms·
> A great starting point is FIX. I beg to differ. FIX could almost be a case study on how not to design a protocol. Session management and recovery is outside
by amluto 5y ago
> A great starting point is FIX.
I beg to differ. FIX could almost be a case study on how not to design a protocol.
Session management and recovery is outside the login protocol? Check! Can’t reliably recover from a failed login or wrong password attempt? Check! [0] Message boundaries are nontrivial to detect? Check! Field values are binary but cannot represent the byte 0x01? Check!
But FIX has a brilliant extensible hierarchical message structure that is more type safe than JSON, you say! Sure, kind of, but the encoding of lists of nested objects is utter nonsense. Not only is it somewhat redundant (the number of repeats is sent first, but the number of repeats is also determinable after parsing the whole thing, resulting in some awkward questions about what to do if the numbers disagree), but it has worse problems. Specifically, if the sender and receiver don’t agree on the precise set of fields and their order in each nested object, then the list cannot be parsed. This makes the whole concept of extensibility someone dubious. (This is because there is no delimiter between repeated objects; instead you notice that the field you’re parsing can’t legally belong to the object you’re parsing, meaning it’s time to use a crystal ball to determine whether a new repeat has started, the whole list is done, or an error has occurred.)
On top of all this, the standard implementation, QuickFIX, is a work of art, and not in a good way.
FIX SBE is IMO not at all better except that it can be serialized and deserialized faster.
[0]. No, really. If you try to log in to someone else’s session and the server rejects the password, you have just fried their session. The protocol cannot recover. Sometimes I wonder if this was deliberate, because screwing this up at the protocol level takes some creativity. Imagine if trying to log in to someone else’s email account caused their next login attempt to fail! Oh wait, some password attempt policies do this, but FIX does it by corrupting the whole protocol state machine rather than merely having a misguided policy.
- exdsq 5y agoThat login stuff sounds… painful. So you’re saying as UserA if I try to login as UserB, UserBs login is corrupted and they have to relogin?
- amluto 5y agoWorse. For a “session” (connection from a client to a server with corresponding IDs — the IDs serve a purpose somewhere between a username and a TCP port), there is some state that is used to recover potentially lost messages if the session drops and reconnects. If a session is down, someone tries to connect and restore that session, and the login fails, then the replay state will get out of sync on both ends and a subsequent valid reconnection will either fail or lose messages.
- dahfizz 5y agoFIX is less ergonomic than JSON, that is not in question. FIX is much faster to generate and parse than JSON, and much easier to deal with than bespoke binary feeds. I think FIX makes the right tradeoffs. I don't understand what you mean regarding killing someone else's session. You can reset sequence numbers in the login message.
- amluto 5y agoThere is absolutely no reason that FIX should be faster to generate than JSON. I agree that one could plausibly parse it faster. JSON wasn’t meant as a real comparison though — FIX is structurally much more like protocol buffers, but protocol buffers are much better. As for sequence numbers: suppose your session drops. You’ve received the message with sequence number 5, but you’ve missed 6 and 7. You’ve sent 10 and 11 but the other end only got 10. Now someone (your own misconfigured software or someone else) connects and sends Logon to the server with your comp id, with sequence number 11 but the wrong authentication data. What is even supposed to happen? The session now contains two different message 11s to the server. It’s broken. You can (sometimes) recover by resetting sequence numbers, but that defeats the entire purpose of having sequence numbers in the first place: the server was supposed to recover the original message 11 on a reconnection attempt and correctly handle it as a possible duplicate. (As a practical matter merely experiencing network trouble during a reconnection also seems to break everything, mostly because common FIX implementations are terrible.) I suppose it must be possible to be so deeply entrenched in the FIX ecosystem that this makes any sense at all, but the whole protocol is backwards. You should connect, authenticate that connection, and then run the session protocol inside the authenticated connection. The Logon should not be sequenced — that gets the layering backwards.
- amluto 5y agoAnd replying further: this isn’t about whether FIX makes the right trade offs. This is about the fact that, even in the position in design space that FIX occupies, FIX is terrible. One could have easily designed a protocol with the same level of security as FIX and precisely the same (application) message semantics and schema that actually worked well. It wouldn’t even be very different from FIX. All you would need is proper layering, group delimiters, delimiters for entire messages, and either proper escaping or length-prefixing of fields. (And length-prefixing would make parsing quite a bit faster without needing to resort to vectorized searches for where fields and groups end.)