3 ms·

Using the x86-64 C struct layout as the serialization format is increasingly common in financial trading protocols. Skipping all the bit twiddling has a huge pe

Using the x86-64 C struct layout as the serialization format is increasingly common in financial trading protocols. Skipping all the bit twiddling has a huge performance impact at high message rates. I once compared two feeds covering the same basic data, where the major difference was one had a complicated serialization and one was C struct layout. The former needed two 16 core servers just to process I/O and deserialize the messages; the latter used 1 core on a single server.

usefulcat 9y ago
"The former needed two 16 core servers just to process I/O and deserialize the messages; the latter used 1 core on a single server." I'm genuinely curious; what was the 'former' protocol? Was it encoded using FAST or zlib or something?
- twic 9y ago
  I would guess FIX/FAST. For a concrete example, have a read of the specifications for Eurex's market data protocols: http://www.eurexchange.com/exchange-en/technology/t7/system-documentation/release6/Release-6.0/3088824?frag=3088828 http://www.eurexchange.com/exchange-en/technology/t7/system-... Specifically, compare the Enhanced Market Data Interface protocol (described in "T7 Market and Reference Data Interfaces") and the Enhanced Order Book Interface protocol. EMDI is FAST, a morass of tags, stop bits, presence bitmaps, and who knows what else (i don't). EOBI is structs.
- scott00 9y ago
  I was consuming the feed in question via an API provided by the vendor and didn't have direct knowledge of the wire protocol, but based on how it was described to me during the sales process it was probably either FAST or a proprietary protocol with a similar design.
yuhong 9y ago
Which also happens to be why we used binary document formats first then moved to XML too.
ycombobreaker 9y ago
Regarding bring lazy on the serialization... yeah packed structures make sense. But excess padding is still wasteful, and endianness is only a bswap away, which is 1-2 cycles apiece and so won't exactly break the latency budget (per Agner Fog's wonderful pdf, looking at Haswell).
- jnordwick 9y ago
  It might only take a cycle to swab, but the big hit is probably the data dependency you introduce.

usefulcat 9y ago

"The former needed two 16 core servers just to process I/O and deserialize the messages; the latter used 1 core on a single server." I'm genuinely curious; what was the 'former' protocol? Was it encoded using FAST or zlib or something?

twic 9y ago
I would guess FIX/FAST. For a concrete example, have a read of the specifications for Eurex's market data protocols: http://www.eurexchange.com/exchange-en/technology/t7/system-documentation/release6/Release-6.0/3088824?frag=3088828 http://www.eurexchange.com/exchange-en/technology/t7/system-... Specifically, compare the Enhanced Market Data Interface protocol (described in "T7 Market and Reference Data Interfaces") and the Enhanced Order Book Interface protocol. EMDI is FAST, a morass of tags, stop bits, presence bitmaps, and who knows what else (i don't). EOBI is structs.
scott00 9y ago
I was consuming the feed in question via an API provided by the vendor and didn't have direct knowledge of the wire protocol, but based on how it was described to me during the sales process it was probably either FAST or a proprietary protocol with a similar design.

yuhong 9y ago

Which also happens to be why we used binary document formats first then moved to XML too.

ycombobreaker 9y ago

Regarding bring lazy on the serialization... yeah packed structures make sense. But excess padding is still wasteful, and endianness is only a bswap away, which is 1-2 cycles apiece and so won't exactly break the latency budget (per Agner Fog's wonderful pdf, looking at Haswell).

jnordwick 9y ago
It might only take a cycle to swab, but the big hit is probably the data dependency you introduce.