4 ms·

I started reading this thinking it would be about FIX tag/value. Now I’m more than a little shocked multiple someones are publishing real time data in JSON for

I started reading this thinking it would be about FIX tag/value. Now I’m more than a little shocked multiple someones are publishing real time data in JSON for algos?? I thought this was mostly a solved problem with OUCH, ITCH, FIX SBE and friends. But great to see this writeup, getting these details right can make a big difference.

Bootvis 5y ago
JSON is common in crypto what Cantor appears to be trading.
- gfd 5y ago
  Does crypto allow colocation like stock trading? If it's gonna be some websocket endpoint, what's the point of optimizing microseconds?
  - lordnacho 5y ago
    This is a good point. You expect more jitter on a public network. My guess is they've come from traditional finance, where performance porn is just irresistible. Some exchanges do offer colo.
  - Bootvis 5y ago
    It kind of does. Most exchanges are hosted in the cloud and I don't think AWS offers colocation similar to the traditional exchanges. I bet Cantor ensures that the machines they are using are close to those used by the exchanges. HFT on a traditional exchange will be faster but that's not the competition. The competition in crypto faces the same problems so you just need to be faster than them. Of course, if the whole process has too much uncontrollable noise (jitter) due to cloud specific reasons it probably doesn't matter. I hope they managed to control this before doing this optimization :)
  - ajoseps 5y ago
    one approach I've heard of some places using is to figure out where the exchange is hosted, spin up multiple instances to test the latency to the exchange, then choose the lowest latency instance. Not sure how often one would need to redo this process though.
  - ajoseps 5y ago
    one exchange that does allow colo is gemini. They're located in NY5 (https://docs.gemini.com/fix-api/#connecting https://docs.gemini.com/fix-api/#connecting) and allow cross connects. When colo-ed, you wouldn't be using websocket but their FIX API. Multiple other crypto providers are also colo-ed in the same location
  - nly 5y ago
    Lots of prop-trading firms aren't comfortable trading over the Internet. Often the concern isn't "someone might see my trades", since reverse engineering alpha from that is hard, or even latency, since not all strategies are super latency sensitive, ... but plain old infosec. These sorts of firms are super paranoid about the exfiltration of proprietary IP. When you add trading over the Internet in to the mix, you just have more ways to do that. You have highly dynamic cloud infrastructure to deal with (whereas traditionally everything is static), DNS security, tricky PKI/certificate management, the need for much tighter access controls, firewalling (both at the network and application protocol level) and the secure logging and monitoring of flow between networks. And everything needs to be tight enough so no single actor (external or internal) can compromise the system.
exdsq 5y ago
Where can I read more about those formats? I work in crypto, same as cantor trading that posted this, and always keen on re-using solved problems if they work :)
- andylynch 5y ago
  A great starting point is FIX. They are the open standards group in this area, at https://www.fixtrading.org/ https://www.fixtrading.org/. ITCH and OUCH came from Island, now part of NASDAQ and the specs are on their website.
  - exdsq 5y ago
    Thanks! I saw they have a working group on crypto so applied to join that, keen to see what they're up to. Standardizing data structures across chains makes a lot of sense but sadly there's a lot of competition rather than collaboration between them "Mine is better than yours hahahaha moon" etc... I'll have to give this some real thought! I think one issue may be that the specification is so complex implementing it within a new DEX without buy-in from others means it's a lot of work with little reward. Might be the case that if larger established front-offices/exchanges start expecting it that crypto could follow. Coinbase released an API standard that blockchains had to conform to for testing the chain prior to listing so it's not unprecedented for that to happen.
    - ajoseps 5y ago
      a couple of exchanges already have FIX APIs. I think Coinbase Pro and Gemini both do.
      - Bootvis 5y ago
        Bitstamp as well
      - exdsq 5y ago
        Interesting! I’ve worked on the other side - blockchains themselves - and only recently started working for a DEX. Lots to learn!

Bootvis 5y ago

JSON is common in crypto what Cantor appears to be trading.

gfd 5y ago
Does crypto allow colocation like stock trading? If it's gonna be some websocket endpoint, what's the point of optimizing microseconds?
- lordnacho 5y ago
  This is a good point. You expect more jitter on a public network. My guess is they've come from traditional finance, where performance porn is just irresistible. Some exchanges do offer colo.
- Bootvis 5y ago
  It kind of does. Most exchanges are hosted in the cloud and I don't think AWS offers colocation similar to the traditional exchanges. I bet Cantor ensures that the machines they are using are close to those used by the exchanges. HFT on a traditional exchange will be faster but that's not the competition. The competition in crypto faces the same problems so you just need to be faster than them. Of course, if the whole process has too much uncontrollable noise (jitter) due to cloud specific reasons it probably doesn't matter. I hope they managed to control this before doing this optimization :)
- ajoseps 5y ago
  one approach I've heard of some places using is to figure out where the exchange is hosted, spin up multiple instances to test the latency to the exchange, then choose the lowest latency instance. Not sure how often one would need to redo this process though.
- ajoseps 5y ago
  one exchange that does allow colo is gemini. They're located in NY5 (https://docs.gemini.com/fix-api/#connecting https://docs.gemini.com/fix-api/#connecting) and allow cross connects. When colo-ed, you wouldn't be using websocket but their FIX API. Multiple other crypto providers are also colo-ed in the same location
- nly 5y ago
  Lots of prop-trading firms aren't comfortable trading over the Internet. Often the concern isn't "someone might see my trades", since reverse engineering alpha from that is hard, or even latency, since not all strategies are super latency sensitive, ... but plain old infosec. These sorts of firms are super paranoid about the exfiltration of proprietary IP. When you add trading over the Internet in to the mix, you just have more ways to do that. You have highly dynamic cloud infrastructure to deal with (whereas traditionally everything is static), DNS security, tricky PKI/certificate management, the need for much tighter access controls, firewalling (both at the network and application protocol level) and the secure logging and monitoring of flow between networks. And everything needs to be tight enough so no single actor (external or internal) can compromise the system.

exdsq 5y ago

Where can I read more about those formats? I work in crypto, same as cantor trading that posted this, and always keen on re-using solved problems if they work :)

andylynch 5y ago
A great starting point is FIX. They are the open standards group in this area, at https://www.fixtrading.org/ https://www.fixtrading.org/. ITCH and OUCH came from Island, now part of NASDAQ and the specs are on their website.
- exdsq 5y ago
  Thanks! I saw they have a working group on crypto so applied to join that, keen to see what they're up to. Standardizing data structures across chains makes a lot of sense but sadly there's a lot of competition rather than collaboration between them "Mine is better than yours hahahaha moon" etc... I'll have to give this some real thought! I think one issue may be that the specification is so complex implementing it within a new DEX without buy-in from others means it's a lot of work with little reward. Might be the case that if larger established front-offices/exchanges start expecting it that crypto could follow. Coinbase released an API standard that blockchains had to conform to for testing the chain prior to listing so it's not unprecedented for that to happen.
  - ajoseps 5y ago
    a couple of exchanges already have FIX APIs. I think Coinbase Pro and Gemini both do.
    - Bootvis 5y ago
      Bitstamp as well
    - exdsq 5y ago
      Interesting! I’ve worked on the other side - blockchains themselves - and only recently started working for a DEX. Lots to learn!