4 ms·

Bootstrapping an Ultra Low Latency Trading Firm, Part 3

91 points by veyron 15y ago

joshu 15y ago
Feels like a jumble of data. You should organize this somehow. The list of "high level" items seems pretty reasonable, but you left out historical data and backtesting.
- veyron 15y ago
  Neither are involved in the intraday production critical path. Maybe I should have explicitly used that phrase. FYI: "Answer: For various reasons, (2) is the natural entry point ( historical data can be used in lieu of real data, strategy requires data, and building an order book simulator can initially obviate the need for true exchange order entry), so let’s start here."
shin_lao 15y ago
The author seems to imply he reimplemented hash tables. I was under the impression there was a fairly good number of high quality implementations of hashes tables in the wild and that it was a thoroughly studied subject. "Not invented here" syndrome?
- deleted 15y ago
  [deleted]
- ig1 15y ago
  It's pretty common for banks to implement their own hashtables, the standard ones often make trade-offs that aren't the best choice for high-performance. For example with Java library hash tables if you have a high insert/removal rate you can end up with frequent GCs. Another common example is to do with synchronization code, unless your synchronization requirements are exactly those provided by the library then the overhead added by the synchronization code can have a significant performance impact.
- nandemo 15y ago
  Well, he didn't write about hash tables in this post... Anyway, even if there are good hash table implementations, sometimes you have to provide a custom hash function. A default hash function will not necessarily give you a table without collisions.
  - shin_lao 15y ago
    There's a teaser on the matter in the end. You're right, generally the work revolves about writing the right hash function. We'll see in the next article.
- veyron 15y ago
  For components not in the critical path, third party implementations are acceptable. However, it is not surprising to see every piece in the critical path to be written in-house (feed handlers, for which there are many companies selling solutions) or rebuilt with heavy modifications (kernel). I had intended to say hash function, but it was very late by the time I posted.
  - orenmazor 15y ago
    I can fully understand it. it's the same in the technical startups that I've been involved with (i.e. hardware or science): - build a quick mvp with off the shelf stuff - start optimizing. by the time you're done, most of your parts are no longer generic and have optimizations where you need them.
  - shin_lao 15y ago
    Looking forward to reading your next post then. We've worked a lot on fast data access and I wonder if you outperformed us.
    - veyron 15y ago
      Do you have any latency numbers you could share? :)
      - shin_lao 15y ago
        We don't directly do HFT, we work with banks to help them do that well. All the numbers I could give you is what our data engine (which is what we sell) gives on our test environment, I cannot share what our customers actually get once deployed and configured on their system. Generally, they are happy. :) We're going to post benchmarks soon as we're going out of stealth mode.
deleted 15y ago
[deleted]
simplekoala 15y ago
Have always been curious about the details. Thanks for these posts
akronim 15y ago
Is interactive brokers actually fast enough? Or do you really need colocation? (for these types of strategy)
- secretasiandan 15y ago
  Max Dama wrote this regarding using IB to do low latency trading while a student but had interned at a HFT shop http://www.maxdama.com/?p=334 http://www.maxdama.com/?p=334 The short of it is no : Indicating there are 255 – 112 = 143 (!!!) hops and 124 milliseconds of latency. Obviously retail traders have no chance with HFT strategies. But places like Lime are accessible to sophisticated traders and offer colo services http://limebrokerage.com/ http://limebrokerage.com/
  - veyron 15y ago
    I agree with the final conclusion, but this is a terrible way to measure system latency! For starters, most exchanges block icmp packets, precluding a ping-based measurement. Latency should be measured by actually going through their systems. For example, after sending a FIX message to IB, it's not known how well their FIX processor performs (at the very least, they have to do risk checks, so its not just cut-through).
    - secretasiandan 15y ago
      I think the point is that if you're not co-located, network transit time from your servers to IB, not even including IB to exchange, is an insurmountable disadvantage.
  - apaprocki 15y ago
    Bloomberg also offers a comprehensive market data product that isn't just a firehose from the exchange. Lots of work went into normalizing the data stream so that consumers of the stream wouldn't have to learn the oddities of hundreds of different market feeds. The latency is actively monitored (in microseconds) and they offer direct co-lo as well. http://www.bloomberg.com/enterprise/data_solutions/market_data/ http://www.bloomberg.com/enterprise/data_solutions/market_da... I agree though, even if you have the best algo around, it could fall flat on its face unless you have the pockets to be side-by-side with the big shops. These places are especially paranoid when it comes to co-lo. I've heard that exchanges guarantee that all the boxes are given the same exact length of network cable to remove any chance that shorter length favors one shop or another.

91 points by veyron 15y ago

joshu 15y ago

Feels like a jumble of data. You should organize this somehow. The list of "high level" items seems pretty reasonable, but you left out historical data and backtesting.

veyron 15y ago
Neither are involved in the intraday production critical path. Maybe I should have explicitly used that phrase. FYI: "Answer: For various reasons, (2) is the natural entry point ( historical data can be used in lieu of real data, strategy requires data, and building an order book simulator can initially obviate the need for true exchange order entry), so let’s start here."

shin_lao 15y ago

The author seems to imply he reimplemented hash tables. I was under the impression there was a fairly good number of high quality implementations of hashes tables in the wild and that it was a thoroughly studied subject. "Not invented here" syndrome?

deleted 15y ago
[deleted]
ig1 15y ago
It's pretty common for banks to implement their own hashtables, the standard ones often make trade-offs that aren't the best choice for high-performance. For example with Java library hash tables if you have a high insert/removal rate you can end up with frequent GCs. Another common example is to do with synchronization code, unless your synchronization requirements are exactly those provided by the library then the overhead added by the synchronization code can have a significant performance impact.
nandemo 15y ago
Well, he didn't write about hash tables in this post... Anyway, even if there are good hash table implementations, sometimes you have to provide a custom hash function. A default hash function will not necessarily give you a table without collisions.
- shin_lao 15y ago
  There's a teaser on the matter in the end. You're right, generally the work revolves about writing the right hash function. We'll see in the next article.
veyron 15y ago
For components not in the critical path, third party implementations are acceptable. However, it is not surprising to see every piece in the critical path to be written in-house (feed handlers, for which there are many companies selling solutions) or rebuilt with heavy modifications (kernel). I had intended to say hash function, but it was very late by the time I posted.
- orenmazor 15y ago
  I can fully understand it. it's the same in the technical startups that I've been involved with (i.e. hardware or science): - build a quick mvp with off the shelf stuff - start optimizing. by the time you're done, most of your parts are no longer generic and have optimizations where you need them.
- shin_lao 15y ago
  Looking forward to reading your next post then. We've worked a lot on fast data access and I wonder if you outperformed us.
  - veyron 15y ago
    Do you have any latency numbers you could share? :)
    - shin_lao 15y ago
      We don't directly do HFT, we work with banks to help them do that well. All the numbers I could give you is what our data engine (which is what we sell) gives on our test environment, I cannot share what our customers actually get once deployed and configured on their system. Generally, they are happy. :) We're going to post benchmarks soon as we're going out of stealth mode.

deleted 15y ago

[deleted]

simplekoala 15y ago

Have always been curious about the details. Thanks for these posts

akronim 15y ago

Is interactive brokers actually fast enough? Or do you really need colocation? (for these types of strategy)

secretasiandan 15y ago
Max Dama wrote this regarding using IB to do low latency trading while a student but had interned at a HFT shop http://www.maxdama.com/?p=334 http://www.maxdama.com/?p=334 The short of it is no : Indicating there are 255 – 112 = 143 (!!!) hops and 124 milliseconds of latency. Obviously retail traders have no chance with HFT strategies. But places like Lime are accessible to sophisticated traders and offer colo services http://limebrokerage.com/ http://limebrokerage.com/
- veyron 15y ago
  I agree with the final conclusion, but this is a terrible way to measure system latency! For starters, most exchanges block icmp packets, precluding a ping-based measurement. Latency should be measured by actually going through their systems. For example, after sending a FIX message to IB, it's not known how well their FIX processor performs (at the very least, they have to do risk checks, so its not just cut-through).
  - secretasiandan 15y ago
    I think the point is that if you're not co-located, network transit time from your servers to IB, not even including IB to exchange, is an insurmountable disadvantage.
- apaprocki 15y ago
  Bloomberg also offers a comprehensive market data product that isn't just a firehose from the exchange. Lots of work went into normalizing the data stream so that consumers of the stream wouldn't have to learn the oddities of hundreds of different market feeds. The latency is actively monitored (in microseconds) and they offer direct co-lo as well. http://www.bloomberg.com/enterprise/data_solutions/market_data/ http://www.bloomberg.com/enterprise/data_solutions/market_da... I agree though, even if you have the best algo around, it could fall flat on its face unless you have the pockets to be side-by-side with the big shops. These places are especially paranoid when it comes to co-lo. I've heard that exchanges guarantee that all the boxes are given the same exact length of network cable to remove any chance that shorter length favors one shop or another.