4 ms·

I'm under the impression that order books tend to be hard to scale horizontally, because they have tremendous atomicity requirement. Can anybody chime in wheth

I'm under the impression that order books tend to be hard to scale horizontally, because they have tremendous atomicity requirement. Can anybody chime in whether these systems, today, tend to be on distributed infra versus e.g. mainframes or other big vertical compute pieces?

hangonhn 6y ago
I don't know the specific answer to your question but I do have a data point to add. When I was just starting out in my career in the early 2000s, I worked for a hedge fund. One curious thing I noticed is that all the traders had time stamping machines on their desks that are inspected and sealed regularly. I was told that during an outage, these timestamp machines allow trades to continue to happen because the trader can write down the order and stamp it and their counter party would presumably do the same. Later the orders can then be reconciled when things are back up and running. This leads me to think that it is at the very least possible for transactions to be recorded in a distributed manner. Example of machine: http://www.carpenterstimesystems.com/Amano-TS3000i-OATS-Time-Validation-Clock-System.html http://www.carpenterstimesystems.com/Amano-TS3000i-OATS-Time... This might all be historical now.
LittlePeter 6y ago
The matching of orders for a single asset happens on one machine, call it matching engine. Matching is done based on price-time priority. Price is set by the trader, the time is set to the time the order was received by one of the order gateways sitting in front of the matching engine. The order gateways' clocks are in sync, up to precision epsilon. So if you send two orders serially from same data center (to ignore effects of public Internet latencies and routing), and if they happen to be processed by different gateways, you still will see same ordering of your orders in matching engine, bar some cosmic ray events. It is all commodity hardware running some flavor of Linux. However this was couple years ago, they may be using ASICs and FPGAs now. The matching of orders for a single asset must still be done in single process otherwise you cannot guarantee price-time priority.
motohagiography 6y ago
Following up, there was a random theory on WSB that since hedge funds were short over %130 of Gamestop's stock, there is a real risk that this will cause a liquidity crunch where there is literally no available stock to cover their positions. As in there does not exist the stock to cover those positions, which means institutions are going to be in a race to avoid bankruptcy and have to outbid each other to acquire the remaining stock held by all these retail investors who know what they have. This is why WSB posters are saying $5000 is a conceivable price. This is what would cause regulatory intervention, and certainly sets up the means/motive/opportunity for interrupting trading. Like the Fed steps in and buys it out or something. This looks like an LTCM level event.

hangonhn 6y ago

I don't know the specific answer to your question but I do have a data point to add. When I was just starting out in my career in the early 2000s, I worked for a hedge fund. One curious thing I noticed is that all the traders had time stamping machines on their desks that are inspected and sealed regularly. I was told that during an outage, these timestamp machines allow trades to continue to happen because the trader can write down the order and stamp it and their counter party would presumably do the same. Later the orders can then be reconciled when things are back up and running. This leads me to think that it is at the very least possible for transactions to be recorded in a distributed manner. Example of machine: http://www.carpenterstimesystems.com/Amano-TS3000i-OATS-Time-Validation-Clock-System.html http://www.carpenterstimesystems.com/Amano-TS3000i-OATS-Time... This might all be historical now.

LittlePeter 6y ago

The matching of orders for a single asset happens on one machine, call it matching engine. Matching is done based on price-time priority. Price is set by the trader, the time is set to the time the order was received by one of the order gateways sitting in front of the matching engine. The order gateways' clocks are in sync, up to precision epsilon. So if you send two orders serially from same data center (to ignore effects of public Internet latencies and routing), and if they happen to be processed by different gateways, you still will see same ordering of your orders in matching engine, bar some cosmic ray events. It is all commodity hardware running some flavor of Linux. However this was couple years ago, they may be using ASICs and FPGAs now. The matching of orders for a single asset must still be done in single process otherwise you cannot guarantee price-time priority.

motohagiography 6y ago

Following up, there was a random theory on WSB that since hedge funds were short over %130 of Gamestop's stock, there is a real risk that this will cause a liquidity crunch where there is literally no available stock to cover their positions. As in there does not exist the stock to cover those positions, which means institutions are going to be in a race to avoid bankruptcy and have to outbid each other to acquire the remaining stock held by all these retail investors who know what they have. This is why WSB posters are saying $5000 is a conceivable price. This is what would cause regulatory intervention, and certainly sets up the means/motive/opportunity for interrupting trading. Like the Fed steps in and buys it out or something. This looks like an LTCM level event.