4 ms·

The author says that he is giving a high level overview of the work done in quantitative trading firms, and then mentions that - "things like co-location, direc

The author says that he is giving a high level overview of the work done in quantitative trading firms, and then mentions that - "things like co-location, direct connection to the exchange without going through an API, using a high-performance language like C++ for production (never Python, R, Matlab, etc), Linux configuration (processor affinity, NUMA, etc), clock synchronization, etc are taken for granted." I'm surprised to see that engineers are required to know hard core VLSI stuff like processors and physical affinity to cores, Non Uniform Memory Access, clock synchronization et al. Thought only Integrated Circuit Digital Design/ Chip design engineers are required to know that. Would anyone kindly shed more light on this subject? Has anyone used VLSI / Digital Design topics as part of his Quant trading / HFT work?

aldanor 9y ago
One of the main reasons here is minimizing the number of (FPGA) cycles it takes you from reading the first byte of the incoming UDP packet to writing the first (or last; depending on exchange gateways configuration) byte of the order that contains your reaction to the market data. If you're also doing computations "inline" (as opposed to what the author described in the article), it's even more complicated since you now have to build the books and calculate various things before committing to a decision. This may affect the general design of the strategy / execution strategy, so quants/engineers involved are often assumed to have the knowledge of how things work, down to the cycle level.
grkvlt 9y ago
This is not really about VLSI style hardware knowledge. The 'processor affinity and NUMA' config is all about minimising the cost of memory access, and maintaining cache consistency. They want to make sure that processes aren't continually being moved to different cores or accessing memory that is further (slower) away, to minimise latency. The 'clock synchronisation' talked about is probably not at the CPU clock level, but rather the wall-clock time, using GPS receivers or atomic clocks and NTP servers to ensure all involved servers are synchronised.
- godelmachine 9y ago
  So, it seems knowledge of heap, stack, memory layout, address space layout, L1 / L2 caches is required.
cbetti 9y ago
Knowing this level of detail was critical for the analytics and database development I've participated in. Any time an algorithm must consult a dataset, and that data is changing rapidly, leveraging knowledge of the underlying system will give you an advantage.
lordnacho 9y ago
I never thought of processor affinity as a VLSI topic. I mean you can muck about with it just using your OS, right? Also I did EE in college and it seemed a lot more low-level than that. Semiconductor principles, digital filters, JK Flip-flops, that kind of thing. I guess you end up meeting NUMA and affinity somewhere along the way. The point the guy is making is that when you're coding this type of thing you need to understand how the computer actually calculates things, rather than just have a vague idea of a machine like you might if you code python. In python and those types of languages a lot of the inner workings are abstracted away. You can sort of just imagine an Oompa Loompa (a thread) reading the code, seeing the instructions (x = a + 2 if y==3 else a - 2 ) and going out to find the value of a in a box and putting in the value of x in another box. No idea where a and x boxes are, how close they are to each other. No idea about how that branching instruction works either, he just checks at the time and decides. By contrast when you're writing for performance you want stuff to be in cache, so you need to have an idea of a machine that includes cache. Specifically the idea that the further away you are from the registers, the longer things take, and that some cache is shared while some isn't. You also want to think about a machine that can speculate and do branch prediction, so you'll need to think about how to write the code so the branch predictor mostly gets it right. I think the top stackoverflow answer is about that. Affinity and NUMA are talking to this kind of machine (which is still abstract of course) which has a bit more detail than the Oompa Loompa.
- godelmachine 9y ago
  Alright. I got your point. Knowledge of Computer Architecture is required ( Reference book - CA: A Quantitiative Approach by Hennessy Paterson). I was thinking more in terms of Digital Electronics.
  - tcbawo 9y ago
    You are probably better off learning more about what Martin Thompson calls 'mechanical sympathy'. Check out the LMAX/Disruptor talks. There are some blog around high performance/low latency computing.
    - 9y ago
hellofunk 9y ago
I took a tour of a high frequency trading firm once and they had a decent-sized division dedicated to their hardware customization for the machines they would place directly into the exchanges, which had chips they programmed directly. And of course it was all C++. Most of the staff who worked there were very specialized in low-latency C++ even if they didn't work directly in the hardware division as well.
- godelmachine 9y ago
  They designed their chips themselves? As in, design the RTL and get it fabricated from a foundary - all by themselves? Did they hire RTL / Digital Design Engineers? Just out of curiosity, may I ask the name of the HFT firm?
  - tcbawo 9y ago
    I would be shocked to hear that they fabricated a chip. The cost and loss of flexibility vs an FPGA makes it unlikely. Very little is so static that the power and latency savings pays for itself. I wouldn't be surprised if these firms fabricated their own crypto ASICs though.
    - godelmachine 9y ago
      That's a good point. Since speed is one of the driving force in this business, software level cryptography won't be able to keep pace with such high speed transactions. Security has to be provided at the hardware level.
  - hellofunk 9y ago
    No, they did not design the actual chips, not did I suggest they did. But they did design the boxes that hold the chips and I was surprised by the amount of effort they dedicated to hardware matters.
    - godelmachine 9y ago
      Would you kindly shed more light on what do you mean by "box"?
HankB99 9y ago
I have heard from someone who develops S/W for one of these firms that they are concerned about instruction timing and have uncovered situations where some instructions occasionally take longer to execute. They go back to the manufacturer to find out why. They remain a couple generations behind the latest processors until they can qualify the newer ones for all of these details. That's how far into the weeds they go.

aldanor 9y ago

One of the main reasons here is minimizing the number of (FPGA) cycles it takes you from reading the first byte of the incoming UDP packet to writing the first (or last; depending on exchange gateways configuration) byte of the order that contains your reaction to the market data. If you're also doing computations "inline" (as opposed to what the author described in the article), it's even more complicated since you now have to build the books and calculate various things before committing to a decision. This may affect the general design of the strategy / execution strategy, so quants/engineers involved are often assumed to have the knowledge of how things work, down to the cycle level.

grkvlt 9y ago

This is not really about VLSI style hardware knowledge. The 'processor affinity and NUMA' config is all about minimising the cost of memory access, and maintaining cache consistency. They want to make sure that processes aren't continually being moved to different cores or accessing memory that is further (slower) away, to minimise latency. The 'clock synchronisation' talked about is probably not at the CPU clock level, but rather the wall-clock time, using GPS receivers or atomic clocks and NTP servers to ensure all involved servers are synchronised.

godelmachine 9y ago
So, it seems knowledge of heap, stack, memory layout, address space layout, L1 / L2 caches is required.

cbetti 9y ago

Knowing this level of detail was critical for the analytics and database development I've participated in. Any time an algorithm must consult a dataset, and that data is changing rapidly, leveraging knowledge of the underlying system will give you an advantage.

lordnacho 9y ago

I never thought of processor affinity as a VLSI topic. I mean you can muck about with it just using your OS, right? Also I did EE in college and it seemed a lot more low-level than that. Semiconductor principles, digital filters, JK Flip-flops, that kind of thing. I guess you end up meeting NUMA and affinity somewhere along the way. The point the guy is making is that when you're coding this type of thing you need to understand how the computer actually calculates things, rather than just have a vague idea of a machine like you might if you code python. In python and those types of languages a lot of the inner workings are abstracted away. You can sort of just imagine an Oompa Loompa (a thread) reading the code, seeing the instructions (x = a + 2 if y==3 else a - 2 ) and going out to find the value of a in a box and putting in the value of x in another box. No idea where a and x boxes are, how close they are to each other. No idea about how that branching instruction works either, he just checks at the time and decides. By contrast when you're writing for performance you want stuff to be in cache, so you need to have an idea of a machine that includes cache. Specifically the idea that the further away you are from the registers, the longer things take, and that some cache is shared while some isn't. You also want to think about a machine that can speculate and do branch prediction, so you'll need to think about how to write the code so the branch predictor mostly gets it right. I think the top stackoverflow answer is about that. Affinity and NUMA are talking to this kind of machine (which is still abstract of course) which has a bit more detail than the Oompa Loompa.

godelmachine 9y ago
Alright. I got your point. Knowledge of Computer Architecture is required ( Reference book - CA: A Quantitiative Approach by Hennessy Paterson). I was thinking more in terms of Digital Electronics.
- tcbawo 9y ago
  You are probably better off learning more about what Martin Thompson calls 'mechanical sympathy'. Check out the LMAX/Disruptor talks. There are some blog around high performance/low latency computing.
  - 9y ago

hellofunk 9y ago

I took a tour of a high frequency trading firm once and they had a decent-sized division dedicated to their hardware customization for the machines they would place directly into the exchanges, which had chips they programmed directly. And of course it was all C++. Most of the staff who worked there were very specialized in low-latency C++ even if they didn't work directly in the hardware division as well.

godelmachine 9y ago
They designed their chips themselves? As in, design the RTL and get it fabricated from a foundary - all by themselves? Did they hire RTL / Digital Design Engineers? Just out of curiosity, may I ask the name of the HFT firm?
- tcbawo 9y ago
  I would be shocked to hear that they fabricated a chip. The cost and loss of flexibility vs an FPGA makes it unlikely. Very little is so static that the power and latency savings pays for itself. I wouldn't be surprised if these firms fabricated their own crypto ASICs though.
  - godelmachine 9y ago
    That's a good point. Since speed is one of the driving force in this business, software level cryptography won't be able to keep pace with such high speed transactions. Security has to be provided at the hardware level.
- hellofunk 9y ago
  No, they did not design the actual chips, not did I suggest they did. But they did design the boxes that hold the chips and I was surprised by the amount of effort they dedicated to hardware matters.
  - godelmachine 9y ago
    Would you kindly shed more light on what do you mean by "box"?

HankB99 9y ago

I have heard from someone who develops S/W for one of these firms that they are concerned about instruction timing and have uncovered situations where some instructions occasionally take longer to execute. They go back to the manufacturer to find out why. They remain a couple generations behind the latest processors until they can qualify the newer ones for all of these details. That's how far into the weeds they go.