8 ms·

One thing I love about the HFT world is the race to be fast. You cant get better by scaling out, you have to make your trading server as fast as possible. This

One thing I love about the HFT world is the race to be fast. You cant get better by scaling out, you have to make your trading server as fast as possible. This leads to some whacky but clever hacks. I've always felt like I was working on a F1 race team when working in HFT. I never came across isolating CPUs, TCP offload, FPGAs, layer 1 switching until I joined a HFT shop. Its like a whole different way of looking at computing system architecture.

roflyear 3y ago
Necessity is an amazing thing: at a company I worked for, the dev team (which I was a part of) "couldn't make things cheaper" than $14k/m for a small app. We spun them into their own org, and now its "their money" they are spending. Their bill is $2k/m! Amazing!
- thr_low_latency 3y ago
  It's easy to make things cheap by making them wrong. Making things fast though is a whole other world.
  - roflyear 3y ago
    "You can't get better by scaling out" disagrees with you. That's my point - if you scale out, it's more expensive.
    - thr_low_latency 3y ago
      The first step of getting decent latency out of modern computers is to build a pile of cash that's taller than you then set it on fire. They are optimized for throughput, not latency. If you want to do things in microseconds instead of anything from milliseconds to seconds you look at all the lovely code everyone has written in the last 40 years then throw it on top of the burning pile of cash. I work in an area where microseconds matter and have to write everything from scratch in kernel mode C because nothing else cuts it.
      - winstonprivacy 3y ago
        Exactly my experience as well.
      - boppo1 3y ago
        > write everything from scratch in kernel mode C because nothing else cuts it. Neat. Is that all for custom processors too?
      - SkipperCat 3y ago
        The cost of writing software isn't as horrible as the cost of colocating in the various financial exchanges. The market data feed costs are massive and you really need to make a significant amount of money trading just to cover the infrastructure costs.
        thr_low_latency 3y ago
        The salaries of the people doing this sort of thing start at mid six figures out of university and go up substantially from there.
        smabie 3y ago
        Compensation for employees in HFT tend to be extremely high. Employee comp number one cost for any trading firm.
        SkipperCat 3y ago
        Yes, the cost is very high. But the cost of an good software engineer at Google, Facebook or another top tier firm is also very high. However, when Google buys a WAN link from NYC to CHI, they pay $XX dollars because they don't care about the latency too much. When a HFT buys that same link, they'll spend three times that much just to get a link that is a few milliseconds faster. Also, the colocation costs at the major exchanges (NASDAQ, NYSE, CBOE) are insane. A cabinet in one of those data centers can be 3x as much as a cabinet somewhere else. I guess what I'm saying is there are a lot things in the HFT world where you have to spend a lot more money than you would if you were doing similar amounts of work at any other tech firm.
jacquesm 3y ago
Optimization for latency is a completely different game than optimization for throughput. The flip side has similar but opposite hacks and you can get a lot of hints on what to do by looking at that polar opposite as things to avoid.
andrepd 3y ago
It's definitely a very interesting field in software engineering terms. Pity it's for such an unproductive use) end x)
- latency-guy2 3y ago
  It's not unproductive at all - you just don't like what it's being used for, and you should explicitly say that rather than proclaiming an industry to be unproductive. Avoid the tendency to exaggerate your feelings.
- amelius 3y ago
  It fits in the "our smartest people are making people click ads" category.
  - hef19898 3y ago
    I'd say it beats big ad tech by miles. First, in the current financial system, HFT serves a tengible purpose, an easily quantifiable one. Well, easier than selling ads so. Also, HFT is not spying on it users to turn a buck. And if HFTs screw up, they loose their own money. If ad tech screws up, they risk screwing up democracy (Cambridge Analytica was a thing, least we forget). For me, HFT is the more honest, and less evil, way of making a buck.
    - delecti 3y ago
      Isn't HFT just being a middle man lots of times? If that understanding is correct, then I disagree that it serves any purpose. At least ads help enable new economic activity, being a middle-man just skims off the top of things that are already happening. Also, HFT has contributed to a flash crash before, so it's not free of externalities either.
      - lordnacho 3y ago
        What's wrong with middle men? People would just cut them out if all they did was skim a bit from everyone.
falcor84 3y ago
Everything in computer science can be solved by adding another layer of indirection, other than latency.
Twixes 3y ago
I'm always impressed how direct the relationship between profit and mathematics/physics/electronics is in HFT. It feels like a marker of civilizational advancement – it's not enough abstraction to just have fungible means of payment and slices of enterprises, not enough to have instruments derived from these, not even enough to have the ability to trade from afar using electricity. At this point there's value even in making the trades as close to instant as possible. Is it _good_? That's what I don't know – in some way it seems more divorced from reality than the engineering of F1, in another it seems more grounded
mgaunard 3y ago
It depends on the market. Most trading venues don't really require a latency much below 100us. The only ones that do are the ones that are fully deterministic. If fastest guy wins, then all you have to do is throw a good tech team at the problem and then money comes in. Targeting a non-deterministic venue comes with its own set of challenges which are not purely technical. You don't know how long it will take for your order to reach the matching engine nor the likelihood of things changing before it gets there. Simulation is also more difficult. So you have to rely on alpha instead, which is a different game.
- parrot987 3y ago
  I'm not too familiar with the space, but what kind of market is fully deterministic?
  - HolyLampshade 3y ago
    Fully is a hard threshold to meet, but most of the US Equity markets have determinism windows in the ballpark of 1-3us. Some Futures markets have got under the 1us window, though I don’t recall exactly what the threshold minimum is there.
  - mgaunard 3y ago
    The most deterministic market is Eurex, which is mostly for trading the eurostoxx and the bund. The CME is also quite deterministic (largest derivative exchange in the world, big names there are the S&P, treasury notes, eurodollar, wti..) Off-the-shelf best-in-class solutions run at 30ns (see STAC T0). The fastest people are faster than this.
  - SkipperCat 3y ago
    No market is fully deterministic, however there are a lot of correlations in all markets. Take options. The option is a derivative and therefore its price is defined by the underlying stock. This was mathematically proven by Black & Scholes (and they got a Nobel prize for it). So, when the underlying moves (the stock) the option should move according to the model. But there are time inefficiencies in the market so a HFT can trade against the misplaced option and lock in a bit of profit. Do that a million times a day any you may make a few bucks.
dahfizz 3y ago
I love it. It feels like a rare opportunity to do "real" software engineering for a job.
maest 3y ago
It's kind of funny because, on the one hand, yes, you need to be fast and have clever hacks. On the other hand, your edge can be explained in very simple terms: in the article, their edge comes from the fact that they had a closer colo than other market participants. ITG made money by front running customers[0], cheaper financing means you can arb an ETF more aggressively than the other guy etc. Like, there are straightforward mechanical reasons why you're making money. No amount of clever hacks can help you if you're at a structural speed, informational or cost disadvantage. [0]: https://www.reuters.com/article/us-securities-regulation-itg-idUSKCN0QH1Q020150812 https://www.reuters.com/article/us-securities-regulation-itg...
- SkipperCat 3y ago
  I've found that most of the HFT strategies are pretty simple. Its exercising a proven mathematical stat/arb condition in the market. The tricky part is finding it first and getting to execute on it first. However, there are plenty of firms that operate in the low frequency space. Long/short shops are typical for this type of trading. They don't care about order speed. They spend all their time trying to find inefficiencies or hidden correlations between financial instruments and profit from them. Low frequency tends to be highly research compute focused, while HFT is highly trade compute focused.
  - aldanor 3y ago
    Here comes the tricky part: "finding it first" may imply you're have more and more false positives as you speed things up beyond crazy. And so, even if you have a "deterministic" model (and no, it's neither "pretty simple" nor "proven mathematical" most of time), it will get fuzzier and fuzzier. Also, HFT is execution-centric, indeed, but it doesn't mean you can't use complicated math models and all that, it's just much harder to do and is more of a computational challenge (which is part of what makes it fun). Source: hft quant
intelVISA 3y ago
It's like wearing glasses and seeing things As They Should Be whilst everybody else is telling you to just accept and deal with their blurry abstractions.

roflyear 3y ago

Necessity is an amazing thing: at a company I worked for, the dev team (which I was a part of) "couldn't make things cheaper" than $14k/m for a small app. We spun them into their own org, and now its "their money" they are spending. Their bill is $2k/m! Amazing!

thr_low_latency 3y ago
It's easy to make things cheap by making them wrong. Making things fast though is a whole other world.
- roflyear 3y ago
  "You can't get better by scaling out" disagrees with you. That's my point - if you scale out, it's more expensive.
  - thr_low_latency 3y ago
    The first step of getting decent latency out of modern computers is to build a pile of cash that's taller than you then set it on fire. They are optimized for throughput, not latency. If you want to do things in microseconds instead of anything from milliseconds to seconds you look at all the lovely code everyone has written in the last 40 years then throw it on top of the burning pile of cash. I work in an area where microseconds matter and have to write everything from scratch in kernel mode C because nothing else cuts it.
    - winstonprivacy 3y ago
      Exactly my experience as well.
    - boppo1 3y ago
      > write everything from scratch in kernel mode C because nothing else cuts it. Neat. Is that all for custom processors too?
    - SkipperCat 3y ago
      The cost of writing software isn't as horrible as the cost of colocating in the various financial exchanges. The market data feed costs are massive and you really need to make a significant amount of money trading just to cover the infrastructure costs.
      - thr_low_latency 3y ago
        The salaries of the people doing this sort of thing start at mid six figures out of university and go up substantially from there.
      - smabie 3y ago
        Compensation for employees in HFT tend to be extremely high. Employee comp number one cost for any trading firm.
        SkipperCat 3y ago
        Yes, the cost is very high. But the cost of an good software engineer at Google, Facebook or another top tier firm is also very high. However, when Google buys a WAN link from NYC to CHI, they pay $XX dollars because they don't care about the latency too much. When a HFT buys that same link, they'll spend three times that much just to get a link that is a few milliseconds faster. Also, the colocation costs at the major exchanges (NASDAQ, NYSE, CBOE) are insane. A cabinet in one of those data centers can be 3x as much as a cabinet somewhere else. I guess what I'm saying is there are a lot things in the HFT world where you have to spend a lot more money than you would if you were doing similar amounts of work at any other tech firm.

jacquesm 3y ago

Optimization for latency is a completely different game than optimization for throughput. The flip side has similar but opposite hacks and you can get a lot of hints on what to do by looking at that polar opposite as things to avoid.

andrepd 3y ago

It's definitely a very interesting field in software engineering terms. Pity it's for such an unproductive use) end x)

latency-guy2 3y ago
It's not unproductive at all - you just don't like what it's being used for, and you should explicitly say that rather than proclaiming an industry to be unproductive. Avoid the tendency to exaggerate your feelings.
amelius 3y ago
It fits in the "our smartest people are making people click ads" category.
- hef19898 3y ago
  I'd say it beats big ad tech by miles. First, in the current financial system, HFT serves a tengible purpose, an easily quantifiable one. Well, easier than selling ads so. Also, HFT is not spying on it users to turn a buck. And if HFTs screw up, they loose their own money. If ad tech screws up, they risk screwing up democracy (Cambridge Analytica was a thing, least we forget). For me, HFT is the more honest, and less evil, way of making a buck.
  - delecti 3y ago
    Isn't HFT just being a middle man lots of times? If that understanding is correct, then I disagree that it serves any purpose. At least ads help enable new economic activity, being a middle-man just skims off the top of things that are already happening. Also, HFT has contributed to a flash crash before, so it's not free of externalities either.
    - lordnacho 3y ago
      What's wrong with middle men? People would just cut them out if all they did was skim a bit from everyone.

falcor84 3y ago

Everything in computer science can be solved by adding another layer of indirection, other than latency.

Twixes 3y ago

I'm always impressed how direct the relationship between profit and mathematics/physics/electronics is in HFT. It feels like a marker of civilizational advancement – it's not enough abstraction to just have fungible means of payment and slices of enterprises, not enough to have instruments derived from these, not even enough to have the ability to trade from afar using electricity. At this point there's value even in making the trades as close to instant as possible. Is it _good_? That's what I don't know – in some way it seems more divorced from reality than the engineering of F1, in another it seems more grounded

mgaunard 3y ago

It depends on the market. Most trading venues don't really require a latency much below 100us. The only ones that do are the ones that are fully deterministic. If fastest guy wins, then all you have to do is throw a good tech team at the problem and then money comes in. Targeting a non-deterministic venue comes with its own set of challenges which are not purely technical. You don't know how long it will take for your order to reach the matching engine nor the likelihood of things changing before it gets there. Simulation is also more difficult. So you have to rely on alpha instead, which is a different game.

parrot987 3y ago
I'm not too familiar with the space, but what kind of market is fully deterministic?
- HolyLampshade 3y ago
  Fully is a hard threshold to meet, but most of the US Equity markets have determinism windows in the ballpark of 1-3us. Some Futures markets have got under the 1us window, though I don’t recall exactly what the threshold minimum is there.
- mgaunard 3y ago
  The most deterministic market is Eurex, which is mostly for trading the eurostoxx and the bund. The CME is also quite deterministic (largest derivative exchange in the world, big names there are the S&P, treasury notes, eurodollar, wti..) Off-the-shelf best-in-class solutions run at 30ns (see STAC T0). The fastest people are faster than this.
- SkipperCat 3y ago
  No market is fully deterministic, however there are a lot of correlations in all markets. Take options. The option is a derivative and therefore its price is defined by the underlying stock. This was mathematically proven by Black & Scholes (and they got a Nobel prize for it). So, when the underlying moves (the stock) the option should move according to the model. But there are time inefficiencies in the market so a HFT can trade against the misplaced option and lock in a bit of profit. Do that a million times a day any you may make a few bucks.

dahfizz 3y ago

I love it. It feels like a rare opportunity to do "real" software engineering for a job.

maest 3y ago

It's kind of funny because, on the one hand, yes, you need to be fast and have clever hacks. On the other hand, your edge can be explained in very simple terms: in the article, their edge comes from the fact that they had a closer colo than other market participants. ITG made money by front running customers[0], cheaper financing means you can arb an ETF more aggressively than the other guy etc. Like, there are straightforward mechanical reasons why you're making money. No amount of clever hacks can help you if you're at a structural speed, informational or cost disadvantage. [0]: https://www.reuters.com/article/us-securities-regulation-itg-idUSKCN0QH1Q020150812 https://www.reuters.com/article/us-securities-regulation-itg...

SkipperCat 3y ago
I've found that most of the HFT strategies are pretty simple. Its exercising a proven mathematical stat/arb condition in the market. The tricky part is finding it first and getting to execute on it first. However, there are plenty of firms that operate in the low frequency space. Long/short shops are typical for this type of trading. They don't care about order speed. They spend all their time trying to find inefficiencies or hidden correlations between financial instruments and profit from them. Low frequency tends to be highly research compute focused, while HFT is highly trade compute focused.
- aldanor 3y ago
  Here comes the tricky part: "finding it first" may imply you're have more and more false positives as you speed things up beyond crazy. And so, even if you have a "deterministic" model (and no, it's neither "pretty simple" nor "proven mathematical" most of time), it will get fuzzier and fuzzier. Also, HFT is execution-centric, indeed, but it doesn't mean you can't use complicated math models and all that, it's just much harder to do and is more of a computational challenge (which is part of what makes it fun). Source: hft quant

intelVISA 3y ago

It's like wearing glasses and seeing things As They Should Be whilst everybody else is telling you to just accept and deal with their blurry abstractions.