7 ms·

Interviewed with HRT awhile back. While I didn't get past the final round, their Python internals interview (which I did pass) was an absolute blast to prepare

Interviewed with HRT awhile back. While I didn't get past the final round, their Python internals interview (which I did pass) was an absolute blast to prepare for, and required a really deep dive into implementation specific details of CPython around things like exactly how collisions are handled in dict, details about memory management, etc. Pretty much had to spend a few weeks in the CPython source to prep, and was, for me, worth the interview just to really learn what's going on. For most teams I would be pretty skeptical of a internal Python fork, but the Python devs at HRT really know their stuff.

htrp 1y ago
when milliseconds mean millions
- PufPufPuf 1y ago
  If that was the case, why use Python in the first place?
  - dmoy 1y ago
    Well there's gonna be people writing code who can't do it in say a high performance C/C++ setup. Not professional programmers, but professional <some finance discipline>. Sometimes it will be worth the tradeoff to put that person and a programmer together to code up a solution in another language. Sometimes it will be worth it to have the non-programmer write it in Python and then do Herculean things in the background to make it fast enough.
    - instig007 1y ago
      > Sometimes it will be worth it to have the non-programmer write it in Python and then do Herculean things in the background to make it fast enough. Nim exists, Crystal exists
      - dmoy 1y ago
        It would not surprise me if some shops end up using less common languages to fill a niche (or hell, invent their own DSL). But it also wouldn't surprise me if a lot of shops land on python because that's what their hiring pool knows.
      - skeledrew 1y ago
        With insignificant ecosystems, compared to Python.
    - procaryote 1y ago
      The gulf between high performance C/C++ and Python is vast and includes most other programming languages, many of which are friendly to write or can be made friendly to write for a limited domain, with significantly less rocket science needed than making python faster.
  - mhh__ 1y ago
    It's not uncommon to have a fast core and then an API that alpha / research teams feed signals into if you are someone like HRT I presume the bulk of their money comes at very short holding periods so you have e.g. fast signals that work short term and then mid frequency alpha signals that spit out a forecast over a few timeframes i.e. it might not be that they buy (aggressively) really quickly but rather than someone sells to them and then they hold onto the position for longer than they would if they have no opinions. Similarly this shapes where you post your orders e.g. if you really want it then you want to be top of the book
  - pgwhalen 1y ago
    Even at the speediest trading firms, the large majority of code is not latency sensitive. Systems and algos are structured such that the fast acting stuff is simple and contained.
    - dumah 1y ago
      There’s tons of latency sensitive code outside of the FPGA systems and it is not simple.
  - kristjansson 1y ago
    They’re not doing the fast stuff in python; nevertheless it’s worth their engineering to make the parts they do in python fast
- shepardrtc 1y ago
  Honestly if you're a millisecond too slow you might as well not trade at all. From my own experience with trying to get Python to go fast for crypto trading, you can get it pretty fast using Cython - single digit microseconds on an average AWS instance for a simple linear regression was my proudest moment. They're probably pushing it even faster because nanoseconds are where the money's at. Many HFT firms are down in the double digit nanoseconds, I believe. Maybe lower.
  - charcircuit 1y ago
    For crypto you pay the miners to put your transaction first. You don't need millisecond precission reaction time.
    - KK7NIL 1y ago
      Crypto trading takes place on exchanges, not the blockchain though?
      - scrlk 1y ago
        In crypto, you have centralised (e.g. Coinbase) and decentralised exchanges (e.g. Uniswap). Decentralised exchanges operate onchain via smart contracts. This article from a16z explains the mechanics of reordering transactions for profit (MEV): https://a16zcrypto.com/posts/article/mev-explained/ https://a16zcrypto.com/posts/article/mev-explained/
  - bobmcnamara 1y ago
    For reference, a byte at 10gbps is almost 1ns long. Though the encoder runs 64/66bits at a time, so you really get around 8B every 7ns or so.
- ActorNightly 1y ago
  Those days are all over btw. Most trading firms are past the whole "beat the other guys to buy". Established large investment firms already have all that on lockdown in terms of infrastructure and influence to the extent where they basically just run the stock market at this point (i.e Tesla posts horrible quarter numbers, but stock goes up). Most of the smaller firms basically try to figure out the patterns of the larger firms and capitalize on that. The timescales have shifted quite a bit.
  - dumah 1y ago
    No, there are absolutely electronic trading markets where a difference of milliseconds of latency to certain events is worth more than a M PnL. That’s a long time.
    - diziet_sma 1y ago
      I took the parent post to mean that a few large firms have emerged as clear winners of the speed game, and most other companies compete on (relatively) longer time scales now.
    - nly 1y ago
      The top trading firms are firing off orders in double digit nanoseconds, not milliseconds. In some cases the order leaving the card starts to emerge before the packet containing the market data event that they're responding to has even finished arriving. Waiting for a full microsecond for the packet to arrive before responding means you're already too slow The speed game is essentially over
      - dumah 1y ago
        That’s irrelevant to the fact that the expected PnL on a millisecond of latency improvement is a lot more than 1M in some markets. Obviously if you are getting what ever trade you are concerned with off in less than one millisecond, the question isn’t well posed. There are many more games to play than delta one takeout and the solutions certainly don’t fit on one or a handful of FPGA’s.
nly 1y ago
I interviewed with them as well. Something like 6-8 interviews only to be told they then, after that, were circulating my CV amongst teams and didn't have a fit for me... But yes, like you I had a great experience
ActorNightly 1y ago
>Python devs at HRT really know their stuff. Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". Scammers are not tech people. And its pretty from their post. > In Python, imports occur at runtime. For each imported name, the interpreter must find, load, and evaluate the contents of a corresponding module. This process gets dramatically slower for large modules, modules on distributed file systems, modules with slow side-effects (code that runs during evaluation), modules with many transitive imports, and C/C++ extension modules with many library dependencies. As they should. The idea that when you type something in the code and then the interpreter just doesn't execute it is how you end up with Java like services, where you have dependency injection chains that are so massive that when the first time everything has to get lazily injected the code takes a massive amount of time to run. Then you have to go figure out where is the initialization code that slows everything down, and start figuring out how to modify your code to make that load first, which leads to a mess. If your python module takes a long time to load, this is a module problem. There is a reason why you can import submodules of modules directly, and overall the __init__.py in the module shouldn't import all the submodules by default. Structure your modules so they don't do massive initialization routines and problem solved. Furthermore, because of pythons dynamic nature, you can do run time imports, including imports in functions. In use, whether you import something up at the top and it gets lazily loaded or you import something right when you have to use it has absolutely no difference other than code syntax, and the latter is actually better because you can see what is going on rather than the lazy loading being hidden away in the interpreter. Or if you really care, you can implement lazy work process inside the modules, so when you import them and use them the first time it works exactly like lazy imports. To basically spend time building a new interpreter with lazy loading just to be able to have all your import statements up at the top just screams that those devs prefer ideology over practicality.
- mhh__ 1y ago
  > "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". You wish lol. How do you think they pay for all the developers? Firms like HRT don't even take outsider money, they don't really need to. And besides, we don't get paid for beating stocks, a lot of funds will do worse than equities in a good year for the latter, the whole point is that you're benchmarked to the risk free rate because your skill is in making money while being overall market neutral. So you rarely take a drawdown anywhere near as badly as equities. As a service this is often a portfolio diversification tool for large allocators rather than something they put all the money into. It is true however that some firms are basically just rubbish beta vehicles that probably should in an ideal world shut down.
sgarland 1y ago
I also interviewed with them a couple of years ago, for their Database SRE (AKA DBRE) role. It was going quite well until I discovered that they required you to live within commuting distance of an office, and unfortunately I had just moved away from one. They didn’t require in-person, supposedly, but needed the option, I guess? I was quite impressed by the interviews, mostly for their pragmatism and skill-fitting. The programming interview wasn’t LC, it was “can you use a language (preferably Python) to parse a CSV and get useful information out of it,” because that’s the skill level the team needs. On the other hand, the Linux and DB interviews were quite in-depth, because again, the team needs those skills. 10/10 would interview again if I’m ever near an office again.

htrp 1y ago

when milliseconds mean millions

PufPufPuf 1y ago
If that was the case, why use Python in the first place?
- dmoy 1y ago
  Well there's gonna be people writing code who can't do it in say a high performance C/C++ setup. Not professional programmers, but professional <some finance discipline>. Sometimes it will be worth the tradeoff to put that person and a programmer together to code up a solution in another language. Sometimes it will be worth it to have the non-programmer write it in Python and then do Herculean things in the background to make it fast enough.
  - instig007 1y ago
    > Sometimes it will be worth it to have the non-programmer write it in Python and then do Herculean things in the background to make it fast enough. Nim exists, Crystal exists
    - dmoy 1y ago
      It would not surprise me if some shops end up using less common languages to fill a niche (or hell, invent their own DSL). But it also wouldn't surprise me if a lot of shops land on python because that's what their hiring pool knows.
    - skeledrew 1y ago
      With insignificant ecosystems, compared to Python.
  - procaryote 1y ago
    The gulf between high performance C/C++ and Python is vast and includes most other programming languages, many of which are friendly to write or can be made friendly to write for a limited domain, with significantly less rocket science needed than making python faster.
- mhh__ 1y ago
  It's not uncommon to have a fast core and then an API that alpha / research teams feed signals into if you are someone like HRT I presume the bulk of their money comes at very short holding periods so you have e.g. fast signals that work short term and then mid frequency alpha signals that spit out a forecast over a few timeframes i.e. it might not be that they buy (aggressively) really quickly but rather than someone sells to them and then they hold onto the position for longer than they would if they have no opinions. Similarly this shapes where you post your orders e.g. if you really want it then you want to be top of the book
- pgwhalen 1y ago
  Even at the speediest trading firms, the large majority of code is not latency sensitive. Systems and algos are structured such that the fast acting stuff is simple and contained.
  - dumah 1y ago
    There’s tons of latency sensitive code outside of the FPGA systems and it is not simple.
- kristjansson 1y ago
  They’re not doing the fast stuff in python; nevertheless it’s worth their engineering to make the parts they do in python fast
shepardrtc 1y ago
Honestly if you're a millisecond too slow you might as well not trade at all. From my own experience with trying to get Python to go fast for crypto trading, you can get it pretty fast using Cython - single digit microseconds on an average AWS instance for a simple linear regression was my proudest moment. They're probably pushing it even faster because nanoseconds are where the money's at. Many HFT firms are down in the double digit nanoseconds, I believe. Maybe lower.
- charcircuit 1y ago
  For crypto you pay the miners to put your transaction first. You don't need millisecond precission reaction time.
  - KK7NIL 1y ago
    Crypto trading takes place on exchanges, not the blockchain though?
    - scrlk 1y ago
      In crypto, you have centralised (e.g. Coinbase) and decentralised exchanges (e.g. Uniswap). Decentralised exchanges operate onchain via smart contracts. This article from a16z explains the mechanics of reordering transactions for profit (MEV): https://a16zcrypto.com/posts/article/mev-explained/ https://a16zcrypto.com/posts/article/mev-explained/
- bobmcnamara 1y ago
  For reference, a byte at 10gbps is almost 1ns long. Though the encoder runs 64/66bits at a time, so you really get around 8B every 7ns or so.
ActorNightly 1y ago
Those days are all over btw. Most trading firms are past the whole "beat the other guys to buy". Established large investment firms already have all that on lockdown in terms of infrastructure and influence to the extent where they basically just run the stock market at this point (i.e Tesla posts horrible quarter numbers, but stock goes up). Most of the smaller firms basically try to figure out the patterns of the larger firms and capitalize on that. The timescales have shifted quite a bit.
- dumah 1y ago
  No, there are absolutely electronic trading markets where a difference of milliseconds of latency to certain events is worth more than a M PnL. That’s a long time.
  - diziet_sma 1y ago
    I took the parent post to mean that a few large firms have emerged as clear winners of the speed game, and most other companies compete on (relatively) longer time scales now.
  - nly 1y ago
    The top trading firms are firing off orders in double digit nanoseconds, not milliseconds. In some cases the order leaving the card starts to emerge before the packet containing the market data event that they're responding to has even finished arriving. Waiting for a full microsecond for the packet to arrive before responding means you're already too slow The speed game is essentially over
    - dumah 1y ago
      That’s irrelevant to the fact that the expected PnL on a millisecond of latency improvement is a lot more than 1M in some markets. Obviously if you are getting what ever trade you are concerned with off in less than one millisecond, the question isn’t well posed. There are many more games to play than delta one takeout and the solutions certainly don’t fit on one or a handful of FPGA’s.

nly 1y ago

I interviewed with them as well. Something like 6-8 interviews only to be told they then, after that, were circulating my CV amongst teams and didn't have a fit for me... But yes, like you I had a great experience

ActorNightly 1y ago

>Python devs at HRT really know their stuff. Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". Scammers are not tech people. And its pretty from their post. > In Python, imports occur at runtime. For each imported name, the interpreter must find, load, and evaluate the contents of a corresponding module. This process gets dramatically slower for large modules, modules on distributed file systems, modules with slow side-effects (code that runs during evaluation), modules with many transitive imports, and C/C++ extension modules with many library dependencies. As they should. The idea that when you type something in the code and then the interpreter just doesn't execute it is how you end up with Java like services, where you have dependency injection chains that are so massive that when the first time everything has to get lazily injected the code takes a massive amount of time to run. Then you have to go figure out where is the initialization code that slows everything down, and start figuring out how to modify your code to make that load first, which leads to a mess. If your python module takes a long time to load, this is a module problem. There is a reason why you can import submodules of modules directly, and overall the __init__.py in the module shouldn't import all the submodules by default. Structure your modules so they don't do massive initialization routines and problem solved. Furthermore, because of pythons dynamic nature, you can do run time imports, including imports in functions. In use, whether you import something up at the top and it gets lazily loaded or you import something right when you have to use it has absolutely no difference other than code syntax, and the latter is actually better because you can see what is going on rather than the lazy loading being hidden away in the interpreter. Or if you really care, you can implement lazy work process inside the modules, so when you import them and use them the first time it works exactly like lazy imports. To basically spend time building a new interpreter with lazy loading just to be able to have all your import statements up at the top just screams that those devs prefer ideology over practicality.

mhh__ 1y ago
> "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". You wish lol. How do you think they pay for all the developers? Firms like HRT don't even take outsider money, they don't really need to. And besides, we don't get paid for beating stocks, a lot of funds will do worse than equities in a good year for the latter, the whole point is that you're benchmarked to the risk free rate because your skill is in making money while being overall market neutral. So you rarely take a drawdown anywhere near as badly as equities. As a service this is often a portfolio diversification tool for large allocators rather than something they put all the money into. It is true however that some firms are basically just rubbish beta vehicles that probably should in an ideal world shut down.

sgarland 1y ago

I also interviewed with them a couple of years ago, for their Database SRE (AKA DBRE) role. It was going quite well until I discovered that they required you to live within commuting distance of an office, and unfortunately I had just moved away from one. They didn’t require in-person, supposedly, but needed the option, I guess? I was quite impressed by the interviews, mostly for their pragmatism and skill-fitting. The programming interview wasn’t LC, it was “can you use a language (preferably Python) to parse a CSV and get useful information out of it,” because that’s the skill level the team needs. On the other hand, the Linux and DB interviews were quite in-depth, because again, the team needs those skills. 10/10 would interview again if I’m ever near an office again.