6 ms·

Bootstrapping an Ultra Low Latency Trading Firm, Part 2

119 points by veyron 15y ago

getsat 15y ago
These are great! Keep posting 'em! :) I'm more interested in the tech than the economics, but it's still a good read.
- veyron 15y ago
  someone left a comment in the first post saying: "... most of the crowd here probably comes from HN and as techies we don’t have much knowledge of the trading part of HFT." I assure you, it's in the pipeline.
  - getsat 15y ago
    Understood. Thanks again for publishing these!
gaika 15y ago
Don't ask who is losing on the other side of your trades but think instead who is your customer. What is the service that you provide and how it helps them. If you're making a profit it means you bought inventory when the price was below fair value (your customers didn't need it and wanted to sell as fast as possible) and you sold it when the price was above (your customers really needed the shares right now). The net benefit to everybody is that the volatility is lower, as you moved the price down when it was too high and moved it up when it was too low. The market efficiency is higher too: a lot less capital is required to establish fair prices as market reacts immediately to any imbalance. It also makes the spread lower and makes buying and selling stock cheaper for your customers. Only a few years ago market makers and specialists would chicken out at the first sign of trouble and would widen the spread between bid and ask prices. Crossing the spread is a huge part of your overall expense of trading. Unfortunately very few investors understand full impact of it on their returns and don't appreciate your contribution. Execution time is better now. Even during flash crash it was possible to buy and sell with retail brokers, where's I still remember times in 2001 when retail broker market orders sometimes took minutes to fill.
- veyron 15y ago
  "If you're making a profit it means you bought inventory when the price was below fair value (your customers didn't need it and wanted to sell as fast as possible), and you sold it when the price was high (your customers really needed the shares right now)." It's a bit more complicated than that, because it's fully possible that the seller in a trade is selling above his fair value and the buyer in that trade is buying below her fair value. Supply/demand certainly plays into it (a trader with a very large position is definitely willing to take some price hit if they are able to quicky their position), which will invariably lead to a discussion of utility functions and slowly bore everyone :P
  - gaika 15y ago
    > invariably lead to a discussion of utility functions and slowly bore everyone But that's the crux of the matter - both sides of the trade are getting some utility gain (otherwise the trade would not happen) and thus it is not a zero sum game. It is still zero sum in short term dollars, which just obscures the subject for the people who equate utility with dollars.
    - CJefferson 15y ago
      No, they are not getting utility gain. Recently I was selling an netbook, aiming to get about £100. A friend told me she was wanting something, and looking to spend about £150. We split the difference and went £125. It would not have been a utility gain if someone had quickly jumped between us, given me £100, sold the laptop to her for £150 and kept the £50 for themselves. We would both have ended up out of pocket. The whole point of exchanges is that they connect buyers and sellers. Parasites living making a living by being faster than anyone else are not helping anyone. If there was a level playing field, and everyone went at the same speed, and they could still make money, then I could believe they were adding some utility.
      - gaius 15y ago
        It's more subtle than that. If you wanted to make the sale now and your friend wanted to buy next week when she got paid, then an intermediary would be adding value, purely by being willing to act more quickly. The difference would cover them sitting on the laptop for a week.
epaulson 15y ago
This is fun reading. It reminded me of another blog, "WK's High Frequency Trading Blog" (howtohft.wordpress.com) which got about 9 posts in with some great details, and then went silent. I found it in my Google Reader feedlist, and the last post was from March, but Wordpress now says the blog has been deleted. Anyone know what happened it? Veyron, I'm glad you're writing these posts. It's cool to have HFT demystified.
- vitobcn 15y ago
  No idea what happened to it, but I found this backup if anyone is interested: http://www.opensourcetrader.com/author/wk-selph/ http://www.opensourcetrader.com/author/wk-selph/
- veyron 15y ago
  I personally think he was writing those so that someone would stumble across them and hire him. The new employer must have asked him to take them down.
snikolic 15y ago
What bothers me is how simple this is. This is simple arbitrage that anyone with access to wikipedia could figure out in a few days. What makes these strategies profitable is exclusive access to information - not actual investing prowess. It's about who can afford to buy the fastest hardware on the fastest pipes in the closest proximity to the exchanges and top tier e-brokers. It's frequently even about who you know. If your only competitive advantage is systemically restricting the public's access to information, you're doing it wrong. (but my inner nerd thinks this is super cool, and my inner Gordon Gekko wants to get started today)
- sliverstorm 15y ago
  They are probably pretty convinced that if your only competitive advantage is restricting access to information, you are doing it right. I mean, isn't capitalism basically wrapped around the idea that without competition, businesses will act in their own best interest, which is diametrically opposed to the interests of consumers? That is to say, given the choice any "smart" business will leap at the chance to lock out competition, because that is in the best interests of the business (if it can get away with it)? Anyway, what this is really about is getting the information faster. I don't really see anything wrong with that; some information is just time sensitive, and a good informant will practically be in the business of getting you information faster than his competitors... I see this as much the same thing.
  - dereg 15y ago
    No, capitalism is not a zero sum game, if that is what you're implying.
    - sliverstorm 15y ago
      That's not what I mean. A corporation and its customers can both profit- one does not have to do poorly for the other to do well. I only mean that a corporation can arguably do better (for itself) if it doesn't worry about things like regulations or competitors. A classic, classic example is dumping sewage into the ocean. Companies continue to do it even when it's legislated against.
maximilianburke 15y ago
What services would you use for executing the trades?
allenbrunson 15y ago
115 days ago, you posted that you had quit your job and struck out on your own. Is this the project you've been working on since then?
- veyron 15y ago
  A friend and I built an iPhone app in pursuit of a new concept, and we did a few field tests, but Apple did not approve the app. At that point, although we could have continued pushing, we decided it was easier to stop the project and part ways. I decided I didn't want to go back and work for someone else (blame HN for being bitten by the startup bug :)
joshu 15y ago
The first example is called "pairs trading". Try extending out the math to multiple sets. I once wrote an ADR arb detector, which even took into account the forward contract for the currency trade.
helwr 15y ago
related: http://code.google.com/p/disruptor/ http://code.google.com/p/disruptor/ , http://martinfowler.com/articles/lmax.html http://martinfowler.com/articles/lmax.html , http://www.quantcup.org/ http://www.quantcup.org/ , http://quant.ly/ http://quant.ly/
sthatipamala 15y ago
Is it feasible to apply these basic HFT principles to the Bitcoin market? I assume it would be easier than the "real" markets because you aren't competing against the big players.
- generators 15y ago
  (IMHO) bitcoin is slow and distributed for HFT i think. although technically its the easiest to implement.
- sthatipamala 15y ago
  Wow, downvotes for asking a question?
Sukotto 15y ago
Nice post veyron. Please keep it up. For those interested, here's some other good reading on this topic while you wait for veyron's next post :) - http://www.technologyreview.com/blog/mimssbits/25308/ http://www.technologyreview.com/blog/mimssbits/25308/ Using textual analysis - http://www.institutionalinvestor.com/Article/2593339/Markets-Exchanges/Inside-the-Machine-A-Journey-into-the-World-of-High-Frequency-Trading.html http://www.institutionalinvestor.com/Article/2593339/Markets... Article on HFT - http://news.ycombinator.com/item?id=1517339 http://news.ycombinator.com/item?id=1517339 Ask HN: Is it feasible to do high-frequency trading as an individual? - http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automated_independent_retail_trader_i/ http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automat... I bootstrapped an automated trading firm ama - http://quant.stackexchange.com/ http://quant.stackexchange.com/ - http://bitstat.wordpress.com/ http://bitstat.wordpress.com/
known 15y ago
You can buy off the shelf HFT solution from http://www.stoneridgetechnology.com/products/pci-e-development-boards/hft-development-kit/ http://www.stoneridgetechnology.com/products/pci-e-developme...
- veyron 15y ago
  There are all kinds of companies selling technology solutions for everything (as targeted as FPGA-driven feed data processors; as all-encompassing as the solution you presented). Most of them are crap and oftentimes have no paying customers. Hopefully, I will be discussing the state of the financial tech industry in a later post.
orenmazor 15y ago
I don't know why I wrote this field off as boring several years ago. This is absolutely fascinating (from a technical perspective, I mean)
deleted 15y ago
[deleted]
veyron 15y ago
next post: http://news.ycombinator.com/item?id=2839952 http://news.ycombinator.com/item?id=2839952

119 points by veyron 15y ago

getsat 15y ago

These are great! Keep posting 'em! :) I'm more interested in the tech than the economics, but it's still a good read.

veyron 15y ago
someone left a comment in the first post saying: "... most of the crowd here probably comes from HN and as techies we don’t have much knowledge of the trading part of HFT." I assure you, it's in the pipeline.
- getsat 15y ago
  Understood. Thanks again for publishing these!

gaika 15y ago

Don't ask who is losing on the other side of your trades but think instead who is your customer. What is the service that you provide and how it helps them. If you're making a profit it means you bought inventory when the price was below fair value (your customers didn't need it and wanted to sell as fast as possible) and you sold it when the price was above (your customers really needed the shares right now). The net benefit to everybody is that the volatility is lower, as you moved the price down when it was too high and moved it up when it was too low. The market efficiency is higher too: a lot less capital is required to establish fair prices as market reacts immediately to any imbalance. It also makes the spread lower and makes buying and selling stock cheaper for your customers. Only a few years ago market makers and specialists would chicken out at the first sign of trouble and would widen the spread between bid and ask prices. Crossing the spread is a huge part of your overall expense of trading. Unfortunately very few investors understand full impact of it on their returns and don't appreciate your contribution. Execution time is better now. Even during flash crash it was possible to buy and sell with retail brokers, where's I still remember times in 2001 when retail broker market orders sometimes took minutes to fill.

veyron 15y ago
"If you're making a profit it means you bought inventory when the price was below fair value (your customers didn't need it and wanted to sell as fast as possible), and you sold it when the price was high (your customers really needed the shares right now)." It's a bit more complicated than that, because it's fully possible that the seller in a trade is selling above his fair value and the buyer in that trade is buying below her fair value. Supply/demand certainly plays into it (a trader with a very large position is definitely willing to take some price hit if they are able to quicky their position), which will invariably lead to a discussion of utility functions and slowly bore everyone :P
- gaika 15y ago
  > invariably lead to a discussion of utility functions and slowly bore everyone But that's the crux of the matter - both sides of the trade are getting some utility gain (otherwise the trade would not happen) and thus it is not a zero sum game. It is still zero sum in short term dollars, which just obscures the subject for the people who equate utility with dollars.
  - CJefferson 15y ago
    No, they are not getting utility gain. Recently I was selling an netbook, aiming to get about £100. A friend told me she was wanting something, and looking to spend about £150. We split the difference and went £125. It would not have been a utility gain if someone had quickly jumped between us, given me £100, sold the laptop to her for £150 and kept the £50 for themselves. We would both have ended up out of pocket. The whole point of exchanges is that they connect buyers and sellers. Parasites living making a living by being faster than anyone else are not helping anyone. If there was a level playing field, and everyone went at the same speed, and they could still make money, then I could believe they were adding some utility.
    - gaius 15y ago
      It's more subtle than that. If you wanted to make the sale now and your friend wanted to buy next week when she got paid, then an intermediary would be adding value, purely by being willing to act more quickly. The difference would cover them sitting on the laptop for a week.

epaulson 15y ago

This is fun reading. It reminded me of another blog, "WK's High Frequency Trading Blog" (howtohft.wordpress.com) which got about 9 posts in with some great details, and then went silent. I found it in my Google Reader feedlist, and the last post was from March, but Wordpress now says the blog has been deleted. Anyone know what happened it? Veyron, I'm glad you're writing these posts. It's cool to have HFT demystified.

vitobcn 15y ago
No idea what happened to it, but I found this backup if anyone is interested: http://www.opensourcetrader.com/author/wk-selph/ http://www.opensourcetrader.com/author/wk-selph/
veyron 15y ago
I personally think he was writing those so that someone would stumble across them and hire him. The new employer must have asked him to take them down.

snikolic 15y ago

What bothers me is how simple this is. This is simple arbitrage that anyone with access to wikipedia could figure out in a few days. What makes these strategies profitable is exclusive access to information - not actual investing prowess. It's about who can afford to buy the fastest hardware on the fastest pipes in the closest proximity to the exchanges and top tier e-brokers. It's frequently even about who you know. If your only competitive advantage is systemically restricting the public's access to information, you're doing it wrong. (but my inner nerd thinks this is super cool, and my inner Gordon Gekko wants to get started today)

sliverstorm 15y ago
They are probably pretty convinced that if your only competitive advantage is restricting access to information, you are doing it right. I mean, isn't capitalism basically wrapped around the idea that without competition, businesses will act in their own best interest, which is diametrically opposed to the interests of consumers? That is to say, given the choice any "smart" business will leap at the chance to lock out competition, because that is in the best interests of the business (if it can get away with it)? Anyway, what this is really about is getting the information faster. I don't really see anything wrong with that; some information is just time sensitive, and a good informant will practically be in the business of getting you information faster than his competitors... I see this as much the same thing.
- dereg 15y ago
  No, capitalism is not a zero sum game, if that is what you're implying.
  - sliverstorm 15y ago
    That's not what I mean. A corporation and its customers can both profit- one does not have to do poorly for the other to do well. I only mean that a corporation can arguably do better (for itself) if it doesn't worry about things like regulations or competitors. A classic, classic example is dumping sewage into the ocean. Companies continue to do it even when it's legislated against.

maximilianburke 15y ago

What services would you use for executing the trades?

allenbrunson 15y ago

115 days ago, you posted that you had quit your job and struck out on your own. Is this the project you've been working on since then?

veyron 15y ago
A friend and I built an iPhone app in pursuit of a new concept, and we did a few field tests, but Apple did not approve the app. At that point, although we could have continued pushing, we decided it was easier to stop the project and part ways. I decided I didn't want to go back and work for someone else (blame HN for being bitten by the startup bug :)

joshu 15y ago

The first example is called "pairs trading". Try extending out the math to multiple sets. I once wrote an ADR arb detector, which even took into account the forward contract for the currency trade.

helwr 15y ago

related: http://code.google.com/p/disruptor/ http://code.google.com/p/disruptor/ , http://martinfowler.com/articles/lmax.html http://martinfowler.com/articles/lmax.html , http://www.quantcup.org/ http://www.quantcup.org/ , http://quant.ly/ http://quant.ly/

sthatipamala 15y ago

Is it feasible to apply these basic HFT principles to the Bitcoin market? I assume it would be easier than the "real" markets because you aren't competing against the big players.

generators 15y ago
(IMHO) bitcoin is slow and distributed for HFT i think. although technically its the easiest to implement.
sthatipamala 15y ago
Wow, downvotes for asking a question?

Sukotto 15y ago

Nice post veyron. Please keep it up. For those interested, here's some other good reading on this topic while you wait for veyron's next post :) - http://www.technologyreview.com/blog/mimssbits/25308/ http://www.technologyreview.com/blog/mimssbits/25308/ Using textual analysis - http://www.institutionalinvestor.com/Article/2593339/Markets-Exchanges/Inside-the-Machine-A-Journey-into-the-World-of-High-Frequency-Trading.html http://www.institutionalinvestor.com/Article/2593339/Markets... Article on HFT - http://news.ycombinator.com/item?id=1517339 http://news.ycombinator.com/item?id=1517339 Ask HN: Is it feasible to do high-frequency trading as an individual? - http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automated_independent_retail_trader_i/ http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automat... I bootstrapped an automated trading firm ama - http://quant.stackexchange.com/ http://quant.stackexchange.com/ - http://bitstat.wordpress.com/ http://bitstat.wordpress.com/

known 15y ago

You can buy off the shelf HFT solution from http://www.stoneridgetechnology.com/products/pci-e-development-boards/hft-development-kit/ http://www.stoneridgetechnology.com/products/pci-e-developme...

veyron 15y ago
There are all kinds of companies selling technology solutions for everything (as targeted as FPGA-driven feed data processors; as all-encompassing as the solution you presented). Most of them are crap and oftentimes have no paying customers. Hopefully, I will be discussing the state of the financial tech industry in a later post.

orenmazor 15y ago

I don't know why I wrote this field off as boring several years ago. This is absolutely fascinating (from a technical perspective, I mean)

deleted 15y ago

[deleted]

veyron 15y ago

next post: http://news.ycombinator.com/item?id=2839952 http://news.ycombinator.com/item?id=2839952