49 ms·

I am curious about this project because I love this subject. However, it is impossible to build an open source HFT system especially because it will be open so

by nazka 14y ago

I am curious about this project because I love this subject. However, it is impossible to build an open source HFT system especially because it will be open source. The other sharks will know how you trade and they will only need to play against you by sending opposite orders every time. That is what happens when the source code of a big US bank were stolen. So it should be more appropriate to say it will be a fast trading system. The other reason is you can't build a perfect HFT trading system because it's a moving target. It is the war in the micro seconde market. The sharks will send you wrong orders just to disrupt your system or they will manipulate the market. And finally there is a hardware problem. To do a HFT trading system you need to rent your own severs inside the market, use FPGA cards, etc. In conclusion, I only say there is a wrong goal to say "I will build the best open source HFT trading system" rather than "I will build the best open source fast trading system". This project is interesting (and there is Haskell cool!) so I will stay connected to see how it evolves.

olalonde 14y ago
Couldn't the infrastructure be open source while the specific trading algorithms kept closed source?
- conformal 14y ago
  i believe that is how it works: open source infrastructure, private algos
  - zygomega 14y ago
    I intend to publish and collaborate on private algos. Even if others have the code they still don't know where I choose to operate and deploy capital. And if they get rich on the code good luck to them - maybe they'll sponsor a dedicated server for 1.0
- nazka 14y ago
  Yes you send your code on the servers of a company and you use their API to hit the market with their servers, or you rent your own racks. It will cost you a lot but in the HFT world it is the only solution. Any nano second counts. You are the fastest or you are not.
- VLM 14y ago
  This could be a lot more fun than classic codewars. For 99% of the "system" there's no particular reason you'd have to connect to the NYSE and trade real stocks for real money. Looks like they're feeding from iqfeed, but I think it would be huge fun to create an imaginary competition exchange, convince about 100 algo writers to compete, and shove them up against each other purely for the fun of it, see who's a better algo writer. With modern virtualization, it should be pretty easy to distribute a pack of images, including practice data, to replicate an entire financial system on a small scale in your basement, then once you think you have a decent algo, upload your image to the competition league. If your hft codewars league wants limits for storage or cpu cycles, virtualization is a pretty easy way to implement it. Superficially the most obvious thing to do is totally make everything up, from top to bottom, but it would probably work just as well to use real live market data and overlay on top of it. Just as air traffic control simulation has very slowly moved over the decades from purely random to sorta realistic to actual airport data. This could be quite a bit of fun.
  - zygomega 14y ago
    I like your thinking V, and that's not too far away from one of our ideas. Take some algos that are representative of typical market makers and trader behaviour and fit the parameters so that the simulated trading starts to look like the live data. Maybe it's a better idea to use real algo makers.
  - windust 14y ago
    We did this in collaboration with UofC, using our Algo trading platform (http://www.optionscity.com/event/uchicago-midwest-trading-competition http://www.optionscity.com/event/uchicago-midwest-trading-co...) and it was really fun. Hm, I wonder if there is an interest to make this a more open event :)
    - fnordfnordfnord 14y ago
      >Hm, I wonder if there is an interest to make this a more open event :) Yes. I bet there is.
zygomega 14y ago
If you need smart algorithms operating autonomously versus motivated enemies your go to must be open source. The security guys proved that a long time ago. You're right in that the HF bit doesn't really fit but 'fast trading' doesn't have the same ring to it.
- danielhughes 14y ago
  I don't think your security analogy applies to trading algorithms. Open source is good for security because it aids in the discovery of vulnerabilities. However, good trading algorithms are usually reliant on some form of information asymmetry. If you make a discovery and develop a trading algorithm to exploit it, I would think it would be advantageous to keep it a secret for as long as possible. This is not my field of expertise so please correct me if I'm misunderstanding your analogy. Edit - I love the project by the way. As others commented already, there's a really strong argument to be made in favor of open sourcing the infrastructure side of this. And that appears to be where most of your effort is focused (e.g. open infrastructure, private trading algorithms).
  - zygomega 14y ago
    Thanks Dan, the analogy is a stretch maybe, but most algorithms fail due to being narrow models of reality, and criticised by too few eyes. If I discover something, it's extremely unlikely that others will believe it anyway (unless I prove it by becoming filthy rich as a result in which case I'm just fine). You're pretty right about good trading algorithms but that doesn't necessarily apply to good traders or good trading systems. A good system - hft or otherwise - needs to be flexible and evolve to stay in touch with an evolving ecology. And to do this, it needs to keep track of many different trading ideas and algorithms, each of which may have its day in the sun.
  - boothead 14y ago
    The algo is only one part of a real trading organization. Granted it's the one that gets all of the credit, and is perceived as the sexy side but operations and reporting is easily as important. If you had the best algo, but you kept making mistakes or mis-reporting because your platform was crap, you'd be out of business as fast as if your algo didn't make money. I'm talking here of hedge funds where assets under management are all import and generally external. It might be a different story in a market maker or prop shop where getting research into production fast is all important.
j_s 14y ago
Just curious... did you actually look at the project before writing all this?

by nazka 14y ago

olalonde 14y ago

Couldn't the infrastructure be open source while the specific trading algorithms kept closed source?

conformal 14y ago
i believe that is how it works: open source infrastructure, private algos
- zygomega 14y ago
  I intend to publish and collaborate on private algos. Even if others have the code they still don't know where I choose to operate and deploy capital. And if they get rich on the code good luck to them - maybe they'll sponsor a dedicated server for 1.0
nazka 14y ago
Yes you send your code on the servers of a company and you use their API to hit the market with their servers, or you rent your own racks. It will cost you a lot but in the HFT world it is the only solution. Any nano second counts. You are the fastest or you are not.
VLM 14y ago
This could be a lot more fun than classic codewars. For 99% of the "system" there's no particular reason you'd have to connect to the NYSE and trade real stocks for real money. Looks like they're feeding from iqfeed, but I think it would be huge fun to create an imaginary competition exchange, convince about 100 algo writers to compete, and shove them up against each other purely for the fun of it, see who's a better algo writer. With modern virtualization, it should be pretty easy to distribute a pack of images, including practice data, to replicate an entire financial system on a small scale in your basement, then once you think you have a decent algo, upload your image to the competition league. If your hft codewars league wants limits for storage or cpu cycles, virtualization is a pretty easy way to implement it. Superficially the most obvious thing to do is totally make everything up, from top to bottom, but it would probably work just as well to use real live market data and overlay on top of it. Just as air traffic control simulation has very slowly moved over the decades from purely random to sorta realistic to actual airport data. This could be quite a bit of fun.
- zygomega 14y ago
  I like your thinking V, and that's not too far away from one of our ideas. Take some algos that are representative of typical market makers and trader behaviour and fit the parameters so that the simulated trading starts to look like the live data. Maybe it's a better idea to use real algo makers.
- windust 14y ago
  We did this in collaboration with UofC, using our Algo trading platform (http://www.optionscity.com/event/uchicago-midwest-trading-competition http://www.optionscity.com/event/uchicago-midwest-trading-co...) and it was really fun. Hm, I wonder if there is an interest to make this a more open event :)
  - fnordfnordfnord 14y ago
    >Hm, I wonder if there is an interest to make this a more open event :) Yes. I bet there is.

zygomega 14y ago

If you need smart algorithms operating autonomously versus motivated enemies your go to must be open source. The security guys proved that a long time ago. You're right in that the HF bit doesn't really fit but 'fast trading' doesn't have the same ring to it.

danielhughes 14y ago
I don't think your security analogy applies to trading algorithms. Open source is good for security because it aids in the discovery of vulnerabilities. However, good trading algorithms are usually reliant on some form of information asymmetry. If you make a discovery and develop a trading algorithm to exploit it, I would think it would be advantageous to keep it a secret for as long as possible. This is not my field of expertise so please correct me if I'm misunderstanding your analogy. Edit - I love the project by the way. As others commented already, there's a really strong argument to be made in favor of open sourcing the infrastructure side of this. And that appears to be where most of your effort is focused (e.g. open infrastructure, private trading algorithms).
- zygomega 14y ago
  Thanks Dan, the analogy is a stretch maybe, but most algorithms fail due to being narrow models of reality, and criticised by too few eyes. If I discover something, it's extremely unlikely that others will believe it anyway (unless I prove it by becoming filthy rich as a result in which case I'm just fine). You're pretty right about good trading algorithms but that doesn't necessarily apply to good traders or good trading systems. A good system - hft or otherwise - needs to be flexible and evolve to stay in touch with an evolving ecology. And to do this, it needs to keep track of many different trading ideas and algorithms, each of which may have its day in the sun.
- boothead 14y ago
  The algo is only one part of a real trading organization. Granted it's the one that gets all of the credit, and is perceived as the sexy side but operations and reporting is easily as important. If you had the best algo, but you kept making mistakes or mis-reporting because your platform was crap, you'd be out of business as fast as if your algo didn't make money. I'm talking here of hedge funds where assets under management are all import and generally external. It might be a different story in a market maker or prop shop where getting research into production fast is all important.

j_s 14y ago

Just curious... did you actually look at the project before writing all this?