9 ms·

As someone who has played with writing trading bots but never traded them with real money, some advice: if your results seem too good to be true, they probably

As someone who has played with writing trading bots but never traded them with real money, some advice: if your results seem too good to be true, they probably are. Your trading bot may be doing unrealistic things or its results may not be reliable if the following are true: - You are trading in a market with low liquidity or one that is controlled by a small number of market participants. I'm not an expert but I think this would apply more to markets like penny stocks and less to big markets like forex for major currency pairs - You are not taking transaction costs into account or not doing so properly - Your bot makes a low number of trades, making the results close or equivalent to lucky coin flips - Your bot is simply making trades that cannot be executed, or may be doing simulated trades of something that is not actually tradable. This applies to a large number of research papers that assume you can just buy and trade the S&P 500 itself. You can trade ETFs that are tied to an index but an index is not a tradable instrument in of itself. Once you realize this, a lot of papers seem very weird - You are not modelling other aspects of the trading process realistically, such as assuming the bot has infinite funds to trade, allowing it to take unlimited losses and continue trading when in reality you'd be hit with a margin call and your trading would be stopped - Your code is committing any number of data snooping errors where the bot is asked to trade at time A (say the open of a trading session) but has access to future data (say the closing price of that day, future data that would not actually exist in a live environment) - Depending on what you believe about how market conditions change over time, your bot may have worked in the past but would not work if used today. I.e., the market may have adapted to whatever edge your bot may have discovered There are probably lots more pitfalls I don't even know about since I'm not an actual trader. I'm not discouraging anyone from playing around or trying things, of course. I think it's great fun, which is why I do it. Here's the good news: if you realize you don't actually have an edge and avoid risking your hard-earned money, you come out ahead of almost all people who ever trade.

pjc50 6y ago
I worked on a unique HFT system which was capable of starting the response packet before the incoming packet's final byte had arrived. Negative latency. If it mispredicted the future, it would simply scramble the trailer and cause the packet to fail UDP checksum. It didn't make money in the real world because the quality of decision that could be made at that speed wasn't good enough.
- KMag 6y ago
  That's super clever!
- inopinatus 6y ago
  From time to time there comes a remark that is the absolute epitome of Hacker News, and I mean that earnestly and without any backhanded rancour, and I thank you for this flawless specimen.
  - pjc50 6y ago
    The original case study: https://web.archive.org/web/20180224141009/http://www.argondesign.com/case-studies/2013/sep/18/high-performance-trading https://web.archive.org/web/20180224141009/http://www.argond...
- lordnacho 6y ago
  What was the rest of the infrastructure? Was this at a proper HFT firm with everything else streamlined?
  - pjc50 6y ago
    We were consultants to an HFT firm; supposedly set up correctly but I didn't see that part of the project.
- tijsvd 6y ago
  This is not unique, it's standard practice. Many exchanges send large UDP packets with the most valuable information at the front. Or the packet is structured such that you can make an informed bet based on size and first sub-message. Failing the checksum was sort of common as well, but exchanges don't like it. These days most tricks have to do with avoiding as much serialisation time as possible, e.g. by sending ahead part of the TCP payload and filling in with some innocent order if there was no opportunity.
  - twic 6y ago
    Exchanges have cracked down on this sort of thing a lot this year. From a letter from the CME to the CFTC dated July 24, 2020: > On July 26, 2020, an enhancement to the Market Segment Gateway (“MSGW”) will be introduced to further safeguard the CME Globex electronic trading platform (“CME Globex”) infrastructure by introducing a delay of at least three microseconds if the MSGW receives a partial order message as a means of ensuring the stability of the platform. Certain participants intentionally submit partial order messages to reduce latency, and only complete the order message upon the happening of an event or trading signal. Implementation of the enhancement is expected to reduce the frequency of intentionally split order messages as the additional processing time will serve as a deterrent. The letter is on the web, but it's a PDF; the Google search result has a redirect link to it, DuckDuckGo can't seem to find it, and Firefox on Android won't tell me the URL it downloads things from, so I'm afraid I can't link to it! CME also made a more general change, where if they decide a participant is sending dodgy messages, they will reroute all their packets to a special gateway for "additional checks", but in practice, to impose a latency penalty. Can't find the documentation on that at all, though.
atomicnumber3 6y ago
I worked in this industry. 2 more common issues: * Doing a latency-sensitive trade when you don't have good execution. It's easy to go wild in simulation and think you can flip in and out of positions. But if you're a retail trader (and in this context, by that I mean "not connected directly to the exchanges, at the minimum") * Not taking into account the impact of your own trading on markets. This is obviously impossible to really simulate. Sometimes you can ignore it (if there's enough liquidity) but I've seen trades that looked great on paper and when trading at small-ish sizes, but then when you try to crank it up and do more volume, prices run away from you. Obviously, there is money to be made in algo trading. It's big business and obviously not everyone is doing crazy latency sensitive stuff- there are quant trades that you could probably do with execution available to retail traders. And honestly I wouldn't be surprised if some retail traders manage it. But I will say that I think for an individual, it's not worth the effort even if you are ones of the ones who can consistently make a profit. Just buy sp500 ETFs and sit on them and do something else with your time.
- sitzkrieg 6y ago
  same here, aits definitely worth pointing out (as you me tion) simulating execution and realisitic fills (and dont forget rt commissions!) is extremely difficult and also will absolutely kill strategies that casually look profitable
- IgorPartola 6y ago
  Would you mind if I asked you a question since I have never worked in this industry but did play with crypto trading a while back. Before Mt Gox was shut down I was trading on a couple of tertiary small exchanges and at the time there was a lot of talk of arbitrage between different exchanges and how transaction latency and fees made it very risky at best and a losing proposition in most cases. But what I was wondering about is whether in a situation like that (one large exchange dominating the market, several smaller exchanges trading the same commodity) if it was possible to use the large exchange as a sort of oracle. Essentially my hypothesis was that Mt Gox sets the price and other exchanges follow on a delay so if I watch Mt Gox I can predict where the price on the secondary exchanges will go a few seconds to a few minutes ahead of it moving. I ran a bunch of historical data through some basic analysis scripts and noticed that indeed there was a pattern but when I actually implemented a bot to trade BTC it lost money more often than not. I am curious if (a) that idea has any validity and (b) was me losing money on this strategy due to latency and implementation errors or due to some fundamental principle of trading that I am missing.
gjvnq 6y ago
I tried making a bitcoin trading bot and almost all of the money (about 10 bucks, not no big problem). My main mistake was using the historical trades instead of the historical offers as a testing dataset.
- kamel3d 6y ago
  did it work at the end? I want to make a crypto trading bot seems simpler than stock one
  - m3at 6y ago
    Not the GP but I made trading algorithm for a living (not on crypto) and have friends still doing that (on crypto): yes it can work, but in 99% of cases you shouldn't do it for the money. Let me expand: If you can afford to spare part of your income and invest that for long term buy and hold, that'll work better (ex. ETF). If you can't spare anything, focusing on leveling up trendy skills, then you can land a better job on the short term. However if you just want to trade for the fun of it, yes crypto can be simpler than stock, mainly because the API are better. Set up a maximum budget as a safety net and enjoy :)
    - elbear 6y ago
      Could you share what you learned in order to make that trading algorithm?
  - imtringued 6y ago
    If you can find some barely known crypto with a lack of liquidity then you might be able to get your feet wet and test out your algorithms. Doesn't mean you will earn a lot of money.
imtringued 6y ago
The reality is that stock market trading is like a normal business selling products except you provide liquidity instead of products. Now imagine the market you have chosen needs 1000 widgets per day. If there are competing factories in the market place that already produce 1000 widgets then your only chance of breaking into that industry is by displacing existing competitors. You need to have a better product (or what the finance guys call an "edge"). The vast majority of people don't have such a product idea. They would probably just produce the same stuff the big guys have been making and then wonder why it isn't making them a profit. The market is already saturated with those widgets and there is no need for more factories. Replace widget with liquidity and factories with traders and you should immediately see why most traders are losing money. They are operating in markets where their liquidity simply isn't needed. If you want to make money off of trading then you have to find a market with low liquidity where having more traders is actually welcome but why do hard work if you can just invest your money and still get good gains without working?
contingencies 6y ago
Good list. You forgot "over-fitting your algorithm to historical data".
ddls 6y ago
Because - management fees, margins, and available capital can easily be modelled properly - you can easily set up constraints (like no fractional trading for your S&P example) - you can set up a proper point-in-time database to avoid snooping (especially if you're using earnings reports or other fundamental data which is often actually released _after_ it's published release date...) - you can set up regime-shifting simulation environments (various market conditions) - you can avoid over-fitting if you're back-testing (with dozens of techniques, most notably : test once and forget about parameter optimization) I would say that with paper trading and back-testing the serious problems are that : - your orders don't show up on the book so no one sees and reacts to your limit orders - your "filled" orders don't affect the book, so you're not affecting liquidity, so the market doesn't change in response to your trading - your bot has no access to market micro-structure strategies and conditional orders (and if you want to trade fast or are placing big trades you need them) These are the problems that make any simulation unrealistic, and they are fundamental. It's shadowboxing, which is not entirely devoid of value, but which is certainly insufficient on its own. (I've worked as a quant developing strategies for several funds these past 15 years)
deanCommie 6y ago
> You can trade ETFs that are tied to an index but an index is not a tradable instrument in of itself. Out of curiousity, what's the difference?

pjc50 6y ago

I worked on a unique HFT system which was capable of starting the response packet before the incoming packet's final byte had arrived. Negative latency. If it mispredicted the future, it would simply scramble the trailer and cause the packet to fail UDP checksum. It didn't make money in the real world because the quality of decision that could be made at that speed wasn't good enough.

KMag 6y ago
That's super clever!
inopinatus 6y ago
From time to time there comes a remark that is the absolute epitome of Hacker News, and I mean that earnestly and without any backhanded rancour, and I thank you for this flawless specimen.
- pjc50 6y ago
  The original case study: https://web.archive.org/web/20180224141009/http://www.argondesign.com/case-studies/2013/sep/18/high-performance-trading https://web.archive.org/web/20180224141009/http://www.argond...
lordnacho 6y ago
What was the rest of the infrastructure? Was this at a proper HFT firm with everything else streamlined?
- pjc50 6y ago
  We were consultants to an HFT firm; supposedly set up correctly but I didn't see that part of the project.
tijsvd 6y ago
This is not unique, it's standard practice. Many exchanges send large UDP packets with the most valuable information at the front. Or the packet is structured such that you can make an informed bet based on size and first sub-message. Failing the checksum was sort of common as well, but exchanges don't like it. These days most tricks have to do with avoiding as much serialisation time as possible, e.g. by sending ahead part of the TCP payload and filling in with some innocent order if there was no opportunity.
- twic 6y ago
  Exchanges have cracked down on this sort of thing a lot this year. From a letter from the CME to the CFTC dated July 24, 2020: > On July 26, 2020, an enhancement to the Market Segment Gateway (“MSGW”) will be introduced to further safeguard the CME Globex electronic trading platform (“CME Globex”) infrastructure by introducing a delay of at least three microseconds if the MSGW receives a partial order message as a means of ensuring the stability of the platform. Certain participants intentionally submit partial order messages to reduce latency, and only complete the order message upon the happening of an event or trading signal. Implementation of the enhancement is expected to reduce the frequency of intentionally split order messages as the additional processing time will serve as a deterrent. The letter is on the web, but it's a PDF; the Google search result has a redirect link to it, DuckDuckGo can't seem to find it, and Firefox on Android won't tell me the URL it downloads things from, so I'm afraid I can't link to it! CME also made a more general change, where if they decide a participant is sending dodgy messages, they will reroute all their packets to a special gateway for "additional checks", but in practice, to impose a latency penalty. Can't find the documentation on that at all, though.

atomicnumber3 6y ago

I worked in this industry. 2 more common issues: * Doing a latency-sensitive trade when you don't have good execution. It's easy to go wild in simulation and think you can flip in and out of positions. But if you're a retail trader (and in this context, by that I mean "not connected directly to the exchanges, at the minimum") * Not taking into account the impact of your own trading on markets. This is obviously impossible to really simulate. Sometimes you can ignore it (if there's enough liquidity) but I've seen trades that looked great on paper and when trading at small-ish sizes, but then when you try to crank it up and do more volume, prices run away from you. Obviously, there is money to be made in algo trading. It's big business and obviously not everyone is doing crazy latency sensitive stuff- there are quant trades that you could probably do with execution available to retail traders. And honestly I wouldn't be surprised if some retail traders manage it. But I will say that I think for an individual, it's not worth the effort even if you are ones of the ones who can consistently make a profit. Just buy sp500 ETFs and sit on them and do something else with your time.

sitzkrieg 6y ago
same here, aits definitely worth pointing out (as you me tion) simulating execution and realisitic fills (and dont forget rt commissions!) is extremely difficult and also will absolutely kill strategies that casually look profitable
IgorPartola 6y ago
Would you mind if I asked you a question since I have never worked in this industry but did play with crypto trading a while back. Before Mt Gox was shut down I was trading on a couple of tertiary small exchanges and at the time there was a lot of talk of arbitrage between different exchanges and how transaction latency and fees made it very risky at best and a losing proposition in most cases. But what I was wondering about is whether in a situation like that (one large exchange dominating the market, several smaller exchanges trading the same commodity) if it was possible to use the large exchange as a sort of oracle. Essentially my hypothesis was that Mt Gox sets the price and other exchanges follow on a delay so if I watch Mt Gox I can predict where the price on the secondary exchanges will go a few seconds to a few minutes ahead of it moving. I ran a bunch of historical data through some basic analysis scripts and noticed that indeed there was a pattern but when I actually implemented a bot to trade BTC it lost money more often than not. I am curious if (a) that idea has any validity and (b) was me losing money on this strategy due to latency and implementation errors or due to some fundamental principle of trading that I am missing.

gjvnq 6y ago

I tried making a bitcoin trading bot and almost all of the money (about 10 bucks, not no big problem). My main mistake was using the historical trades instead of the historical offers as a testing dataset.

kamel3d 6y ago
did it work at the end? I want to make a crypto trading bot seems simpler than stock one
- m3at 6y ago
  Not the GP but I made trading algorithm for a living (not on crypto) and have friends still doing that (on crypto): yes it can work, but in 99% of cases you shouldn't do it for the money. Let me expand: If you can afford to spare part of your income and invest that for long term buy and hold, that'll work better (ex. ETF). If you can't spare anything, focusing on leveling up trendy skills, then you can land a better job on the short term. However if you just want to trade for the fun of it, yes crypto can be simpler than stock, mainly because the API are better. Set up a maximum budget as a safety net and enjoy :)
  - elbear 6y ago
    Could you share what you learned in order to make that trading algorithm?
- imtringued 6y ago
  If you can find some barely known crypto with a lack of liquidity then you might be able to get your feet wet and test out your algorithms. Doesn't mean you will earn a lot of money.

imtringued 6y ago

The reality is that stock market trading is like a normal business selling products except you provide liquidity instead of products. Now imagine the market you have chosen needs 1000 widgets per day. If there are competing factories in the market place that already produce 1000 widgets then your only chance of breaking into that industry is by displacing existing competitors. You need to have a better product (or what the finance guys call an "edge"). The vast majority of people don't have such a product idea. They would probably just produce the same stuff the big guys have been making and then wonder why it isn't making them a profit. The market is already saturated with those widgets and there is no need for more factories. Replace widget with liquidity and factories with traders and you should immediately see why most traders are losing money. They are operating in markets where their liquidity simply isn't needed. If you want to make money off of trading then you have to find a market with low liquidity where having more traders is actually welcome but why do hard work if you can just invest your money and still get good gains without working?

contingencies 6y ago

Good list. You forgot "over-fitting your algorithm to historical data".

ddls 6y ago

Because - management fees, margins, and available capital can easily be modelled properly - you can easily set up constraints (like no fractional trading for your S&P example) - you can set up a proper point-in-time database to avoid snooping (especially if you're using earnings reports or other fundamental data which is often actually released _after_ it's published release date...) - you can set up regime-shifting simulation environments (various market conditions) - you can avoid over-fitting if you're back-testing (with dozens of techniques, most notably : test once and forget about parameter optimization) I would say that with paper trading and back-testing the serious problems are that : - your orders don't show up on the book so no one sees and reacts to your limit orders - your "filled" orders don't affect the book, so you're not affecting liquidity, so the market doesn't change in response to your trading - your bot has no access to market micro-structure strategies and conditional orders (and if you want to trade fast or are placing big trades you need them) These are the problems that make any simulation unrealistic, and they are fundamental. It's shadowboxing, which is not entirely devoid of value, but which is certainly insufficient on its own. (I've worked as a quant developing strategies for several funds these past 15 years)

deanCommie 6y ago

> You can trade ETFs that are tied to an index but an index is not a tradable instrument in of itself. Out of curiousity, what's the difference?