3 ms·

If quote stuffing is a bug that harms the people carrying it out, then why don't HFT programs have a rate limiter on them that make them stop? They do. But a r

If quote stuffing is a bug that harms the people carrying it out, then why don't HFT programs have a rate limiter on them that make them stop? They do. But a rate limiter is just another signal - you still need to set the rate limit high enough that you can correctly respond to market conditions, but low enough that you don't go wild. "Never, ever do it" is like "zero tolerance for bugs". It's a goal to strive for, but it's not achievable.

mikeash 13y ago
It's completely achievable. Rate limit to one trade per second. Done. You don't want to do that, because it doesn't let you make as much money as you otherwise could? I totally understand. But in that case, quote stuffing is not a bug, it's a deliberate tradeoff. But I suppose that "we let this happen on purpose because it allows higher profits, even though this particular behavior doesn't help us" doesn't sound as nice.
- yummyfajitas 13y ago
  The point I'm making is that it's undesirable behavior that doesn't directly help the HFT. If they could stop it without going to ridiculous lengths like your proposed "give up trading entirely", they would. And given that quote stuffing doesn't actually hurt anyone, it's not like stopping it is a high priority.
  - mikeash 13y ago
    Well, the point the article is making is that quote stuffing is a bug, but it's not. It's a side effect, but one that's allowed on purpose, because it's necessary to do the things they want to do. I have no strong opinion about HFT. I don't have a lot of interest in the stock market in general. I do, however, have a low tolerance for weaselly language and misdirection.
    - yummyfajitas 13y ago
      It's a side effect, but one that's allowed on purpose... Similarly, if you build a system on MySQL and later see dates like `0000-00-00` (yes, really), that's not a bug. That's a "side effect, but one that's allowed on purpose". Pretty much all bugs fall into that category - after all, you could have spent more time on formal verification/testing or just wrote the whole thing in Coq. I'm sorry you found the language "weaselly". My sole intent here was explaining the mechanics behind quote stuffing. How would you prefer I phrased it?
      - mikeash 13y ago
        I'm afraid I'm not familiar enough with MySQL to understand the analogy you're making there. As for wording, don't call it a bug. Say that it's a necessary side effect of making money with HFT. It's an inherent design limitation. The word "bug" implies that it can be fixed, that you can theoretically do HFT at the same speed as you are now without any quote stuffing if only you had perfect code. Let's take a vaguely similar example of streaming audio. I worked on a near-realtime streaming audio product for a while so it's something I'm familiar with. Latency and reliability trade off pretty directly. The product I worked on defaulted to a latency of two seconds. This made it pretty robust against network hiccups like wifi interference, but the substantial lag made it annoying or just plain unsuitable for some applications. It was possible to adjust the latency (although this was an advanced technique to the extent that we hid the setting pretty thoroughly) and you could get it down to a fraction of a second and still have the system work overall, but it would be much more prone to dropouts due to network problems. Now, there are some things that could be done to improve this problem absolutely, like adding forward error correction to the protocol, but nothing is going to eliminate the tradeoff entirely. Because of that, I'd never describe it as a bug. If someone complained about the latency, I told them that it was an unfortunate but necessary consequence of ensuring reliability. The latency can be eliminated easily, but at the cost of less reliability. Likewise here, quote stuffing is an unfortunate but necessary consequence of HFTs. It could be lessened or eliminated by making HFTs less HF, but you don't want to do that, because you feel that the Hest possible F is more important. I call the writing weaselly because it portrays quote stuffing as completely unavoidable, but the real scenario is that quote stuffing is completely avoidable, it's just that you prefer to make more money instead of stopping it.
  - gd1 13y ago
    >And given that quote stuffing doesn't actually hurt anyone, it's not like stopping it is a high priority. It does hurt people in some way. It increases the amount of data we need to handle. More storage space and more processing required. And you've been a bit disingenuous in your article - there is a theory behind the "2.???" part. The theory is that slower participants are overwhelmed by the quote traffic and end up lagging, and are then taken advantage of. I don't personally believe it, and I'm sure 99% of these things are just feedback loops, but it is theoretically possible. The faster exchanges are cycling in say 20-30 micros these days, so any participant who can't match that rate will easily fall behind. Who knows, weaker participants may still be taking 1-2 millis to handle an update. So just before some known vol event, say an economic figure, you flood the exchange and lag the other participants by say 50 millis, then pick them off after the new information arrives.
    - yummyfajitas 13y ago
      There are traders A and B in addition to yourself. A is as fast as you, B is slower. How does it help you to slow yourself and C down? That just means A will beat both of you.
      - gd1 13y ago
        You send your burst a few milliseconds before an event. It chokes up C (all of those depth updates sit in his queue and he's busy for the next 50 millis churning through them). Then both you and A are free to pick him off. The thing is, you know to ignore the burst on the public feed, since you know it is coming (you created it). You still have to process the order book updates, but you can skip your decision logic. So any participant who has an internal latency (packet decode + orderbook update + decision logic) greater than 1 exchange cycle, and who doesn't have an "out" clause to stop making decisions if it is falling behind, will be vulnerable. Like I said, I don't buy it, but it is possible in theory.
        yummyfajitas 13y ago
        What prevents A from "picking me off" (which I assume means "get to the top of the queue")? Also, "know to ignore the burst" doesn't mean you don't have to process it. It merely means you need to subtract the result of the burst from your signals.
- minimax 13y ago
  You don't have to be that heavy handed about it. One simple check is to see if your new order is basically the same as the last order you just canceled (same price and side of the market). If it is, and it has been less than some delta, don't send the new order. I think this sort of duplicate detection is relatively common among HFTs but I could be wrong.

mikeash 13y ago

It's completely achievable. Rate limit to one trade per second. Done. You don't want to do that, because it doesn't let you make as much money as you otherwise could? I totally understand. But in that case, quote stuffing is not a bug, it's a deliberate tradeoff. But I suppose that "we let this happen on purpose because it allows higher profits, even though this particular behavior doesn't help us" doesn't sound as nice.

yummyfajitas 13y ago
The point I'm making is that it's undesirable behavior that doesn't directly help the HFT. If they could stop it without going to ridiculous lengths like your proposed "give up trading entirely", they would. And given that quote stuffing doesn't actually hurt anyone, it's not like stopping it is a high priority.
- mikeash 13y ago
  Well, the point the article is making is that quote stuffing is a bug, but it's not. It's a side effect, but one that's allowed on purpose, because it's necessary to do the things they want to do. I have no strong opinion about HFT. I don't have a lot of interest in the stock market in general. I do, however, have a low tolerance for weaselly language and misdirection.
  - yummyfajitas 13y ago
    It's a side effect, but one that's allowed on purpose... Similarly, if you build a system on MySQL and later see dates like `0000-00-00` (yes, really), that's not a bug. That's a "side effect, but one that's allowed on purpose". Pretty much all bugs fall into that category - after all, you could have spent more time on formal verification/testing or just wrote the whole thing in Coq. I'm sorry you found the language "weaselly". My sole intent here was explaining the mechanics behind quote stuffing. How would you prefer I phrased it?
    - mikeash 13y ago
      I'm afraid I'm not familiar enough with MySQL to understand the analogy you're making there. As for wording, don't call it a bug. Say that it's a necessary side effect of making money with HFT. It's an inherent design limitation. The word "bug" implies that it can be fixed, that you can theoretically do HFT at the same speed as you are now without any quote stuffing if only you had perfect code. Let's take a vaguely similar example of streaming audio. I worked on a near-realtime streaming audio product for a while so it's something I'm familiar with. Latency and reliability trade off pretty directly. The product I worked on defaulted to a latency of two seconds. This made it pretty robust against network hiccups like wifi interference, but the substantial lag made it annoying or just plain unsuitable for some applications. It was possible to adjust the latency (although this was an advanced technique to the extent that we hid the setting pretty thoroughly) and you could get it down to a fraction of a second and still have the system work overall, but it would be much more prone to dropouts due to network problems. Now, there are some things that could be done to improve this problem absolutely, like adding forward error correction to the protocol, but nothing is going to eliminate the tradeoff entirely. Because of that, I'd never describe it as a bug. If someone complained about the latency, I told them that it was an unfortunate but necessary consequence of ensuring reliability. The latency can be eliminated easily, but at the cost of less reliability. Likewise here, quote stuffing is an unfortunate but necessary consequence of HFTs. It could be lessened or eliminated by making HFTs less HF, but you don't want to do that, because you feel that the Hest possible F is more important. I call the writing weaselly because it portrays quote stuffing as completely unavoidable, but the real scenario is that quote stuffing is completely avoidable, it's just that you prefer to make more money instead of stopping it.
- gd1 13y ago
  >And given that quote stuffing doesn't actually hurt anyone, it's not like stopping it is a high priority. It does hurt people in some way. It increases the amount of data we need to handle. More storage space and more processing required. And you've been a bit disingenuous in your article - there is a theory behind the "2.???" part. The theory is that slower participants are overwhelmed by the quote traffic and end up lagging, and are then taken advantage of. I don't personally believe it, and I'm sure 99% of these things are just feedback loops, but it is theoretically possible. The faster exchanges are cycling in say 20-30 micros these days, so any participant who can't match that rate will easily fall behind. Who knows, weaker participants may still be taking 1-2 millis to handle an update. So just before some known vol event, say an economic figure, you flood the exchange and lag the other participants by say 50 millis, then pick them off after the new information arrives.
  - yummyfajitas 13y ago
    There are traders A and B in addition to yourself. A is as fast as you, B is slower. How does it help you to slow yourself and C down? That just means A will beat both of you.
    - gd1 13y ago
      You send your burst a few milliseconds before an event. It chokes up C (all of those depth updates sit in his queue and he's busy for the next 50 millis churning through them). Then both you and A are free to pick him off. The thing is, you know to ignore the burst on the public feed, since you know it is coming (you created it). You still have to process the order book updates, but you can skip your decision logic. So any participant who has an internal latency (packet decode + orderbook update + decision logic) greater than 1 exchange cycle, and who doesn't have an "out" clause to stop making decisions if it is falling behind, will be vulnerable. Like I said, I don't buy it, but it is possible in theory.
      - yummyfajitas 13y ago
        What prevents A from "picking me off" (which I assume means "get to the top of the queue")? Also, "know to ignore the burst" doesn't mean you don't have to process it. It merely means you need to subtract the result of the burst from your signals.
minimax 13y ago
You don't have to be that heavy handed about it. One simple check is to see if your new order is basically the same as the last order you just canceled (same price and side of the market). If it is, and it has been less than some delta, don't send the new order. I think this sort of duplicate detection is relatively common among HFTs but I could be wrong.