7 ms·

Are you trying to get hired by the financial industry, or is there some other reason for doing this? As you may well be aware, HFT is a scourge on the world's

Are you trying to get hired by the financial industry, or is there some other reason for doing this? As you may well be aware, HFT is a scourge on the world's economy, and it's a game only the biggest and best-connected players benefit from.

cmdkeen 13y ago
HFT isn't necessarily a scourge - if anything it discourages people from short term investing, which is a Good Thing. Long term investing isn't affected by HFT, except in providing more liquidity at the point you want to exit any position. If you're planning on holding for 5+ years you're competing against relatively few people and can do good research. Any shorter timeframe and you're competing against millions, against computers and against unknowables that will distort the price in the short term.
- white_devil 13y ago
  It's weird how this "providing liquidity" distraction never seems to go out of fashion. HFT provides liquidity if you happen to be another HFT algorithm. Otherwise, not so much. Why would short term investing be bad, and how long do you think HFT algorithms hold their "investments"? I hope you're not just trying to blow smoke up my ass here.
  - cmdkeen 13y ago
    It isn't just about liquidity, though it does. But it also reduces the bid/ask spread, i.e. price discovery. The HFT trader may make a tiny profit when it processes a trade from me, but I've benefited because I can buy/sell at a price close to the quoted exchange price, which didn't use to be the case. In terms of liquidity those trades are available to all, they love buying from non-HFT trades because they can make a small profit from their less accurate pricing. You benefit from the lower spreads and that liquidity. Before HFT you'd have paid a greater cost due to the spread on the price. Short term investing is "bad" because you're not investing in the future success of the company you're trading on the basis of what you think the market is going to do. To the extent to which that is ever knowable it is unlikely you have the skill, experience and data to be able to beat the large number of professionals doing it.
    - white_devil 13y ago
      It sounds like you work for the financial industry too. > The HFT trader You say it like it's a person making trades. > I've benefited because I can buy/sell at a price close to the quoted exchange price, which didn't use to be the case. What's a "quoted exchange price" if it's not the price you actually pay? > Short term investing is "bad" because you're not investing in the future success of the company you're trading on the basis of what you think the market is going to do. Isn't this what HFT is all about? Trading on insider information is bad, of course. But if you're on a level playing field, I don't see anything wrong with profiting from something you correctly predicted short-term.
      - cmdkeen 13y ago
        I do - but a 'proper' long term investor that cares about the price in 20 years time. The quoted exchange price is an average of the buy/sell offers available, there is no guarantee if you try and trade that you will pay that amount. Especially if you are shifting large blocks of stock. There is nothing wrong with "short term predicting" but it isn't investing. It is often a very easy way to lose money, "bad" doesn't just mean morally so. It comes down to time horizons, how much you want to make, and how many people are also trying to predict that event. You can correctly predict that event and still not make money as it is already priced in.
      - TheCoelacanth 13y ago
        > What's a "quoted exchange price" if it's not the price you actually pay? The actual price you have to pay is based on how much you want to buy and how much other people are willing to sell at what price. People who want to buy give a "bid" price and people who want to sell give an "ask" price. Whenever there is an ask price that is lower than a bid price, a sale takes place. This results in the lowest ask price always being higher than the highest bid price. The quoted exchange price will be somewhere in between the two prices. If you want to sell right away, the most you can get is the highest bid price on the books. If you want to buy right away, the cheapest you can get it is the lowest bid price on the books. This means that you have to pay more than the quoted exchange price to buy and receive less than the quoted exchange price to sell.
- zygomega 13y ago
  Long-term investors trade every day. You wake up and the market has gone down a few percent, you're going to rebalance at least. You decide to buy 30 year treasuries you're not going to hold them for thirty years - you're going to sell 29.5 year bonds and buy 30 year bonds in a few months time. And managers turn over long-term investor portfolios quickly. The average holding period for an SP500 company shareholder is 100 days. And every time a trade happens you run the risk of getting clipped by the faster guys who see you coming.
  - cmdkeen 13y ago
    But long term investors don't care about the "clip" that HFTs take, they care about the 80% rise in price over the next 5 years. Rebalancing portfolios again isn't something where getting a few basis points off the HFT best execution price is an issue. I work for a proper long term investor, and we trade over a period of days for the big trades. HFTs aren't even an issue apart from their occasional spectacular blowups that cause "could this happen to us?" memos from upstairs. Long term investors don't need to evolve from HFTs, they're more than happy making money not involving computers or trying to compete with them.
  - ycombobreaker 13y ago
    > And every time a trade happens you run the risk of getting clipped by the faster guys who see you coming. I think it is natural, but fallacious, to apply line-of-sight properties to trading. If you plan to trade, then there are two ways that another participant can "see you coming": 1. If you don't have direct market access, your order gets routed through a broker. The broker sees your order before it hits the market, and if he jumps in line ahead of you then that is front-running, and a Bad Thing. Your broker can get in a lot of trouble for this sort of thing. 2. If you are trying to move a large position by sending multiple orders to the market (one after another), then all market participants have the chance to react to the first order. It's really a game to try to move a lot of inventory at once, without tipping your hand to anyone else in the room. Thems the breaks. Nobody else gets to see anyone's order before the matching engine has already processed it, so there's no way to jump "ahead" of it. OTOH, maybe your long-term investor is trying to time the market: wait for a signal intraday, and pick that moment to send an order. In that case, if it is a good intraday signal then it is likely that someone else will compete. It is unlikely for a long-term trader to have spent as much on infrastructure as a HF trader, so the juicy signals will result in missed executions that _look_ like front-running.
zygomega 13y ago
No, I'm already in the finance industry. There are two main reasons I'm doing this: - I think the finance industry is very closed when it comes to intellectual property development, and an open source approach can be seriously competitive. An open approach may well be the future when it comes to being 'connected' - HFT is an interesting multi-disciplinary problem and the shear breadth of expertise required - modern chip design, low-latency software/hardware interaction, lock-free concurrency, fault-tolerant system design, adaptive learning algorithms, k-means clustering - means I'm learning heaps every day. I just don't agree that HFT is a scourge. It's an ecological shift (neither good or evil) and longer-horizon investors need to evolve.
- nezza-_- 13y ago
  No, long-horizon investors are normally overall not interested in HFT because they won't 'suffer' from it.
- zamalek 13y ago
  "HFT is an interesting multi-disciplinary problem" This makes this a very worthwhile project: whether or not anything comes of it (I considered writing technical analysis trading software for the exact same reason). Some documentation about the patterns and techniques you used would likely benefit the community a tiny bit more. Not everyone understands Haskell (or, forbid, has the time to peruse large codebases). Best of luck to you!
  - xradionut 13y ago
    There are many other interesting multi-disciplinary problems that are not as contentious as HFT and probably more publically acceptable and worthwhile. But if HFT is the domain you care about, go for it.
- white_devil 13y ago
  > an open source approach can be seriously competitive Do you really think any Joe Schmoe off the street could just grab your open source HFT and start making money with it? If not, how does your project benefit anyone? Are you working in the industry, but not in HFT? Maybe this is project is just practise for getting a job in HFT? I imagine that's where programmers get the fattest paychecks in the world. Only a fraction of what the sociopaths running the show get, of course, but good money nevertheless. > I just don't agree that HFT is a scourge. Good thing you're not at all biased.
topbanana 13y ago
> As you may well be aware, HFT is a scourge on the world's economy, and it's a game only the biggest and best-connected players benefit from. This is the line peddled by the popular press but it isn't true. HFT works to reduce spreads, which benefits both buyers and sellers.
- fnordfnordfnord 13y ago
  Latency arbing is a scourge, and makes limit orders practically useless. Quote stuffing, would be illegal if trading were still done on little slips of paper. Imagine dumping 10,000 slips of paper on the trading desk, and then shouting "just kidding!" In the words of Lawrence from Office Space, "You'd get your ass kicked" Or even worse, 10,000 empty bids. That's why people hate HFT.
  - kasey_junk 13y ago
    I'm not trying to be argumentative, but I am curious why you find latency arb so problematic, and what you think it's impact is on limit orders? I for one think latency arb is one of the bigger net wins for hft. As a market participant, each venue I have to maintain a presence at is a cost to me. I'm willing to pay the latency arb shops their cut to provide me price consistency because for my models it is much cheaper to do so than to continually reevaluate and redeploy to every possible venue. It frees me to shop for venues that provide the best features and fees. As for quote stuffing, you are absolutely right it is awful. That's why almost every venue out there has taken or is taking steps to curtail it. They did so because their customers agree with you.
    - dchichkov 13y ago
      'Quote stuffing' is there, because of absolutely ridiculous SEC rules. If you'd put in fractional prices and remove NBBO rules, there wouldn't be any 'quote stuffing'. And latency arb. What's wrong with latency arb? How else can you move information from one exchange to another fairly? Think of HFT's as of part of the financial network infrastructure. Where exchanges play role of nodes and HFT companies role of links/queues/buffers.
fnordfnordfnord 13y ago
>HFT is a scourge on the world's economy I'd point out that a lot of the things that are going on in HFT, are rehashes of old trading scams, and either are or would be illegal if there was any adult supervision. I was tempted to say something about the SEC being left behind the technology, and doesn't understand it. But for that to be true, and have this crap go on for so long, they either need to be complete fools, or they benefit from the status quo. >and it's a game only the biggest and best-connected players benefit from. That's definitely true in the case of latency arbitrage. But here again. It's the exchanges themselves that have decided that some players get an advantage over others. As an example, latency arbitrage, and a whole host of other problems could be sorted out by putting traders on exchange-hosted virtual machines. http://www.dailyfinance.com/2010/06/05/rigged-market-latency-arbitrage-3-billion/ http://www.dailyfinance.com/2010/06/05/rigged-market-latency... If not virtual machines, there are other ways to fix these problems, but first you need to get the players to acknowledge that there is a problem. http://www.zerohedge.com/article/do-it-yourself-latency-arbitrage-how-hfts-can-manipulate-nbbo-whim-courtesy-nyse-quote-stuff http://www.zerohedge.com/article/do-it-yourself-latency-arbi... Nanex has done a lot of good analysis of HFT, some of which is published here: nanex.net/FlashCrash/OngoingResearch.html
- ycombobreaker 13y ago
  Nanex is wonderful to read, but IMO they frequently misinterpret the data. Off the top of my head: - They only use consolidated feeds for US equities, never direct market-data feeds. The consolidated feed necessarily contains less information that direct feeds (to satisfy more stringent bandwidth requirements), which masks some "interesting" effects of how the exchanges publish their data. - They disregard that the CME feed publishes a fixed depth-of-book, and whenever they look at total liquidity in the book it can appear to flicker when deep levels fall "out" of the back of the book, even if liquidity is actually improving with the presence of a new inside level. - They make a big deal about wholesaler matching only occurring when the consolidated book is not locked. Their rationale is that subpenny prices are always wholesalers, and (erronously) therefore a lack of subpenny-priced trades must mean a lack of wholesale matching. These mistakes sound believable, but they do not hold up to any of scrutiny. Use their site to find interesting events, but be very careful about taking their conclusions at face value. Edit: bullet-list formatting

cmdkeen 13y ago

HFT isn't necessarily a scourge - if anything it discourages people from short term investing, which is a Good Thing. Long term investing isn't affected by HFT, except in providing more liquidity at the point you want to exit any position. If you're planning on holding for 5+ years you're competing against relatively few people and can do good research. Any shorter timeframe and you're competing against millions, against computers and against unknowables that will distort the price in the short term.

white_devil 13y ago
It's weird how this "providing liquidity" distraction never seems to go out of fashion. HFT provides liquidity if you happen to be another HFT algorithm. Otherwise, not so much. Why would short term investing be bad, and how long do you think HFT algorithms hold their "investments"? I hope you're not just trying to blow smoke up my ass here.
- cmdkeen 13y ago
  It isn't just about liquidity, though it does. But it also reduces the bid/ask spread, i.e. price discovery. The HFT trader may make a tiny profit when it processes a trade from me, but I've benefited because I can buy/sell at a price close to the quoted exchange price, which didn't use to be the case. In terms of liquidity those trades are available to all, they love buying from non-HFT trades because they can make a small profit from their less accurate pricing. You benefit from the lower spreads and that liquidity. Before HFT you'd have paid a greater cost due to the spread on the price. Short term investing is "bad" because you're not investing in the future success of the company you're trading on the basis of what you think the market is going to do. To the extent to which that is ever knowable it is unlikely you have the skill, experience and data to be able to beat the large number of professionals doing it.
  - white_devil 13y ago
    It sounds like you work for the financial industry too. > The HFT trader You say it like it's a person making trades. > I've benefited because I can buy/sell at a price close to the quoted exchange price, which didn't use to be the case. What's a "quoted exchange price" if it's not the price you actually pay? > Short term investing is "bad" because you're not investing in the future success of the company you're trading on the basis of what you think the market is going to do. Isn't this what HFT is all about? Trading on insider information is bad, of course. But if you're on a level playing field, I don't see anything wrong with profiting from something you correctly predicted short-term.
    - cmdkeen 13y ago
      I do - but a 'proper' long term investor that cares about the price in 20 years time. The quoted exchange price is an average of the buy/sell offers available, there is no guarantee if you try and trade that you will pay that amount. Especially if you are shifting large blocks of stock. There is nothing wrong with "short term predicting" but it isn't investing. It is often a very easy way to lose money, "bad" doesn't just mean morally so. It comes down to time horizons, how much you want to make, and how many people are also trying to predict that event. You can correctly predict that event and still not make money as it is already priced in.
    - TheCoelacanth 13y ago
      > What's a "quoted exchange price" if it's not the price you actually pay? The actual price you have to pay is based on how much you want to buy and how much other people are willing to sell at what price. People who want to buy give a "bid" price and people who want to sell give an "ask" price. Whenever there is an ask price that is lower than a bid price, a sale takes place. This results in the lowest ask price always being higher than the highest bid price. The quoted exchange price will be somewhere in between the two prices. If you want to sell right away, the most you can get is the highest bid price on the books. If you want to buy right away, the cheapest you can get it is the lowest bid price on the books. This means that you have to pay more than the quoted exchange price to buy and receive less than the quoted exchange price to sell.
zygomega 13y ago
Long-term investors trade every day. You wake up and the market has gone down a few percent, you're going to rebalance at least. You decide to buy 30 year treasuries you're not going to hold them for thirty years - you're going to sell 29.5 year bonds and buy 30 year bonds in a few months time. And managers turn over long-term investor portfolios quickly. The average holding period for an SP500 company shareholder is 100 days. And every time a trade happens you run the risk of getting clipped by the faster guys who see you coming.
- cmdkeen 13y ago
  But long term investors don't care about the "clip" that HFTs take, they care about the 80% rise in price over the next 5 years. Rebalancing portfolios again isn't something where getting a few basis points off the HFT best execution price is an issue. I work for a proper long term investor, and we trade over a period of days for the big trades. HFTs aren't even an issue apart from their occasional spectacular blowups that cause "could this happen to us?" memos from upstairs. Long term investors don't need to evolve from HFTs, they're more than happy making money not involving computers or trying to compete with them.
- ycombobreaker 13y ago
  > And every time a trade happens you run the risk of getting clipped by the faster guys who see you coming. I think it is natural, but fallacious, to apply line-of-sight properties to trading. If you plan to trade, then there are two ways that another participant can "see you coming": 1. If you don't have direct market access, your order gets routed through a broker. The broker sees your order before it hits the market, and if he jumps in line ahead of you then that is front-running, and a Bad Thing. Your broker can get in a lot of trouble for this sort of thing. 2. If you are trying to move a large position by sending multiple orders to the market (one after another), then all market participants have the chance to react to the first order. It's really a game to try to move a lot of inventory at once, without tipping your hand to anyone else in the room. Thems the breaks. Nobody else gets to see anyone's order before the matching engine has already processed it, so there's no way to jump "ahead" of it. OTOH, maybe your long-term investor is trying to time the market: wait for a signal intraday, and pick that moment to send an order. In that case, if it is a good intraday signal then it is likely that someone else will compete. It is unlikely for a long-term trader to have spent as much on infrastructure as a HF trader, so the juicy signals will result in missed executions that _look_ like front-running.

zygomega 13y ago

No, I'm already in the finance industry. There are two main reasons I'm doing this: - I think the finance industry is very closed when it comes to intellectual property development, and an open source approach can be seriously competitive. An open approach may well be the future when it comes to being 'connected' - HFT is an interesting multi-disciplinary problem and the shear breadth of expertise required - modern chip design, low-latency software/hardware interaction, lock-free concurrency, fault-tolerant system design, adaptive learning algorithms, k-means clustering - means I'm learning heaps every day. I just don't agree that HFT is a scourge. It's an ecological shift (neither good or evil) and longer-horizon investors need to evolve.

nezza-_- 13y ago
No, long-horizon investors are normally overall not interested in HFT because they won't 'suffer' from it.
zamalek 13y ago
"HFT is an interesting multi-disciplinary problem" This makes this a very worthwhile project: whether or not anything comes of it (I considered writing technical analysis trading software for the exact same reason). Some documentation about the patterns and techniques you used would likely benefit the community a tiny bit more. Not everyone understands Haskell (or, forbid, has the time to peruse large codebases). Best of luck to you!
- xradionut 13y ago
  There are many other interesting multi-disciplinary problems that are not as contentious as HFT and probably more publically acceptable and worthwhile. But if HFT is the domain you care about, go for it.
white_devil 13y ago
> an open source approach can be seriously competitive Do you really think any Joe Schmoe off the street could just grab your open source HFT and start making money with it? If not, how does your project benefit anyone? Are you working in the industry, but not in HFT? Maybe this is project is just practise for getting a job in HFT? I imagine that's where programmers get the fattest paychecks in the world. Only a fraction of what the sociopaths running the show get, of course, but good money nevertheless. > I just don't agree that HFT is a scourge. Good thing you're not at all biased.

topbanana 13y ago

> As you may well be aware, HFT is a scourge on the world's economy, and it's a game only the biggest and best-connected players benefit from. This is the line peddled by the popular press but it isn't true. HFT works to reduce spreads, which benefits both buyers and sellers.

fnordfnordfnord 13y ago
Latency arbing is a scourge, and makes limit orders practically useless. Quote stuffing, would be illegal if trading were still done on little slips of paper. Imagine dumping 10,000 slips of paper on the trading desk, and then shouting "just kidding!" In the words of Lawrence from Office Space, "You'd get your ass kicked" Or even worse, 10,000 empty bids. That's why people hate HFT.
- kasey_junk 13y ago
  I'm not trying to be argumentative, but I am curious why you find latency arb so problematic, and what you think it's impact is on limit orders? I for one think latency arb is one of the bigger net wins for hft. As a market participant, each venue I have to maintain a presence at is a cost to me. I'm willing to pay the latency arb shops their cut to provide me price consistency because for my models it is much cheaper to do so than to continually reevaluate and redeploy to every possible venue. It frees me to shop for venues that provide the best features and fees. As for quote stuffing, you are absolutely right it is awful. That's why almost every venue out there has taken or is taking steps to curtail it. They did so because their customers agree with you.
  - dchichkov 13y ago
    'Quote stuffing' is there, because of absolutely ridiculous SEC rules. If you'd put in fractional prices and remove NBBO rules, there wouldn't be any 'quote stuffing'. And latency arb. What's wrong with latency arb? How else can you move information from one exchange to another fairly? Think of HFT's as of part of the financial network infrastructure. Where exchanges play role of nodes and HFT companies role of links/queues/buffers.

fnordfnordfnord 13y ago

>HFT is a scourge on the world's economy I'd point out that a lot of the things that are going on in HFT, are rehashes of old trading scams, and either are or would be illegal if there was any adult supervision. I was tempted to say something about the SEC being left behind the technology, and doesn't understand it. But for that to be true, and have this crap go on for so long, they either need to be complete fools, or they benefit from the status quo. >and it's a game only the biggest and best-connected players benefit from. That's definitely true in the case of latency arbitrage. But here again. It's the exchanges themselves that have decided that some players get an advantage over others. As an example, latency arbitrage, and a whole host of other problems could be sorted out by putting traders on exchange-hosted virtual machines. http://www.dailyfinance.com/2010/06/05/rigged-market-latency-arbitrage-3-billion/ http://www.dailyfinance.com/2010/06/05/rigged-market-latency... If not virtual machines, there are other ways to fix these problems, but first you need to get the players to acknowledge that there is a problem. http://www.zerohedge.com/article/do-it-yourself-latency-arbitrage-how-hfts-can-manipulate-nbbo-whim-courtesy-nyse-quote-stuff http://www.zerohedge.com/article/do-it-yourself-latency-arbi... Nanex has done a lot of good analysis of HFT, some of which is published here: nanex.net/FlashCrash/OngoingResearch.html

ycombobreaker 13y ago
Nanex is wonderful to read, but IMO they frequently misinterpret the data. Off the top of my head: - They only use consolidated feeds for US equities, never direct market-data feeds. The consolidated feed necessarily contains less information that direct feeds (to satisfy more stringent bandwidth requirements), which masks some "interesting" effects of how the exchanges publish their data. - They disregard that the CME feed publishes a fixed depth-of-book, and whenever they look at total liquidity in the book it can appear to flicker when deep levels fall "out" of the back of the book, even if liquidity is actually improving with the presence of a new inside level. - They make a big deal about wholesaler matching only occurring when the consolidated book is not locked. Their rationale is that subpenny prices are always wholesalers, and (erronously) therefore a lack of subpenny-priced trades must mean a lack of wholesale matching. These mistakes sound believable, but they do not hold up to any of scrutiny. Use their site to find interesting events, but be very careful about taking their conclusions at face value. Edit: bullet-list formatting