5 ms·

"The trade occurred 19 seconds before the tweet, and one second after a headline appeared on the Dow Jones Newswire." I don't understand the mystery here. It

"The trade occurred 19 seconds before the tweet, and one second after a headline appeared on the Dow Jones Newswire." I don't understand the mystery here. It would be trivial to write a bot that could interpret a headline and make a trade on it in under a second, and that's without any special high-speed links. With API documentation for a brokerage house, I could write such a bot in probably half a day. Given the astronomical sums of money involved, I have a hard time believing that this isn't occurring on a daily basis.

xyzzy123 12y ago
Probably because you've never bet a lot of money on natural language processing before. Backtest and check alpha. cpercival's idea is pretty good though. I'm reminded of: https://xkcd.com/793/ https://xkcd.com/793/
- downandout 12y ago
  The headline of that news was "Intel in Talks to Buy Altera". You are acting like it would be difficult to write software that would be able to figure out what that means. It's not. These headlines also seem to follow a very specific pattern [1]. So if WSJ has a headline that X is in talks to buy Y, that's pretty much that. Also you could eliminate most of your risk by placing a limit order based upon the last trade prior to the news coming out. The market will either a) confirm that you are one of the first people to understand what has happened by filling your order at a price that doesn't reflect the news, locking in profits, or b) your limit order won't be filled because the price already is already above your limit, in which case you have assumed no risk. [1] https://www.google.com/#q=site:wsj.com+%22in+talks+to+buy%22 https://www.google.com/#q=site:wsj.com+%22in+talks+to+buy%22
  - catshirt 12y ago
    let us know how it works out, then. :)
    - deleted 12y ago
      [deleted]
  - 82xx 12y ago
    It would be easy to write software to parse that particular phrasing of that particular relationship ("Company X in Talks to Buy Company Y"), but to catch any arbitrary relationship between two entities in a given domain? That's no longer an easy problem. Not to mention that you'll need to represent that world knowledge coherently in ontologies / knowledge bases and write complex logic around it to make the data actionable.
    - barrkel 12y ago
      All you need is a positive correlation between an action taken on a weighted evaluation of a headline containing key words, and profit. You're overestimating the difficulty of the problem. It's model training; there's lots of historical data to tune on.
      - discardorama 12y ago
        > It's model training; there's lots of historical data to tune on. Is there a timestamped archive of DowJones Newswire articles? I found it difficult to find archived news articles the last time I was looking for them.
    - philjohn 12y ago
      Sentiment analysis has plenty of research around it, and once you've got a big enough training and validation set, it can give very good results. My old boss works for a company that runs sentiment analysis on comments made about companies to automatically highlight positive/negative messages - it's not a massive leap from there to repurpose that to analyse positive or negative news reports about a company.
      - deleted 12y ago
        [deleted]
- onewaystreet 12y ago
  These bots don't actually do much analysis. They just weight keywords and sources. "Buys" or "acquisition" in a WSJ headline has a pretty clear cut meaning. Financial headlines also tend to be pretty easy to parse (e.g. "X Buys Y").
  - giarc 12y ago
    Do you think "buys" would be less lucrative than "in talks to buy"? I imagine by the time company A buys company B, there has already been rumor and the markets have already moved to reflect that speculation.
ddeck 12y ago
Agreed. I don't really understand the surprise in article and wince at the "My friend reckons....." style. This is indeed occurring on a daily basis. A number of funds trade automatically based on fully automated analysis of the news wire and other sources. I believe most use in house analytics but there are 3rd party solutions: http://www.ravenpack.com/ http://www.ravenpack.com/ I think people outside the industry don't appreciate the depth and breath of automatic trading that is going on. Understandable given that it's in the interest of insiders to stay out of the spotlight, and the public mostly only sees the HFT headlines.
- IkmoIkmo 12y ago
  Yeah you remember a few weeks ago the Tesla stock moved $2 when Tesla announced its april fools new product? That was completely automated. It may not even have had to discern a new product announcement or launch. It may have just looked at twitter volume mentioning Tesla which doubled that day, and did sentiment analysis (much easier, which showed the communication that day was very positive), and purchased a bit of stock based on that. Who knows.
  - novaleaf 12y ago
    The irony is that likely those bots actually did make money. It's the other bots that later "jumped on the bandwagon" (and perhaps a couple humans) that probably lost big.
- xyzzy123 12y ago
  Oh yup, how come they're an analytics provider and not a fund? ;-p
  - maffydub 12y ago
    I don't know anything about the provider in question, but funds might integrate multiple data sources (e.g. the described tweet analysis, last financial results, competing companies) - in other words, a lot more than what an analytics provider needs to do. Maybe the analytics provider wants to focus on doing one thing and (hopefully) doing it well?
needhelpplz 12y ago
Well, if everybody started doing it, won't the returns fall to zero? Sort of like if arbitrage opportunities are present (in this case its a technological one), this would go away. I bet since writing this article the opportunities has pretty much slipped. Now anyone living close to the exchanges will do this.
- toxicFork 12y ago
  What if many people are not doing it because they think that many people are already doing it, and actually as a result of this, very few people are doing it?
  - needhelpplz 12y ago
    they would lose money and only to be replaced by others who think the same thing?
x0054 12y ago
Automatic trading is an automatic occurrence in deed, and many bots are design to respond to news, tweets, and all kinds of things. That said, if you really think you can write a bot that can do really good in the market, why not just make one. For the sake of argument, just assume that you can confirm a trade in 1-5 seconds, make a bot that just does fake trades, give it $10k in fake currency, and let it play. Come to think about it, such a system, with an API, would be really nice to have. Something that allows anyone to create a trading bot and see how it performs. Any one what to make one? :)
- IkmoIkmo 12y ago
  These APIs exist, and many of them allow two cool things: 1) Virtual money. So you can try out your bot and see how it'd do without risking money. (with $10k on say the Forex market which trades about 5 trillion per day, it doesn't matter if your trades are virtual) 2) Backlogging. So you can write a bot that'll analyse prices, volume, the orderbook etc of any moment in the past 6 months, which allows you to immediately simulate your bot's success in that market over a long period of time. (of course this is if you want to write a bot that analyses the exchange data, as opposed to things like news, for which you'll need to find your own database going back 6 months which isn't too hard btw, but a lot of data to process quickly).
- def_illiterate 12y ago
  I imagine there would be pretty good money in licensing an API. That's probably where the real money is--let the idiots make the trades, and you sell them the software.
  - x0054 12y ago
    Yep, it's like the someone else said in one of the other comments, don't dig for gold, sell buckets and the shovels to those who do. :)
    - noir_lord 12y ago
      Always heard that as "In a gold rush, be the one selling pickaxes"
stevewilhelm 12y ago
Most financial news feeds are distributed in a standard format. [1] https://iptc.org/standards/ https://iptc.org/standards/

xyzzy123 12y ago

Probably because you've never bet a lot of money on natural language processing before. Backtest and check alpha. cpercival's idea is pretty good though. I'm reminded of: https://xkcd.com/793/ https://xkcd.com/793/

downandout 12y ago
The headline of that news was "Intel in Talks to Buy Altera". You are acting like it would be difficult to write software that would be able to figure out what that means. It's not. These headlines also seem to follow a very specific pattern [1]. So if WSJ has a headline that X is in talks to buy Y, that's pretty much that. Also you could eliminate most of your risk by placing a limit order based upon the last trade prior to the news coming out. The market will either a) confirm that you are one of the first people to understand what has happened by filling your order at a price that doesn't reflect the news, locking in profits, or b) your limit order won't be filled because the price already is already above your limit, in which case you have assumed no risk. [1] https://www.google.com/#q=site:wsj.com+%22in+talks+to+buy%22 https://www.google.com/#q=site:wsj.com+%22in+talks+to+buy%22
- catshirt 12y ago
  let us know how it works out, then. :)
  - deleted 12y ago
    [deleted]
- 82xx 12y ago
  It would be easy to write software to parse that particular phrasing of that particular relationship ("Company X in Talks to Buy Company Y"), but to catch any arbitrary relationship between two entities in a given domain? That's no longer an easy problem. Not to mention that you'll need to represent that world knowledge coherently in ontologies / knowledge bases and write complex logic around it to make the data actionable.
  - barrkel 12y ago
    All you need is a positive correlation between an action taken on a weighted evaluation of a headline containing key words, and profit. You're overestimating the difficulty of the problem. It's model training; there's lots of historical data to tune on.
    - discardorama 12y ago
      > It's model training; there's lots of historical data to tune on. Is there a timestamped archive of DowJones Newswire articles? I found it difficult to find archived news articles the last time I was looking for them.
  - philjohn 12y ago
    Sentiment analysis has plenty of research around it, and once you've got a big enough training and validation set, it can give very good results. My old boss works for a company that runs sentiment analysis on comments made about companies to automatically highlight positive/negative messages - it's not a massive leap from there to repurpose that to analyse positive or negative news reports about a company.
    - deleted 12y ago
      [deleted]
onewaystreet 12y ago
These bots don't actually do much analysis. They just weight keywords and sources. "Buys" or "acquisition" in a WSJ headline has a pretty clear cut meaning. Financial headlines also tend to be pretty easy to parse (e.g. "X Buys Y").
- giarc 12y ago
  Do you think "buys" would be less lucrative than "in talks to buy"? I imagine by the time company A buys company B, there has already been rumor and the markets have already moved to reflect that speculation.

ddeck 12y ago

Agreed. I don't really understand the surprise in article and wince at the "My friend reckons....." style. This is indeed occurring on a daily basis. A number of funds trade automatically based on fully automated analysis of the news wire and other sources. I believe most use in house analytics but there are 3rd party solutions: http://www.ravenpack.com/ http://www.ravenpack.com/ I think people outside the industry don't appreciate the depth and breath of automatic trading that is going on. Understandable given that it's in the interest of insiders to stay out of the spotlight, and the public mostly only sees the HFT headlines.

IkmoIkmo 12y ago
Yeah you remember a few weeks ago the Tesla stock moved $2 when Tesla announced its april fools new product? That was completely automated. It may not even have had to discern a new product announcement or launch. It may have just looked at twitter volume mentioning Tesla which doubled that day, and did sentiment analysis (much easier, which showed the communication that day was very positive), and purchased a bit of stock based on that. Who knows.
- novaleaf 12y ago
  The irony is that likely those bots actually did make money. It's the other bots that later "jumped on the bandwagon" (and perhaps a couple humans) that probably lost big.
xyzzy123 12y ago
Oh yup, how come they're an analytics provider and not a fund? ;-p
- maffydub 12y ago
  I don't know anything about the provider in question, but funds might integrate multiple data sources (e.g. the described tweet analysis, last financial results, competing companies) - in other words, a lot more than what an analytics provider needs to do. Maybe the analytics provider wants to focus on doing one thing and (hopefully) doing it well?

needhelpplz 12y ago

Well, if everybody started doing it, won't the returns fall to zero? Sort of like if arbitrage opportunities are present (in this case its a technological one), this would go away. I bet since writing this article the opportunities has pretty much slipped. Now anyone living close to the exchanges will do this.

toxicFork 12y ago
What if many people are not doing it because they think that many people are already doing it, and actually as a result of this, very few people are doing it?
- needhelpplz 12y ago
  they would lose money and only to be replaced by others who think the same thing?

x0054 12y ago

Automatic trading is an automatic occurrence in deed, and many bots are design to respond to news, tweets, and all kinds of things. That said, if you really think you can write a bot that can do really good in the market, why not just make one. For the sake of argument, just assume that you can confirm a trade in 1-5 seconds, make a bot that just does fake trades, give it $10k in fake currency, and let it play. Come to think about it, such a system, with an API, would be really nice to have. Something that allows anyone to create a trading bot and see how it performs. Any one what to make one? :)

IkmoIkmo 12y ago
These APIs exist, and many of them allow two cool things: 1) Virtual money. So you can try out your bot and see how it'd do without risking money. (with $10k on say the Forex market which trades about 5 trillion per day, it doesn't matter if your trades are virtual) 2) Backlogging. So you can write a bot that'll analyse prices, volume, the orderbook etc of any moment in the past 6 months, which allows you to immediately simulate your bot's success in that market over a long period of time. (of course this is if you want to write a bot that analyses the exchange data, as opposed to things like news, for which you'll need to find your own database going back 6 months which isn't too hard btw, but a lot of data to process quickly).
def_illiterate 12y ago
I imagine there would be pretty good money in licensing an API. That's probably where the real money is--let the idiots make the trades, and you sell them the software.
- x0054 12y ago
  Yep, it's like the someone else said in one of the other comments, don't dig for gold, sell buckets and the shovels to those who do. :)
  - noir_lord 12y ago
    Always heard that as "In a gold rush, be the one selling pickaxes"

stevewilhelm 12y ago

Most financial news feeds are distributed in a standard format. [1] https://iptc.org/standards/ https://iptc.org/standards/