6 ms·

"Artificial Intelligence" Gains Fans Among Investors

32 points by ctkrohn 16y ago

captaincrunch 16y ago
I have my hand at this now with http://www.algxchange.com http://www.algxchange.com - its really cool, but training is a nightmare. I have 4 situations that required training, and an actual decision model that needed training on what situation model to use. So many factors, Bear market, Bull market, etc...
motters 16y ago
Having tried to do this sort of thing in the past, I'm fairly skeptical. Maybe if you're trading in large volumes over short spaces of time you can make money with AI, but at least in my simulated runs with small scale investment the results didn't look encouraging. Also it should be borne in mind that this kind of activity isn't really making any constructive or lasting contribution to society. It's really just moving money around slightly more efficiently than previously. If you really want to change the world, do something else.
- dmix 16y ago
  > this kind of activity isn't really making any constructive or lasting contribution to society There was an interesting discussion about this on quora: http://www.quora.com/Is-high-frequency-trading-good-for-the-economy http://www.quora.com/Is-high-frequency-trading-good-for-the-...
- vecter 16y ago
  Also it should be borne in mind that this kind of activity isn't really making any constructive or lasting contribution to society. It's really just moving money around slightly more efficiently than previously. I disagree. Efficient allocation of capital is what allowed modern society to advance as far as it did. Capital markets are a tremendous innovation and add tons of value to society. Almost every startup turned major corporation out there was capitalized by the financial markets. You trivialize this value when you say investing is simply "moving money around more efficiently". That money NEEDS to be moved around more efficiently so the most productive members and organizations in society are (a) given the resources to expand their business and (b) rewarded for the value they add to society.
  - cameldrv 16y ago
    Perhaps, but most of those funding decisions were not made in the public markets. Those capital allocations were made by angels, VCs, and the people who committed to investing in the IPO of the company. The only thing that the overall stock market directly controls for capital allocation is secondary offerings, which are a very small part of the allocation of productive capital. The stock market just serves to trade pieces of capital that already exist. Having the stock market be a bit more accurate, particularly on short timescales, doesn't really affect capital allocation.
  - jey 16y ago
    He isn't disputing the fact that it has net-positive value creation. He's just saying that it's a tiny lever in terms of impact. You'll immeasurably increase market efficiency, but you'll still get a lot of money because finance is an area with a very high money-to-value ratio.
  - JabavuAdams 16y ago
    Yes, increased liquidity has allowed us to more efficiently raise capital, but beware the liquidity dogma. Increased liquidity has been good in the past, so anything that increases liquidity must be good. Not necessarily. What are the other effects? How much do we value those? Can we even estimate them a priori? OTOH, I would argue that most web apps proposed on this site aren't really making any lasting or constructive contribution to society. The one thing that can be said is that web-startup founders as a whole are less powerful and perhaps less cynically self-serving than financiers, so they distort government policy less.
- lkozma 16y ago
  "isn't really making any constructive or lasting contribution to society" The common argument is that if you are increasing overall efficiency, that means that capital can be more readily available for real businesses which do make lasting contributions (often in the form of polluting the environment, but that's another question). It seems your argument is a bit simplistic. Couldn't the same logic be applied to the distribution of goods, middlemen, marketplaces, banking, etc. Are you dismissing all those?
  - dagheti 16y ago
    This is true only if we accept that AI trading is actually making the market more efficient. The 'wisdom' of crowds is often foolishness, and this could be even more true for a crowd of computers. It is very possible that by using AI that looks mostly at numbers for trading is just destabilizing by relying too heavily on specific variables (the ones the algorithms can actually use as inputs) , and reduces efficient allocation of capital.
    - vecter 16y ago
      This is false. You can't make money by being consistently wrong about the "true" value of an asset. Then, the only way you can stay in business is to sell the asset when it's too high, and buy it when it's too low. Notice how doing this actually decreases the amplitude of the oscillations. If you're selling at a high point, you're pushing the price back down. Likewise, if you're buying at a low point, you're pushing the price back up (both times, towards its "true" value). The reason I use "true" in quotes is because no one really knows the value of an asset. The only real estimate we have is the market estimate. Let me put it another way: if you don't believe markets efficiently allocate capital, then how would you propose we do it? By fiat? Once again, I want to emphasize that you cannot be wrong about the value of an asset and consistently make money. If these AI algorithms are consistently making money, then they are, more often than not, accurately predicting the value of that asset (where value is defined as the future market price). Of course, my argument hinges on the market price being "right", but like I said, markets are the best way we know how to value things.
- orp 16y ago
  'Also it should be borne in mind that this kind of activity isn't really making any constructive or lasting contribution to society. It's really just moving money around slightly more efficiently than previously. If you really want to change the world, do something else.' I agree completely. While have reasonably efficient capital markets is necessary for society, its very hard to show that the marginal improvement offered by this kind of activity justifies the massive amounts of effort that go into it, just because: 1. It's cool. 2. It can make a lot of money. But unlike making money out of making something people want (e.g. providing value), here you're making money (if you succeed) out of very small market inefficiencies that are probably not hurting anyone, or even just by being 1ms faster then the other guys who were already catching them anyway,
- wisty 16y ago
  Putting Wall Street traders out of work seems like an efficient use of capital ;)
  - yummyfajitas 16y ago
    Putting anyone out of work is an efficient use of capital.
Tichy 16y ago
Thinking about it, if an investment fund has a strategy, maybe it should really be possible to put it into code after all. If not, then what is the basis of the investment fund - the gut feeling of it's managers? You could also still add the gut feeling of the managers as one input variable for the system. Probably even then there would be lots of things left to automate. Just because it is AI doesn't mean it has to do everything by itself.
- HSO 16y ago
  Yes in principle. Except that those who have a gut feeling for markets and economics usually are not the same as those with technical acumen. Usually, those with technical acumen find manipulating economic and financial data by hand, and doing so repeatedly, boring. Which, I think, is necessary to gain intuition. On the other hand, those who are willing to do this and perhaps even enjoy studying balance sheets and cash flow statements and product markets spend so much time doing this that they wouldn't have time to learn math and science and AI. In other words, you will always have either/or guys. Those who understand the algos and those who understand the markets. Which, by the way, doesn't keep me from trying ;)
sethg 16y ago
If human beings can’t understand the rationale that an AI-driven hedge fund is using to make its trades, then how can humans tell the difference between an AI that has actually figured out a strategy to beat the market and an AI that is just having a run of good luck?
- vecter 16y ago
  When you have lots of samples, you're usually pretty certain.
reader5000 16y ago
And of course as with all AI and investing, there is huge incentive for the stuff that works to remain secret.
cageface 16y ago
Isn't the underlying assumption behind all these machine learning models that an incomprehensibly complex system can be modeled with a far simpler approximation of that system? I only have limited experience with ML techniques but my impression was that they're all essentially statistical and the essence of a statistical approach is that you don't really model the underlying mechanics. I'm not saying that this approach can't work, but just that an insanely complex macro-system like the market is never really going to be reducible to a tractable statistical model.
- reader5000 16y ago
  The complexity of your statistical model is arbitrary. For example agent-based models can consist of hundreds of thousands of heterogenous agents interacting in an economy (fitted to real data). The only requirement for a predictive model is it computes faster than reality.
  - cageface 16y ago
    Sure but that's still a fairly coarse approximation of the actual problem domain in the case of the market, right? You're making an educated guess about how much of that raw information you can ignore. You probably can ignore a lot of it but I wouldn't be surprised if a lot of the serious left-turns in the market in the last century wouldn't have been predicted by the same models that could track day-to-day trends well.
Vivtek 16y ago
Oh, great. AI will be over-hyped again, just like in the 80's, and we'll have to call it something else for a decade when our families ask us what we're doing lately.
- wheaties 16y ago
  Yes, it's true but 1) there will be a renewed interest in the science with more funding leading to more discoveries which could, potentially benefit man. Who knows? 2) it may also be derided much like in the 80's when there were many funds employing the same "fail safe" measures to guard against loss.
tkahnoski 16y ago
Hypothetical: What if an AI driven investment fund significantly out performed other funds? The focus on number crunching seems contrary to advice given by Warren Buffet and other successful investors about investing in what you know. i.e. "I think the market is under-estimating the impact of Product X, I'll invest in that company" versus "Analyze market cap, yearly revenue...."
- icey 16y ago
  I've always wondered if there was a way to make some money in the market by performing lexical analysis of news articles about stocks and gauging the performance of those stocks based on the contents of the article. So, you'd train your software by analyzing every news article you could find over the past N years for every company in the market you're targeting. It would look for words or phrases that occurred with higher frequency a few days before stocks made significant moves in either direction. I suppose you'd probably want to weight the source as well, articles sourced from the WSJ might earn a better reputation than ones from the NY Post for example. At some point there would hopefully be enough confidence in the data that you could test it out with $1000 or something small; news-driven automation... There are enough people who buy and sell on news and opinion pieces that there may be enough fat there to make some profits.
  - tkahnoski 16y ago
    There was actually an interesting study done on quarterly earnings releases that had some interesting implications as far as "soft data" versus hard numbers: http://ideas.repec.org/p/fip/fedgif/951.html http://ideas.repec.org/p/fip/fedgif/951.html There was a company "Quant the News" that launched a product that did sentiment analysis on company related news, but their website no longer exists. http://www.crunchbase.com/company/quant-the-news http://www.crunchbase.com/company/quant-the-news
T_S_ 16y ago
Market prediction is one of the hardest areas for machine learning/statistics/AI to prove their value. What the system can do is monitor more inputs than a single trader, so the screening process can be scaled up--provided you have relevant inputs. With more signals becoming available electronically it is possible to automatically incorporate non-market prices into trading decisions. All this will require lots of non-ML engineering to supply the data (just like Google search). When monthly-issued government economic statistic stop moving the market you'll know we are there.
- HSO 16y ago
  The reason for this may be that the market is made up of interacting learning machines (if you subsume humans among them). Thus, every algorithm introduced changes the basis on whcih the algorithm was selected. Talk about moving targets...
  - T_S_ 16y ago
    I agree, and the word I would use is competing machines. Every time me (or my algo) bid up a security price I remove that "cheapness" signal from a competitor, increasing the noise they have to deal with and vice versa. ML hates noisy data. Returns are very noisy. ML has a much easier time with data generated from situations that are lacking competition, such as data from any process inside a company, e.g. consumer behavior. These are problems that want to be solved.
dododo 16y ago
it seems people call these techniques machine learning/AI simply because it sounds cooler: a lot of the techniques have a 30+ year history in statistics. this is the same story as neural networks: neural networks are just nested logistic regression, a technique with much history in statistics. a lot of high frequency trading is simply linear/logistic regression (on the right features of course). anything more complicated is too slow.
- jimbokun 16y ago
  http://www-stat.stanford.edu/~tibs/stat315a/glossary.pdf http://www-stat.stanford.edu/~tibs/stat315a/glossary.pdf Note entries for "large grant" and "nice place to have a meeting."
- deleted 16y ago
  [deleted]
apu 16y ago
Hello, submarine! http://www.paulgraham.com/submarine.html http://www.paulgraham.com/submarine.html

32 points by ctkrohn 16y ago

I have my hand at this now with http://www.algxchange.com http://www.algxchange.com - its really cool, but training is a nightmare. I have 4 situations that required training, and an actual decision model that needed training on what situation model to use. So many factors, Bear market, Bull market, etc...

motters 16y ago

Having tried to do this sort of thing in the past, I'm fairly skeptical. Maybe if you're trading in large volumes over short spaces of time you can make money with AI, but at least in my simulated runs with small scale investment the results didn't look encouraging. Also it should be borne in mind that this kind of activity isn't really making any constructive or lasting contribution to society. It's really just moving money around slightly more efficiently than previously. If you really want to change the world, do something else.

dmix 16y ago
> this kind of activity isn't really making any constructive or lasting contribution to society There was an interesting discussion about this on quora: http://www.quora.com/Is-high-frequency-trading-good-for-the-economy http://www.quora.com/Is-high-frequency-trading-good-for-the-...
vecter 16y ago
Also it should be borne in mind that this kind of activity isn't really making any constructive or lasting contribution to society. It's really just moving money around slightly more efficiently than previously. I disagree. Efficient allocation of capital is what allowed modern society to advance as far as it did. Capital markets are a tremendous innovation and add tons of value to society. Almost every startup turned major corporation out there was capitalized by the financial markets. You trivialize this value when you say investing is simply "moving money around more efficiently". That money NEEDS to be moved around more efficiently so the most productive members and organizations in society are (a) given the resources to expand their business and (b) rewarded for the value they add to society.
- cameldrv 16y ago
  Perhaps, but most of those funding decisions were not made in the public markets. Those capital allocations were made by angels, VCs, and the people who committed to investing in the IPO of the company. The only thing that the overall stock market directly controls for capital allocation is secondary offerings, which are a very small part of the allocation of productive capital. The stock market just serves to trade pieces of capital that already exist. Having the stock market be a bit more accurate, particularly on short timescales, doesn't really affect capital allocation.
- jey 16y ago
  He isn't disputing the fact that it has net-positive value creation. He's just saying that it's a tiny lever in terms of impact. You'll immeasurably increase market efficiency, but you'll still get a lot of money because finance is an area with a very high money-to-value ratio.
- JabavuAdams 16y ago
  Yes, increased liquidity has allowed us to more efficiently raise capital, but beware the liquidity dogma. Increased liquidity has been good in the past, so anything that increases liquidity must be good. Not necessarily. What are the other effects? How much do we value those? Can we even estimate them a priori? OTOH, I would argue that most web apps proposed on this site aren't really making any lasting or constructive contribution to society. The one thing that can be said is that web-startup founders as a whole are less powerful and perhaps less cynically self-serving than financiers, so they distort government policy less.
lkozma 16y ago
"isn't really making any constructive or lasting contribution to society" The common argument is that if you are increasing overall efficiency, that means that capital can be more readily available for real businesses which do make lasting contributions (often in the form of polluting the environment, but that's another question). It seems your argument is a bit simplistic. Couldn't the same logic be applied to the distribution of goods, middlemen, marketplaces, banking, etc. Are you dismissing all those?
- dagheti 16y ago
  This is true only if we accept that AI trading is actually making the market more efficient. The 'wisdom' of crowds is often foolishness, and this could be even more true for a crowd of computers. It is very possible that by using AI that looks mostly at numbers for trading is just destabilizing by relying too heavily on specific variables (the ones the algorithms can actually use as inputs) , and reduces efficient allocation of capital.
  - vecter 16y ago
    This is false. You can't make money by being consistently wrong about the "true" value of an asset. Then, the only way you can stay in business is to sell the asset when it's too high, and buy it when it's too low. Notice how doing this actually decreases the amplitude of the oscillations. If you're selling at a high point, you're pushing the price back down. Likewise, if you're buying at a low point, you're pushing the price back up (both times, towards its "true" value). The reason I use "true" in quotes is because no one really knows the value of an asset. The only real estimate we have is the market estimate. Let me put it another way: if you don't believe markets efficiently allocate capital, then how would you propose we do it? By fiat? Once again, I want to emphasize that you cannot be wrong about the value of an asset and consistently make money. If these AI algorithms are consistently making money, then they are, more often than not, accurately predicting the value of that asset (where value is defined as the future market price). Of course, my argument hinges on the market price being "right", but like I said, markets are the best way we know how to value things.
orp 16y ago
'Also it should be borne in mind that this kind of activity isn't really making any constructive or lasting contribution to society. It's really just moving money around slightly more efficiently than previously. If you really want to change the world, do something else.' I agree completely. While have reasonably efficient capital markets is necessary for society, its very hard to show that the marginal improvement offered by this kind of activity justifies the massive amounts of effort that go into it, just because: 1. It's cool. 2. It can make a lot of money. But unlike making money out of making something people want (e.g. providing value), here you're making money (if you succeed) out of very small market inefficiencies that are probably not hurting anyone, or even just by being 1ms faster then the other guys who were already catching them anyway,
wisty 16y ago
Putting Wall Street traders out of work seems like an efficient use of capital ;)
- yummyfajitas 16y ago
  Putting anyone out of work is an efficient use of capital.

Tichy 16y ago

Thinking about it, if an investment fund has a strategy, maybe it should really be possible to put it into code after all. If not, then what is the basis of the investment fund - the gut feeling of it's managers? You could also still add the gut feeling of the managers as one input variable for the system. Probably even then there would be lots of things left to automate. Just because it is AI doesn't mean it has to do everything by itself.

HSO 16y ago
Yes in principle. Except that those who have a gut feeling for markets and economics usually are not the same as those with technical acumen. Usually, those with technical acumen find manipulating economic and financial data by hand, and doing so repeatedly, boring. Which, I think, is necessary to gain intuition. On the other hand, those who are willing to do this and perhaps even enjoy studying balance sheets and cash flow statements and product markets spend so much time doing this that they wouldn't have time to learn math and science and AI. In other words, you will always have either/or guys. Those who understand the algos and those who understand the markets. Which, by the way, doesn't keep me from trying ;)

sethg 16y ago

If human beings can’t understand the rationale that an AI-driven hedge fund is using to make its trades, then how can humans tell the difference between an AI that has actually figured out a strategy to beat the market and an AI that is just having a run of good luck?

vecter 16y ago
When you have lots of samples, you're usually pretty certain.

reader5000 16y ago

And of course as with all AI and investing, there is huge incentive for the stuff that works to remain secret.

cageface 16y ago

Isn't the underlying assumption behind all these machine learning models that an incomprehensibly complex system can be modeled with a far simpler approximation of that system? I only have limited experience with ML techniques but my impression was that they're all essentially statistical and the essence of a statistical approach is that you don't really model the underlying mechanics. I'm not saying that this approach can't work, but just that an insanely complex macro-system like the market is never really going to be reducible to a tractable statistical model.

reader5000 16y ago
The complexity of your statistical model is arbitrary. For example agent-based models can consist of hundreds of thousands of heterogenous agents interacting in an economy (fitted to real data). The only requirement for a predictive model is it computes faster than reality.
- cageface 16y ago
  Sure but that's still a fairly coarse approximation of the actual problem domain in the case of the market, right? You're making an educated guess about how much of that raw information you can ignore. You probably can ignore a lot of it but I wouldn't be surprised if a lot of the serious left-turns in the market in the last century wouldn't have been predicted by the same models that could track day-to-day trends well.

Vivtek 16y ago

Oh, great. AI will be over-hyped again, just like in the 80's, and we'll have to call it something else for a decade when our families ask us what we're doing lately.

wheaties 16y ago
Yes, it's true but 1) there will be a renewed interest in the science with more funding leading to more discoveries which could, potentially benefit man. Who knows? 2) it may also be derided much like in the 80's when there were many funds employing the same "fail safe" measures to guard against loss.

tkahnoski 16y ago

Hypothetical: What if an AI driven investment fund significantly out performed other funds? The focus on number crunching seems contrary to advice given by Warren Buffet and other successful investors about investing in what you know. i.e. "I think the market is under-estimating the impact of Product X, I'll invest in that company" versus "Analyze market cap, yearly revenue...."

icey 16y ago
I've always wondered if there was a way to make some money in the market by performing lexical analysis of news articles about stocks and gauging the performance of those stocks based on the contents of the article. So, you'd train your software by analyzing every news article you could find over the past N years for every company in the market you're targeting. It would look for words or phrases that occurred with higher frequency a few days before stocks made significant moves in either direction. I suppose you'd probably want to weight the source as well, articles sourced from the WSJ might earn a better reputation than ones from the NY Post for example. At some point there would hopefully be enough confidence in the data that you could test it out with $1000 or something small; news-driven automation... There are enough people who buy and sell on news and opinion pieces that there may be enough fat there to make some profits.
- tkahnoski 16y ago
  There was actually an interesting study done on quarterly earnings releases that had some interesting implications as far as "soft data" versus hard numbers: http://ideas.repec.org/p/fip/fedgif/951.html http://ideas.repec.org/p/fip/fedgif/951.html There was a company "Quant the News" that launched a product that did sentiment analysis on company related news, but their website no longer exists. http://www.crunchbase.com/company/quant-the-news http://www.crunchbase.com/company/quant-the-news

T_S_ 16y ago

Market prediction is one of the hardest areas for machine learning/statistics/AI to prove their value. What the system can do is monitor more inputs than a single trader, so the screening process can be scaled up--provided you have relevant inputs. With more signals becoming available electronically it is possible to automatically incorporate non-market prices into trading decisions. All this will require lots of non-ML engineering to supply the data (just like Google search). When monthly-issued government economic statistic stop moving the market you'll know we are there.

HSO 16y ago
The reason for this may be that the market is made up of interacting learning machines (if you subsume humans among them). Thus, every algorithm introduced changes the basis on whcih the algorithm was selected. Talk about moving targets...
- T_S_ 16y ago
  I agree, and the word I would use is competing machines. Every time me (or my algo) bid up a security price I remove that "cheapness" signal from a competitor, increasing the noise they have to deal with and vice versa. ML hates noisy data. Returns are very noisy. ML has a much easier time with data generated from situations that are lacking competition, such as data from any process inside a company, e.g. consumer behavior. These are problems that want to be solved.

dododo 16y ago

it seems people call these techniques machine learning/AI simply because it sounds cooler: a lot of the techniques have a 30+ year history in statistics. this is the same story as neural networks: neural networks are just nested logistic regression, a technique with much history in statistics. a lot of high frequency trading is simply linear/logistic regression (on the right features of course). anything more complicated is too slow.

jimbokun 16y ago
http://www-stat.stanford.edu/~tibs/stat315a/glossary.pdf http://www-stat.stanford.edu/~tibs/stat315a/glossary.pdf Note entries for "large grant" and "nice place to have a meeting."
deleted 16y ago
[deleted]

apu 16y ago

Hello, submarine! http://www.paulgraham.com/submarine.html http://www.paulgraham.com/submarine.html