5 ms·

it won't be long before we hearing more headlines like: "AI beats top 10 hedge fund managers" to "AI run hedge fund blows up due to black swan event" regard

it won't be long before we hearing more headlines like: "AI beats top 10 hedge fund managers" to "AI run hedge fund blows up due to black swan event" regardless it's an incredible feat. It really casts questions into what our edge as humans are which is slowly disappearing and we didn't even need to put a brain in a jar and hook it up to a computer....it's deep learning reinforced algorithms that is appearing to outlearn, outthink the best of humans. I just can't emphasize what a monumental period in history we are at. Humans are producing specialized algorithms that learn and hold information about the deep web of relationships between myriads of parameters to produce superior performance than humans. It's almost like we've uncovered ways to automate our intelligence very much like we've been automating human and animal labor in the past couple centuries. So the question is, how does an average joe hacker like me exploit and leverage this wonderful thing called deep learning? I'm not interested in reading PHD papers with advanced calculus. I want to have a map of what AI, ML, DL, NN methodologies to use and when and who to hire based on that. This is no time to be a luddite and don't count on basic income from appeasing the masses anytime soon. Much like people took the most hit in the early rise of industrial revolution, our generation and immediate generation will be hit the hardest.

bduerst 10y ago
>AI beats top 10 hedge fund managers" Hedge funds are already using A.I. successfully. I talked one manager who's team was trying to layer successful individual neural networks into a "bigger brain".
- deepnotderp 10y ago
  Been there, done that (well kind, of, via several small specialized nets). Actually a pretty good trading strategy, but works better if you actually have capital to use it....
  - sulam 10y ago
    I was under the impression that you need some capital, but not so much that you move the market with your trades. True?
    - philipov 10y ago
      eh, if you've got the money to be worrying about moving the market, you're using brokers' advanced execution algos and darkpools.
      - JumpCrisscross 10y ago
        > if you've got the money to be worrying about moving the market, you're using advanced execution strategies and darkpools These aren't silver bullets. Both cost more, in fees, and have a habit of leaking order information across informal channels. They're also, generally, slower--there are, on average, fewer buyers and sellers for a given security in a dark pool than in the open market.
    - deepnotderp 10y ago
      IDK that much about the finer parts of trading, since I just approached it from a ML perspective. What I do know is that it wasn't worth it for me personally to occupy my GPUs. By contrast, larger trading firms would probably like it.
  - brilliantcode 10y ago
    probably not going to get any details out of you but worth trying: can you give any pointers to implement such strategies and the minimum capital involved?
    - xapata 10y ago
      A Google search result for "machine learning stock trading": https://www.udacity.com/course/machine-learning-for-trading--ud501 https://www.udacity.com/course/machine-learning-for-trading-...
      - brilliantcode 10y ago
        well to be honest I never bothered googling that exact term but I was more interested in the details around his deployment of smaller and successful neural networks as part of a portfolio.
        deepnotderp 10y ago
        Basically, certain neural networks perform better at predicting different factors at different market conditions. You can string them together to get a decent trader, enough to make a profit, but not enough to justify hogging your GPUs.
        BigJono 10y ago
        As in the profit generated isn't enough to sustain the electricity cost of running the GPUs? Are you able to scale up with more initial investment? How much did you invest and what kind of return on investment were you getting?
        deepnotderp 10y ago
        Not enough to justify using up the compute capabilities of my GPUs since I need them for research. I averaged a return of 7-13% a year. With less than 1000 in capital, it wasn't very profitable. I don't see why it shouldn't scale.
  - bduerst 10y ago
    Rule #1: It's always better to trade with OPM (Other People's Money)
feral 10y ago
> "AI run hedge fund blows up due to black swan event" Knight Capital, 2012: https://www.bloomberg.com/news/articles/2012-08-02/knight-shows-how-to-lose-440-million-in-30-minutes https://www.bloomberg.com/news/articles/2012-08-02/knight-sh...
- brilliantcode 10y ago
  How many of these hedge funds running on AI are out there in America? I'd think one would make a lot of money shorting such outfits.
  - sanswork 10y ago
    That would only work if you'd expect these events to be frequent enough that you wouldn't have to short for very long or else the costs of your short would outpace any possible profit very quickly.
  - gnaritas 10y ago
    Only if you can time when they're going to fail. It's not enough to be right, you have to be right at the right time.
    - brilliantcode 10y ago
      I know with high confidence that these AI run hedge fund outfits reverts to a zero sum mean. Doesn't mean that there won't be one or two that beats the market for sometime as a whataboutism. The timing is a major issue, it's tough to at what year these AI hedge funds fail. The other capital that blew up from the comments suggest it was more of a human error that led to deployment failures which is exactly why AI hedge funds is such a paradox. At the end of the day, it's humans that's deploying and building the model to be approximately right with high degree of accuracy but in the day trading environment it is a poor model for success-not only do you have to be right your monetary exposure must be right...."but proper money management and you will be fine" said everyone who took part in this zero sum game. I don't know, prove me wrong, I'm sure those ivy league engineers on wall street aren't getting paid dimes for what they do.
deleted 10y ago
[deleted]
econner 10y ago
> "AI run hedge fund blows up due to black swan event" Why's this an argument against AI run hedge funds? Funds run by humans also blow up due to black swan events.
eva1984 10y ago
> So the question is, how does an average joe hacker like me exploit and leverage this wonderful thing called deep learning? I'm not interested in reading PHD papers with advanced calculus. Well reading paper is a must to get to deep learning. Those papers may not be that math heavy once you are used to it. Most of the time, it is about network architecture and loss objectives.
deepnotderp 10y ago
> "So the question is, how does an average joe hacker like me exploit and leverage this wonderful thing called deep learning? I'm not interested in reading PHD papers with advanced calculus." FFS, if you're not willing to read a paper with BASIC CALCULUS (it's HS/college, not advanced like fractional), then I'm not sure machine learning is the right place for you.
- brilliantcode 10y ago
  Sure it is. Much like most developers don't spend their time reading on computing science papers to do their job, I question why such academic rigour is to be demanded from somebody trying to capitalize on the arbitrage opportunity. Math is good but my time won't be best used if I have to learn calculus all over again just to begin understanding the linguo. Rather have a generic model of what to use and when, hire those that have that capacity to dive deep when needed and implement the expected business outcome.
  - ori_b 10y ago
    Because this is the language of the domain. Much like code is the language of programming. This isn't academic rigor or lingo. This is how things work. To me, your comment reads similarly to "Much like most business people don't spend their time reading code to do the job, I question why such academic rigor is to be demanded by someone who is translating business requirements into database operations".
    - brilliantcode 10y ago
      I think we are after different things. My goal is not to understand the underlying technology at the academic level but just enough to make money by utilizing the people that do, just like any good capitalist. You don't need to be an oil rig worker to understand where the opportunity lies is what I'm saying. It's possible to understand systems from a blackbox point of view and still be able to exploit them. Afterall, this is what we do whenever we hire a new developer, it's impossible to know what they know so I leave that up to smarter and talented people while I allocate the capitals and set a vision for what's required to capitalize on the business opportunity that AI presents with it's superior performance, scalability and economies to scale. You outsource what you don't know or can't do and if you can't afford it you do end up doing it yourself. This is why capitalism works. It's not necessary for you to dwindle with all the underlying complexities when you model the business opportunity in terms of basic expenditure and investment returns.
ifdefdebug 10y ago
It's just that AI researchers are exploring and finding every single field where deep learning algorithms perform better than humans. Play poker, run hedge funds... But for instance understanding and specially producing meaningful language, let it be natural language or programming language? I hope not so. Because otherwise, your average joe hacker might as well shut down their IDE and say good bye.
- russdill 10y ago
  Looking at the recent leap in translation between natural languages, it seems that producing meaningful language about various situations is not far off.
  - nazgul17 10y ago
    Well, but that is not really "production", because you need the language to translate from.
arnarbi 10y ago
> It's almost like we've uncovered ways to automate our intelligence very much like we've been automating human and animal labor in the past couple centuries. Actually, a lot of mental work isn't based on intelligence but rather training in pattern recognition, thinking faster, translation of information between different coding systems, and developing instincts about the behavior if complex systems. This isn't very different from manual labor when you think about it.
- brilliantcode 10y ago
  hmmm interesting way to put things. I often held such cynicisms about "intelligent" and "professionals". At the end of the day it's still labor, white collared, but none the less value is trapped in the blackbox which are humans. Now it appears the essence of that is being distilled into parameters for AI algorithms which can produce superior output by teaching it with decades of professional human knowledge which maybe dumbed down with simple machine learning.
narrator 10y ago
>"AI beats top 10 hedge fund managers" Renaissance Technologies already is one of the top hedge funds in the world and it's fully automated. [1]https://en.wikipedia.org/wiki/Renaissance_Technologies https://en.wikipedia.org/wiki/Renaissance_Technologies
tqkxzugoaupvwqr 10y ago
> It's almost like we've uncovered ways to automate our intelligence I wonder if humankind will be the creator of a new form of life. One that is not bound by chemical processes. One that moves at the speed of light and spans the whole planet. This being, or these beings, with access to billions of sensors, will know everything, see everything, hear everything. They won’t be limited to visible light, nor to the frequencies we can hear. There are no limitations except for what sensors can record. They will be able to interpret, predict, plan. They are not bound by time as they themselves don’t physically age. They will use vessels to carry out tasks and build the infrastructure they need to live. In comparison, we will look like single-celled organisms. So dumb. Until we disappear.
orless 10y ago
> So the question is, how does an average joe hacker like me exploit and leverage this wonderful thing called deep learning? I'm not interested in reading PHD papers with advanced calculus. You've got a lot of comments like "if you're not willing to read a paper with BASIC CALCULUS ... then I'm not sure machine learning is the right place for you" to this, but I second you on this. I do have a PhD in CS and not afraid of calculus in papers. I just don't think reading PhD papers is the best path to getting practical results in DL/ML. "Not right place for you" is the sign of the "exclusiveness" problem. I'd recommend checking the following course by Jeremy Howard and Rachel Thomas: http://course.fast.ai/ http://course.fast.ai/ It is designed to make DL accessible and achieves its goal very well. I'm not sure if the average Joe Hacker will be able to write top-pocker-players-beating program after this course, but it definitely gives enough insight and practice to get started in DL and be able to design and code solutions in a large number of problem classess.
StringEpsilon 10y ago
I mean, by definition Black Swans can't be anticipated. So, no matter if it's an AI or human, black swans will throw you over. But, an artificial intelligence probably has different black swans than humans, as their perception is inherently different. A tweet from Trump might not surprise us (anymore) but to an AI player, it might not be evident that a small amount of text from one Human can cause an uproar on the market. And it might not even have access to the relevant data (Twitter) at all.

bduerst 10y ago

>AI beats top 10 hedge fund managers" Hedge funds are already using A.I. successfully. I talked one manager who's team was trying to layer successful individual neural networks into a "bigger brain".

deepnotderp 10y ago
Been there, done that (well kind, of, via several small specialized nets). Actually a pretty good trading strategy, but works better if you actually have capital to use it....
- sulam 10y ago
  I was under the impression that you need some capital, but not so much that you move the market with your trades. True?
  - philipov 10y ago
    eh, if you've got the money to be worrying about moving the market, you're using brokers' advanced execution algos and darkpools.
    - JumpCrisscross 10y ago
      > if you've got the money to be worrying about moving the market, you're using advanced execution strategies and darkpools These aren't silver bullets. Both cost more, in fees, and have a habit of leaking order information across informal channels. They're also, generally, slower--there are, on average, fewer buyers and sellers for a given security in a dark pool than in the open market.
  - deepnotderp 10y ago
    IDK that much about the finer parts of trading, since I just approached it from a ML perspective. What I do know is that it wasn't worth it for me personally to occupy my GPUs. By contrast, larger trading firms would probably like it.
- brilliantcode 10y ago
  probably not going to get any details out of you but worth trying: can you give any pointers to implement such strategies and the minimum capital involved?
  - xapata 10y ago
    A Google search result for "machine learning stock trading": https://www.udacity.com/course/machine-learning-for-trading--ud501 https://www.udacity.com/course/machine-learning-for-trading-...
    - brilliantcode 10y ago
      well to be honest I never bothered googling that exact term but I was more interested in the details around his deployment of smaller and successful neural networks as part of a portfolio.
      - deepnotderp 10y ago
        Basically, certain neural networks perform better at predicting different factors at different market conditions. You can string them together to get a decent trader, enough to make a profit, but not enough to justify hogging your GPUs.
        BigJono 10y ago
        As in the profit generated isn't enough to sustain the electricity cost of running the GPUs? Are you able to scale up with more initial investment? How much did you invest and what kind of return on investment were you getting?
        deepnotderp 10y ago
        Not enough to justify using up the compute capabilities of my GPUs since I need them for research. I averaged a return of 7-13% a year. With less than 1000 in capital, it wasn't very profitable. I don't see why it shouldn't scale.
- bduerst 10y ago
  Rule #1: It's always better to trade with OPM (Other People's Money)

feral 10y ago

> "AI run hedge fund blows up due to black swan event" Knight Capital, 2012: https://www.bloomberg.com/news/articles/2012-08-02/knight-shows-how-to-lose-440-million-in-30-minutes https://www.bloomberg.com/news/articles/2012-08-02/knight-sh...

brilliantcode 10y ago
How many of these hedge funds running on AI are out there in America? I'd think one would make a lot of money shorting such outfits.
- sanswork 10y ago
  That would only work if you'd expect these events to be frequent enough that you wouldn't have to short for very long or else the costs of your short would outpace any possible profit very quickly.
- gnaritas 10y ago
  Only if you can time when they're going to fail. It's not enough to be right, you have to be right at the right time.
  - brilliantcode 10y ago
    I know with high confidence that these AI run hedge fund outfits reverts to a zero sum mean. Doesn't mean that there won't be one or two that beats the market for sometime as a whataboutism. The timing is a major issue, it's tough to at what year these AI hedge funds fail. The other capital that blew up from the comments suggest it was more of a human error that led to deployment failures which is exactly why AI hedge funds is such a paradox. At the end of the day, it's humans that's deploying and building the model to be approximately right with high degree of accuracy but in the day trading environment it is a poor model for success-not only do you have to be right your monetary exposure must be right...."but proper money management and you will be fine" said everyone who took part in this zero sum game. I don't know, prove me wrong, I'm sure those ivy league engineers on wall street aren't getting paid dimes for what they do.

deleted 10y ago

[deleted]

econner 10y ago

> "AI run hedge fund blows up due to black swan event" Why's this an argument against AI run hedge funds? Funds run by humans also blow up due to black swan events.

eva1984 10y ago

> So the question is, how does an average joe hacker like me exploit and leverage this wonderful thing called deep learning? I'm not interested in reading PHD papers with advanced calculus. Well reading paper is a must to get to deep learning. Those papers may not be that math heavy once you are used to it. Most of the time, it is about network architecture and loss objectives.

deepnotderp 10y ago

> "So the question is, how does an average joe hacker like me exploit and leverage this wonderful thing called deep learning? I'm not interested in reading PHD papers with advanced calculus." FFS, if you're not willing to read a paper with BASIC CALCULUS (it's HS/college, not advanced like fractional), then I'm not sure machine learning is the right place for you.

brilliantcode 10y ago
Sure it is. Much like most developers don't spend their time reading on computing science papers to do their job, I question why such academic rigour is to be demanded from somebody trying to capitalize on the arbitrage opportunity. Math is good but my time won't be best used if I have to learn calculus all over again just to begin understanding the linguo. Rather have a generic model of what to use and when, hire those that have that capacity to dive deep when needed and implement the expected business outcome.
- ori_b 10y ago
  Because this is the language of the domain. Much like code is the language of programming. This isn't academic rigor or lingo. This is how things work. To me, your comment reads similarly to "Much like most business people don't spend their time reading code to do the job, I question why such academic rigor is to be demanded by someone who is translating business requirements into database operations".
  - brilliantcode 10y ago
    I think we are after different things. My goal is not to understand the underlying technology at the academic level but just enough to make money by utilizing the people that do, just like any good capitalist. You don't need to be an oil rig worker to understand where the opportunity lies is what I'm saying. It's possible to understand systems from a blackbox point of view and still be able to exploit them. Afterall, this is what we do whenever we hire a new developer, it's impossible to know what they know so I leave that up to smarter and talented people while I allocate the capitals and set a vision for what's required to capitalize on the business opportunity that AI presents with it's superior performance, scalability and economies to scale. You outsource what you don't know or can't do and if you can't afford it you do end up doing it yourself. This is why capitalism works. It's not necessary for you to dwindle with all the underlying complexities when you model the business opportunity in terms of basic expenditure and investment returns.

ifdefdebug 10y ago

It's just that AI researchers are exploring and finding every single field where deep learning algorithms perform better than humans. Play poker, run hedge funds... But for instance understanding and specially producing meaningful language, let it be natural language or programming language? I hope not so. Because otherwise, your average joe hacker might as well shut down their IDE and say good bye.

russdill 10y ago
Looking at the recent leap in translation between natural languages, it seems that producing meaningful language about various situations is not far off.
- nazgul17 10y ago
  Well, but that is not really "production", because you need the language to translate from.

arnarbi 10y ago

> It's almost like we've uncovered ways to automate our intelligence very much like we've been automating human and animal labor in the past couple centuries. Actually, a lot of mental work isn't based on intelligence but rather training in pattern recognition, thinking faster, translation of information between different coding systems, and developing instincts about the behavior if complex systems. This isn't very different from manual labor when you think about it.

brilliantcode 10y ago
hmmm interesting way to put things. I often held such cynicisms about "intelligent" and "professionals". At the end of the day it's still labor, white collared, but none the less value is trapped in the blackbox which are humans. Now it appears the essence of that is being distilled into parameters for AI algorithms which can produce superior output by teaching it with decades of professional human knowledge which maybe dumbed down with simple machine learning.

narrator 10y ago

>"AI beats top 10 hedge fund managers" Renaissance Technologies already is one of the top hedge funds in the world and it's fully automated. [1]https://en.wikipedia.org/wiki/Renaissance_Technologies https://en.wikipedia.org/wiki/Renaissance_Technologies

tqkxzugoaupvwqr 10y ago

> It's almost like we've uncovered ways to automate our intelligence I wonder if humankind will be the creator of a new form of life. One that is not bound by chemical processes. One that moves at the speed of light and spans the whole planet. This being, or these beings, with access to billions of sensors, will know everything, see everything, hear everything. They won’t be limited to visible light, nor to the frequencies we can hear. There are no limitations except for what sensors can record. They will be able to interpret, predict, plan. They are not bound by time as they themselves don’t physically age. They will use vessels to carry out tasks and build the infrastructure they need to live. In comparison, we will look like single-celled organisms. So dumb. Until we disappear.

orless 10y ago

> So the question is, how does an average joe hacker like me exploit and leverage this wonderful thing called deep learning? I'm not interested in reading PHD papers with advanced calculus. You've got a lot of comments like "if you're not willing to read a paper with BASIC CALCULUS ... then I'm not sure machine learning is the right place for you" to this, but I second you on this. I do have a PhD in CS and not afraid of calculus in papers. I just don't think reading PhD papers is the best path to getting practical results in DL/ML. "Not right place for you" is the sign of the "exclusiveness" problem. I'd recommend checking the following course by Jeremy Howard and Rachel Thomas: http://course.fast.ai/ http://course.fast.ai/ It is designed to make DL accessible and achieves its goal very well. I'm not sure if the average Joe Hacker will be able to write top-pocker-players-beating program after this course, but it definitely gives enough insight and practice to get started in DL and be able to design and code solutions in a large number of problem classess.

StringEpsilon 10y ago

I mean, by definition Black Swans can't be anticipated. So, no matter if it's an AI or human, black swans will throw you over. But, an artificial intelligence probably has different black swans than humans, as their perception is inherently different. A tweet from Trump might not surprise us (anymore) but to an AI player, it might not be evident that a small amount of text from one Human can cause an uproar on the market. And it might not even have access to the relevant data (Twitter) at all.