3 ms·

I spent the last year working fulltime on a system similar to the one described here. I trade the top ~20 cryptos on binance. I use deep learning models (combin

I spent the last year working fulltime on a system similar to the one described here. I trade the top ~20 cryptos on binance. I use deep learning models (combination of temporal, causal convnets and RNNs) with heavy data augmentation. I built my own tooling for data collection, training, backtesting and live deployment. Having a data engineer background coming into this was hugely helpful: most of my time was spent manipulating data in some way (and not playing around with the models). One of the most demanding parts was estimating spread/slippage costs and including it into the loss function. Most of what the author talked about, I learned the hard way. I'm now at the point where I ran some tests (trading small amounts) live on binance and the results are positive: I do manage to make small profits, but more importantly, the recorded live trades reflect very closely the backtest trades (for a given period). I'm currently scaling up my model and adding better monitoring / reporting / CI. I'd be happy to chat with anyone having done similar projects or willing to exchange ideas.

alexcnwy 7y ago
I'd love to hear more about what kind of data augmentation you're doing. A friend of mine recently got a GAN to work for timeseries which is really interesting. I've done a lot of work in the space and would love to chat - just emailed you :)
- ghgr 7y ago
  Indeed GANs are showing very promising results. There's a series of blog posts from Fernando de Meer which discuss this topic in a very approachable way. https://quantdare.com/generating-financial-series-with-generative-adversarial-networks/ https://quantdare.com/generating-financial-series-with-gener...
  - alexcnwy 7y ago
    Very cool, thanks for the link!
- pinouchon 7y ago
  I use a supervised learning setup (although with a custom loss function). The kind of data augmentation I do is adding different candles sizes. I validate with 5m candles, but I train with 2,3,4,5,6,7m ones. I also sample more frequently more recent data. I train jointly with ~22 symbols, but in each X with those symbols, I randomly set some to 0, some I invert their price, some I invert time-wise. This helps generalization for some reason. I tried many kinds of noise, but what I described above is what I found to work best in my case. I have a more ambitious idea to generate synthetic data using self play: have a bunch of agents trading one against another. This create new price data I can train the agents with, and repeat (this self-play training scheme would be similar to what DeepMind did with AlphaGo/AlphaZero). The issue with it is the need to tune the parameters exactly so that the resulting synthetic data is realistic enough that I can tranfer the agents to real data. For example, during self-play, should you have only trading agents or should you add "retail traders" that buy during bubbles, "normal buyers" that buy only below, sell above certain prices, institutional buyers that randomly move the price a lot in a given direction. This is a lot of parameters to get right, and it's an optitization problem on it own. You could treat this a as two-fold optimization problem such as in this paper: https://arxiv.org/pdf/1810.02513.pdf https://arxiv.org/pdf/1810.02513.pdf, but it gets tricky very fast.
Traster 7y ago
Are you concerned by other commenters who mention that once they were successful the exchange moved in to either stop them or extract a fee?
- pinouchon 7y ago
  Yes I am. Without more detail about the case it's hard to say what was the issue. Binance has what they call "machine learning limits": https://www.binance.com/en/support/articles/360004492232 https://www.binance.com/en/support/articles/360004492232. What lowers my concerns is that such issues might come once the bot is profitable, and by then I might have found other ways to raise capital (I'm doing a trading bot as a way to raise capital to create an AI research lab). Being able to setup a profitable crypto trader is a good thing to add on a resume or for personal branding even if I shut it down at some point. I'll be in a very different position by this point so it's a bit premature to be concerned about that now, although it's still somewhat of a concern. On the positive side, I hope this scares the competition away.

alexcnwy 7y ago

I'd love to hear more about what kind of data augmentation you're doing. A friend of mine recently got a GAN to work for timeseries which is really interesting. I've done a lot of work in the space and would love to chat - just emailed you :)

ghgr 7y ago
Indeed GANs are showing very promising results. There's a series of blog posts from Fernando de Meer which discuss this topic in a very approachable way. https://quantdare.com/generating-financial-series-with-generative-adversarial-networks/ https://quantdare.com/generating-financial-series-with-gener...
- alexcnwy 7y ago
  Very cool, thanks for the link!
pinouchon 7y ago
I use a supervised learning setup (although with a custom loss function). The kind of data augmentation I do is adding different candles sizes. I validate with 5m candles, but I train with 2,3,4,5,6,7m ones. I also sample more frequently more recent data. I train jointly with ~22 symbols, but in each X with those symbols, I randomly set some to 0, some I invert their price, some I invert time-wise. This helps generalization for some reason. I tried many kinds of noise, but what I described above is what I found to work best in my case. I have a more ambitious idea to generate synthetic data using self play: have a bunch of agents trading one against another. This create new price data I can train the agents with, and repeat (this self-play training scheme would be similar to what DeepMind did with AlphaGo/AlphaZero). The issue with it is the need to tune the parameters exactly so that the resulting synthetic data is realistic enough that I can tranfer the agents to real data. For example, during self-play, should you have only trading agents or should you add "retail traders" that buy during bubbles, "normal buyers" that buy only below, sell above certain prices, institutional buyers that randomly move the price a lot in a given direction. This is a lot of parameters to get right, and it's an optitization problem on it own. You could treat this a as two-fold optimization problem such as in this paper: https://arxiv.org/pdf/1810.02513.pdf https://arxiv.org/pdf/1810.02513.pdf, but it gets tricky very fast.

Traster 7y ago

Are you concerned by other commenters who mention that once they were successful the exchange moved in to either stop them or extract a fee?

pinouchon 7y ago
Yes I am. Without more detail about the case it's hard to say what was the issue. Binance has what they call "machine learning limits": https://www.binance.com/en/support/articles/360004492232 https://www.binance.com/en/support/articles/360004492232. What lowers my concerns is that such issues might come once the bot is profitable, and by then I might have found other ways to raise capital (I'm doing a trading bot as a way to raise capital to create an AI research lab). Being able to setup a profitable crypto trader is a good thing to add on a resume or for personal branding even if I shut it down at some point. I'll be in a very different position by this point so it's a bit premature to be concerned about that now, although it's still somewhat of a concern. On the positive side, I hope this scares the competition away.