4 ms·

Why does everyone naively try to predict price? No ‘traders’ are interested in predicting it - what traders do is identify good locations to enter or exit the m

by jarym 8y ago

Why does everyone naively try to predict price? No ‘traders’ are interested in predicting it - what traders do is identify good locations to enter or exit the market. I.e. places with defined risk where you will know if you’re wrong if it goes against you by x% while you expect a y% gain if you’re right AND y>x is worth more than the number of times you’re wrong. The types of Algos that work well for this are edge identification ones - I know this because I am (not as well as I’d like) successfully doing it. LSTMs haven’t performed so well for me in this task but non-NN algos have. CNNs however were promising but didn’t match what I’d come up with - still searching for the holy grail that’ll make me rich!

rahimnathwani 8y ago
Can you point to any good sources for this? A search for 'edge identification stocks' yielded mostly irrelevant results.
- jarym 8y ago
  For example this blog post (not mine): http://jon.io/machine-beats-human-using-machine-learning-in-forex.html http://jon.io/machine-beats-human-using-machine-learning-in-... Talks about the Mean Shift algorithm described here: https://en.wikipedia.org/wiki/Mean_shift https://en.wikipedia.org/wiki/Mean_shift
polskibus 8y ago
Can you share some pointers to more reading material on algos you consider more successful?
foxes 8y ago
Better, why do people think they can predict long term dynamics of what is basically a chaotic system? I think finding some small local dynamics is fine, but applying a neural network to try say something about global long term dynamics complete garbage - ie long term weather simulation, stock markets etc. Chaotic systems can be deterministic, just that you will never be able to accurately measure all the variables to make a long term prediction accurately. In the weather example, people know the equations that approximate how it works. Why value does a neural network bring? Knowing the equations is better understanding.
- jkabrg 8y ago
  There was a Quanta Magazine article talking about predicting the evolution of a flame-front using ML. The ML algorithm remained accurate for eight Lyapunov intervals; eight times longer than the previous SOTA. https://www.quantamagazine.org/machine-learnings-amazing-ability-to-predict-chaos-20180418/ https://www.quantamagazine.org/machine-learnings-amazing-abi...
  - foxes 8y ago
    The only real argument I think you can make is that it might be more efficient to have a neural network quickly spit out an approximate solution instead of solving the actual equations. But if you have the time having the actual equations is more valuable?
    - PeterisP 8y ago
      Assuming that "having the actual equations" is possible, which in market predictions it most likely is not.
bougiefever 8y ago
Is this what you are talking about? https://www.dukascopy.com/fxcomm/fx-article-contest/?Automated-High-Frequency-Trading-With=&action=read&id=1835&language=en https://www.dukascopy.com/fxcomm/fx-article-contest/?Automat...
cmroanirgo 8y ago
> Why does everyone naively try to predict price? No ‘traders’ are interested in predicting it - what traders do is identify good locations to enter or exit the market. I agree. I've built many systems in this area, but it wasn't until I started working in the Indian market (>10 yrs ago) that it became abundantly clear that trying to calculate the long/shorts signals using historical (/time series) data was a waste of time. (And yet my primary role was to provide tools that did exactly that). Back then, in the indian market, you could see that most of the stocks, although skyrocketing upwards, all followed the slow vs fast moving averages to buy and sell! Back then, they weren't looking at RSI, stochastics, support lines, etc, etc. It was crazily predictable...but over time it was really interesting to see it become more haphazard and like western stocks. That is, the fundamentals came into play and as you say, the traders began to use other metrics to buy and sell.
- person_of_color 8y ago
  How much did you make?
- leeuwnhawk 8y ago
  > I've built many systems in this area, but it wasn't until I started working in the Indian market (>10 yrs ago) that it became abundantly clear that trying to calculate the long/shorts signals using historical (/time series) data was a waste of time. (And yet my primary role was to provide tools that did exactly that). I'm currently working on building similar tools in my area of work for the Indian market and would really appreciate if you could shed some more light into the things you learned from your experience in working in this domain.
beagle3 8y ago
.. because you buy at the price, and sell at the price (spread and fees ignored for now). Which means, regardless of your philosophy, you are predicting a price change - a long signal is a prediction for positive price change; a short signal is a prediction for a negative price change. If that wasn’t true, your system would not be able to profit. Predicting price change and predicting price are semantically equivalent, although a specific algorithm might be better at one than the other.
- jarym 8y ago
  Predicting price means you’re predicting one variable with no idea of hot likely you are to be wrong and how wrong you’re likely to be and says nothing of where your expectations are for price to go after. It is semantically different to say: if price goes to Y then you have odds that it will then go to Target 1 and then slightly lower odds it goes to Target 2.
  - beagle3 8y ago
    Do you long now, or short now, or neither? If you go long, you’ve predicted price goes higher. Regardless of what you think about the entire path. Mathematically nothing else makes sense.
- qeternity 8y ago
  Given less than 100% certainty, traders don't want to predict price, they want to predict future distribution of price over some time period. Source: hedge fund trader
  - beagle3 8y ago
    True. That’s still considered a prediction of price among my trading colleagues.
- noelsusman 8y ago
  You want to predict price, but a price prediction is useless without an accurate estimate of the error of your prediction.

by jarym 8y ago

rahimnathwani 8y ago

Can you point to any good sources for this? A search for 'edge identification stocks' yielded mostly irrelevant results.

jarym 8y ago
For example this blog post (not mine): http://jon.io/machine-beats-human-using-machine-learning-in-forex.html http://jon.io/machine-beats-human-using-machine-learning-in-... Talks about the Mean Shift algorithm described here: https://en.wikipedia.org/wiki/Mean_shift https://en.wikipedia.org/wiki/Mean_shift

polskibus 8y ago

Can you share some pointers to more reading material on algos you consider more successful?

foxes 8y ago

Better, why do people think they can predict long term dynamics of what is basically a chaotic system? I think finding some small local dynamics is fine, but applying a neural network to try say something about global long term dynamics complete garbage - ie long term weather simulation, stock markets etc. Chaotic systems can be deterministic, just that you will never be able to accurately measure all the variables to make a long term prediction accurately. In the weather example, people know the equations that approximate how it works. Why value does a neural network bring? Knowing the equations is better understanding.

jkabrg 8y ago
There was a Quanta Magazine article talking about predicting the evolution of a flame-front using ML. The ML algorithm remained accurate for eight Lyapunov intervals; eight times longer than the previous SOTA. https://www.quantamagazine.org/machine-learnings-amazing-ability-to-predict-chaos-20180418/ https://www.quantamagazine.org/machine-learnings-amazing-abi...
- foxes 8y ago
  The only real argument I think you can make is that it might be more efficient to have a neural network quickly spit out an approximate solution instead of solving the actual equations. But if you have the time having the actual equations is more valuable?
  - PeterisP 8y ago
    Assuming that "having the actual equations" is possible, which in market predictions it most likely is not.

bougiefever 8y ago

Is this what you are talking about? https://www.dukascopy.com/fxcomm/fx-article-contest/?Automated-High-Frequency-Trading-With=&action=read&id=1835&language=en https://www.dukascopy.com/fxcomm/fx-article-contest/?Automat...

cmroanirgo 8y ago

> Why does everyone naively try to predict price? No ‘traders’ are interested in predicting it - what traders do is identify good locations to enter or exit the market. I agree. I've built many systems in this area, but it wasn't until I started working in the Indian market (>10 yrs ago) that it became abundantly clear that trying to calculate the long/shorts signals using historical (/time series) data was a waste of time. (And yet my primary role was to provide tools that did exactly that). Back then, in the indian market, you could see that most of the stocks, although skyrocketing upwards, all followed the slow vs fast moving averages to buy and sell! Back then, they weren't looking at RSI, stochastics, support lines, etc, etc. It was crazily predictable...but over time it was really interesting to see it become more haphazard and like western stocks. That is, the fundamentals came into play and as you say, the traders began to use other metrics to buy and sell.

person_of_color 8y ago
How much did you make?
leeuwnhawk 8y ago
> I've built many systems in this area, but it wasn't until I started working in the Indian market (>10 yrs ago) that it became abundantly clear that trying to calculate the long/shorts signals using historical (/time series) data was a waste of time. (And yet my primary role was to provide tools that did exactly that). I'm currently working on building similar tools in my area of work for the Indian market and would really appreciate if you could shed some more light into the things you learned from your experience in working in this domain.

beagle3 8y ago

.. because you buy at the price, and sell at the price (spread and fees ignored for now). Which means, regardless of your philosophy, you are predicting a price change - a long signal is a prediction for positive price change; a short signal is a prediction for a negative price change. If that wasn’t true, your system would not be able to profit. Predicting price change and predicting price are semantically equivalent, although a specific algorithm might be better at one than the other.

jarym 8y ago
Predicting price means you’re predicting one variable with no idea of hot likely you are to be wrong and how wrong you’re likely to be and says nothing of where your expectations are for price to go after. It is semantically different to say: if price goes to Y then you have odds that it will then go to Target 1 and then slightly lower odds it goes to Target 2.
- beagle3 8y ago
  Do you long now, or short now, or neither? If you go long, you’ve predicted price goes higher. Regardless of what you think about the entire path. Mathematically nothing else makes sense.
qeternity 8y ago
Given less than 100% certainty, traders don't want to predict price, they want to predict future distribution of price over some time period. Source: hedge fund trader
- beagle3 8y ago
  True. That’s still considered a prediction of price among my trading colleagues.
noelsusman 8y ago
You want to predict price, but a price prediction is useless without an accurate estimate of the error of your prediction.