3 ms·

I mean ultimately this is an exercise in frustration because if you do that you will have trained your model on market patterns that might not be in place anymo

I mean ultimately this is an exercise in frustration because if you do that you will have trained your model on market patterns that might not be in place anymore. For example after the 2008 recession regulations changed. So do market dynamics actually work the same in 2025 as in 2005? I honestly don’t know but intuitively I would say that it is possible that they do not. I think a potentially better way would be to segment the market up to today but take half or 10% of all the stocks and make only those available to the LLM. Then run the test on the rest. This accounts for rules and external forces changing how markets operate over time. And you can do this over and over picking a different 10% market slice for training data each time. But then your problem is that if you exclude let’s say Intel from your training data and AMD from your testing data then there ups and downs don’t really make sense since they are direct competitors. If you separate by market segment then does training the model on software tech companies might not actually tell you accurately how it would do for commodities or currency training. Or maybe I am wrong and trading is trading no matter what you are trading.

chris_st 10mo ago
> you will have trained your model on market patterns that might not be in place anymore My working definition of technical analysis [0] [0]: https://en.wikipedia.org/wiki/Technical_analysis https://en.wikipedia.org/wiki/Technical_analysis
- IgorPartola 10mo ago
  It is always fun (in a broad sense of that word) when I make a comment on an industry I know nothing about and somehow stumble onto a thing that not only has a name but also research. I am sure there is a German word for that feel of discovering something that countless others have already discovered.
  - chris_st 10mo ago
    XKCD calls it the "Lucky 10,000" [0] [0]: https://xkcd.com/1053/ https://xkcd.com/1053/
    - mewpmewp2 10mo ago
      That is referring to something completely else. This is referring to some common fact that the person didn't figure out by themself. OP is referring to something they came up with themselves in a field they have no experience with, realizing it is actually a thing in a way feeling validated and clever.
      - gcr 10mo ago
        XKCD calls it "Engineering Syllogism" [0] [0]: https://xkcd.com/1570/ https://xkcd.com/1570/
  - taneq 10mo ago
    Any time I invent a cool thing, I go and try and find it online. Usually it's already an established product, which totally validates my feeling that the thing I invented is cool and would be a good product. :D Occasionally it's (as far as I can tell) a legitimately new 'wow that's obvious' style thing and I consider prototyping it. :)
    - chasing0entropy 10mo ago
      What have you prototyped recently? Anything you have released to market? I'm in the same general area by am teetering on actually launching products wouldn't mind connecting with a like minded e gineer
      - taneq 10mo ago
        One thing that I think has real potential is a portable split system aircon for workshops and garages. The all-in-one aircon units have terrible efficiency but split systems require expensive installation that can cost as much as the unit itself. This compromise gets you the best of both worlds. I built one a year or two ago out of a crusty old second hand split system, and a scrap metal frame that holds the indoor heat exchanger at about head height, and the outdoor heat exchanger back-to-back with it with a ~500mm gap. You open the roller door, position the frame with the outdoor side outside and the indoor side inside, close the roller door, and plug it in. Voila! Aircon! It was way undersized (2.5kW cooling) for my workshop but it still made a noticeable difference. Gonna build a 7kW cooling version for this summer. I think there's heaps of people with garage workshops that would benefit from something like this. You could also make one where the frame slips through a vertical gap rather than a horizontal one, so it can be set up through a sliding door, allowing renters to use one in houses where the owner refuses to install it.
  - biztos 10mo ago
    > there is a German word Zeitgeistüberspannungsfreude
- stouset 10mo ago
  I am frankly astonished at the number of otherwise-intelligent people who actually seem to believe in this stuff. One of the worst possible things to do in a competitive market is to trade by some publicly-available formulaic strategy. It’s like announcing your rock-paper-scissors move to your opponent in advance.
  - tim333 10mo ago
    A couple of subtleties in that. Rather than rock paper scissors with three options, there are hundreds of technical strategies out there so you may still be doing something unusual. Secondly the mass of the public are kind of following a technical strategy of just buy index funds because the index has gone up the past. Which is ignoring the fundamental issue of whether stocks decent value for money at the moment.
  - intalentive 10mo ago
    Technical analysis is a basket of heuristics. Support / resistance / breakout (especially around whole numbers) seems to reflect persistent behavior rooted in human psychology. Look at the heavy buying at the $30 mark here, putting a floor under silver: https://finviz.com/futures_charts.ashx?p=d&t=SI https://finviz.com/futures_charts.ashx?p=d&t=SI This is a common pattern it can be useful to know.
  - timacles 10mo ago
    Technical Analysis isnt a system of predicting the future. Its an analysis of what has happened in the past. You are fundamentally misunderstanding what you're talking about. Its just a system of interpreting money flows and trends on a graph.
0manrho 10mo ago
> you will have trained your model on market patterns that might not be in place anymore How is that relevant to what was proposed? If it's trading and training on 2010 data, what relevance does todays market dynamics and regulations have? Which further begs the question, what's the point of this exercise? Is it to develop a model than compete effectively in today's market? If so then yeah, the 2010 trading/training idea probably isn't the best idea for the reasons you've outlined. Or is it to determine the capacity of an AI to learn and compete effectively within any given arbitrary market/era? If so, then today's dynamics/constraints are irrelevant unless you're explicitly trying to train/trade on todays markets (which isn't what the person you're replying to proposed, but is obviously a valid desire and test case to evaluate in it's own right) Or is it evaluating its ability to identify what those constraints/limitations are and then build strategies based on it? In which case it doesn't matter when you're training/trading so much as your ability to feed it accurate and complete data for that time period be it today, or 15 years ago or whenever, which is no small ask.
noduerme 10mo ago
Just to name a different but related approach, as a hobby project I built a (non LLM) model that trained mainly on data from stocks that didn't move much over the past decade, seeking ways to beat the performance of those particular stocks. I put it into practice for a couple of years, and came out roughly even by constantly rebalancing a basket of stocks that, as a whole, dropped by about 20%. I considered that to be a success, although it would've been nicer to make money.
godelski 10mo ago
> I think a potentially better way would be to segment the market up to today but take half or 10% of all the stocks and make only those available to the LLM. Autocorrelation is going to bite you in the ass. Those stocks are going to be coupled. Let's take an easy example. Suppose you include Nvidia in the training data and hold out AMD for test. Is there information leakage? Yes. The problem is that each company isn't independent. You have information leakage in both the setting where companies grow together as well as zero sum games (since x + y = 0, if you know x then you know y). But in this example AMD tends with Nvidia. Maybe not as much, but they go in the same direction. They're coupled Not to mention that in the specific setting the LLMs were given news and other information.

chris_st 10mo ago

> you will have trained your model on market patterns that might not be in place anymore My working definition of technical analysis [0] [0]: https://en.wikipedia.org/wiki/Technical_analysis https://en.wikipedia.org/wiki/Technical_analysis

IgorPartola 10mo ago
It is always fun (in a broad sense of that word) when I make a comment on an industry I know nothing about and somehow stumble onto a thing that not only has a name but also research. I am sure there is a German word for that feel of discovering something that countless others have already discovered.
- chris_st 10mo ago
  XKCD calls it the "Lucky 10,000" [0] [0]: https://xkcd.com/1053/ https://xkcd.com/1053/
  - mewpmewp2 10mo ago
    That is referring to something completely else. This is referring to some common fact that the person didn't figure out by themself. OP is referring to something they came up with themselves in a field they have no experience with, realizing it is actually a thing in a way feeling validated and clever.
    - gcr 10mo ago
      XKCD calls it "Engineering Syllogism" [0] [0]: https://xkcd.com/1570/ https://xkcd.com/1570/
- taneq 10mo ago
  Any time I invent a cool thing, I go and try and find it online. Usually it's already an established product, which totally validates my feeling that the thing I invented is cool and would be a good product. :D Occasionally it's (as far as I can tell) a legitimately new 'wow that's obvious' style thing and I consider prototyping it. :)
  - chasing0entropy 10mo ago
    What have you prototyped recently? Anything you have released to market? I'm in the same general area by am teetering on actually launching products wouldn't mind connecting with a like minded e gineer
    - taneq 10mo ago
      One thing that I think has real potential is a portable split system aircon for workshops and garages. The all-in-one aircon units have terrible efficiency but split systems require expensive installation that can cost as much as the unit itself. This compromise gets you the best of both worlds. I built one a year or two ago out of a crusty old second hand split system, and a scrap metal frame that holds the indoor heat exchanger at about head height, and the outdoor heat exchanger back-to-back with it with a ~500mm gap. You open the roller door, position the frame with the outdoor side outside and the indoor side inside, close the roller door, and plug it in. Voila! Aircon! It was way undersized (2.5kW cooling) for my workshop but it still made a noticeable difference. Gonna build a 7kW cooling version for this summer. I think there's heaps of people with garage workshops that would benefit from something like this. You could also make one where the frame slips through a vertical gap rather than a horizontal one, so it can be set up through a sliding door, allowing renters to use one in houses where the owner refuses to install it.
- biztos 10mo ago
  > there is a German word Zeitgeistüberspannungsfreude
stouset 10mo ago
I am frankly astonished at the number of otherwise-intelligent people who actually seem to believe in this stuff. One of the worst possible things to do in a competitive market is to trade by some publicly-available formulaic strategy. It’s like announcing your rock-paper-scissors move to your opponent in advance.
- tim333 10mo ago
  A couple of subtleties in that. Rather than rock paper scissors with three options, there are hundreds of technical strategies out there so you may still be doing something unusual. Secondly the mass of the public are kind of following a technical strategy of just buy index funds because the index has gone up the past. Which is ignoring the fundamental issue of whether stocks decent value for money at the moment.
- intalentive 10mo ago
  Technical analysis is a basket of heuristics. Support / resistance / breakout (especially around whole numbers) seems to reflect persistent behavior rooted in human psychology. Look at the heavy buying at the $30 mark here, putting a floor under silver: https://finviz.com/futures_charts.ashx?p=d&t=SI https://finviz.com/futures_charts.ashx?p=d&t=SI This is a common pattern it can be useful to know.
- timacles 10mo ago
  Technical Analysis isnt a system of predicting the future. Its an analysis of what has happened in the past. You are fundamentally misunderstanding what you're talking about. Its just a system of interpreting money flows and trends on a graph.

0manrho 10mo ago

> you will have trained your model on market patterns that might not be in place anymore How is that relevant to what was proposed? If it's trading and training on 2010 data, what relevance does todays market dynamics and regulations have? Which further begs the question, what's the point of this exercise? Is it to develop a model than compete effectively in today's market? If so then yeah, the 2010 trading/training idea probably isn't the best idea for the reasons you've outlined. Or is it to determine the capacity of an AI to learn and compete effectively within any given arbitrary market/era? If so, then today's dynamics/constraints are irrelevant unless you're explicitly trying to train/trade on todays markets (which isn't what the person you're replying to proposed, but is obviously a valid desire and test case to evaluate in it's own right) Or is it evaluating its ability to identify what those constraints/limitations are and then build strategies based on it? In which case it doesn't matter when you're training/trading so much as your ability to feed it accurate and complete data for that time period be it today, or 15 years ago or whenever, which is no small ask.

noduerme 10mo ago

Just to name a different but related approach, as a hobby project I built a (non LLM) model that trained mainly on data from stocks that didn't move much over the past decade, seeking ways to beat the performance of those particular stocks. I put it into practice for a couple of years, and came out roughly even by constantly rebalancing a basket of stocks that, as a whole, dropped by about 20%. I considered that to be a success, although it would've been nicer to make money.

godelski 10mo ago

> I think a potentially better way would be to segment the market up to today but take half or 10% of all the stocks and make only those available to the LLM. Autocorrelation is going to bite you in the ass. Those stocks are going to be coupled. Let's take an easy example. Suppose you include Nvidia in the training data and hold out AMD for test. Is there information leakage? Yes. The problem is that each company isn't independent. You have information leakage in both the setting where companies grow together as well as zero sum games (since x + y = 0, if you know x then you know y). But in this example AMD tends with Nvidia. Maybe not as much, but they go in the same direction. They're coupled Not to mention that in the specific setting the LLMs were given news and other information.