5 ms·

I built an algorithm to mass-analyze all the stocks on the US market. I don’t put too much faith in backtesting, but several basic Buffett-style value strategi

I built an algorithm to mass-analyze all the stocks on the US market. I don’t put too much faith in backtesting, but several basic Buffett-style value strategies seem to beat the S&P 500 handily, with less volatility.

jiqiren 4y ago
GitHub link?
satvikpendem 4y ago
Care to share the alpha with the rest of us? ;)
Galanwe 4y ago
That makes no sense. The last 10 to 20 years have seen a bullish round of momentum stocks negatively correlated to value loaded stocks. Value has consistently lost money in the last decade except for short times during momentum crashes in 2018 and 2023. Either your definition of value is inconsistent with the rest of the world, or your analysis is biased. Considering your sentence: > an algorithm to mass-analyze all the stocks on the US market I guess you reinvented some variation of"low vol" or "small vs big" signal. If you want to say you've beaten the S&P, you would have to show me a long short, beta neutral portfolio that does so using the same investment universe in a point in time manner.
- jrockway 4y ago
  An algorithm could be anything, right? Maybe they analyzed the middle school report cards of executives and majority shareholders, and tied that to value over time in a way that the S&P 500 did. Maybe there's some telling data point that people have never looked at before. I agree that predicting future price based on the current price has likely been tried to death by billions of hedge funds, but who knows, maybe they found something new that nobody has ever noticed. And, you can come up with the best stock trading algorithm in history and still not have the capital necessary to execute it, so it's possible that portfolio results don't matter. I'm not saying anything is likely here, merely that we have nowhere near enough data to even speculate or offer advice.
  - Galanwe 4y ago
    > Maybe they analyzed the middle school report cards of executives and majority shareholders, and tied that to value over time in a way that the S&P 500 did. You are talking about ad-hoc signals here. Of course there are a lot of them that beat the benchmark, it's the basis of active management. But we're not talking about any signal here, we're talking about "value" - a signal so well known, studied and persistent we call it a "factor", and often decompose beta/sector/country adjusted residual returns _just_ based on a set of 4/5 of these factors.
- carabiner 4y ago
  Most finance comment threads on HN don't make sense. Just wait for Jumpcrisscross to drop in and forget the rest.
- lordnacho 4y ago
  The issue with backtesting this sort of thing is the data: - Was survivorship bias accounted for? If you don't have a list of stocks that includes dead issues (bankrupt, merged, or closed) then all your value stocks will be the ones that had a bad period and then miraculously recovered. Your backtest would buy those and make money, while not seeing the ones that had a bad period and then went to zero. This also means you have to track whether the stock was tradable, whether due to the exchange announcing a freeze, or the market just seizing up as participants decided not to risk much. If your strategy does shorting this is going to cause even more pain as you look for historical hard-to-borrow data. - Was the data backfilled? This is the hardest one to drag out of data sales people, because they often just don't get it. If I have a company that reports its Q2 earnings about six weeks into Q3, is the data labelled as coming out in Q2? Because if it is, I'm backtesting on future information. Also firms that are in trouble may also have timeliness issues in getting their numbers out. The real reason why the index effect is gone is actually simple. For years now there's been a bunch of traders who build their models around the effect. Their methods are common sense: screen the market for firms that are not yet in the index, and rank them according to how likely they are to get included. Buy a lot of what you're certain will get in, a bit less if you're less certain, and so on. The effect of this is to give you a smoothed out effect so that you can't just hop on board right before inclusion anymore. It also seems likely that the big index investors actually hire a few of these experts to optimize their purchases. In fact I had a chat with a guy who did this yesterday. It turns out to follow an index, you don't need to buy the whole index. Stocks tend to move together so you can cover most of the principal components with just a tiny amount of stocks. You're not going to do this by buying the little stocks at the bottom of the index, you buy the big ones at the top.
- FredPret 4y ago
  Makes perfect sense. There were massive increases in certain SP500 stocks, but the majority goes nowhere. On the whole, if you have the financials of all companies, and apply certain variations of the cigar-butt strategy, you get out ahead. This not only makes sense logically but also works in backtesting. Value investing is a wide term that includes many strategies. Some of them have made quite a lot of money, very consistently.
  - Galanwe 4y ago
    > if you have the financials of all companies, and apply certain variations of the cigar-butt strategy, you get out ahead > This not only makes sense logically but also works in backtesting. No, it does not work at all in backtests. That's exactly what I'm talking about: a long short, normalized, rank based on e. g EPS. A typical value signal.
    - FredPret 4y ago
      Doesn’t work at all? Using “a common value signal”? There are many value signals that can be used. Some of which print money.
      - Galanwe 4y ago
        > There are many value signals that can be used. Some of which print money Please share portfolio weights of such a signal, so that I can check the value exposure and adjusted returns.
        edgyquant 4y ago
        The burden of proof is on the person claiming no form of value investing will have passed a backtest over the last decade. That’s you, not the other person.
        Galanwe 4y ago
        > The burden of proof is on the person claiming no form of value investing will have passed a backtest over the last decade Says who? In my book the burden of proof is on the argument being the less in line with general consensus. Value being a bad investment in the last decade is not an argument, it's a fact. Value is not something you find, it is a defined factor with academic consensus on the definition. You can have some leaway and build a portfolio highly correlated to value, but that only goes so far. Claiming to have found a good value signal is then akin to claiming being able to doing consistently good factor timing on value. This is a feat the top hedge funds on the planet would be weary to claim. Allow me to be skeptical when someone on HN claims to have done so with "basic value indicators" and found something that "prints money".
andrewmcwatters 4y ago
I think he says one thing to the crowd and does another thing in private. I remember him taking about discounted cash flow calculations and what maybe, supposedly, he and Munger use as the discount rate, but as soon as you apply this calculation to all 3000+ publicly traded US securities, you find that most of them are completely overvalued and have been for years. You can adjust the discount rate, but it doesn’t help in practice. Anyone with a computer and the patience to parse financial reports can do this, so of course people have done it. The problem is there aren’t many good investments actually there anymore. Period. If you’ve been investing the last few years and have made returns as all of us collectively have, you know those returns definitely aren’t coming from real gains. They’ve almost entirely been speculative returns. I was so jaded calculating this stuff and seeing marketable securities histories across various sectors that I came to realize all the free lunch has already been eaten. People are searching among the crumbs now. It’s just entirely an entrenched practice. There’s too many people in that game. I’m also a fan of the textbook guidelines from Security Analysis and The Intelligent Investor, but of course using those specific measurements also yield nothing worse investing in at the present date, and haven’t for years.[1] Everything is way more expensive than it should be valued at, and I suspect a part of it, too, is simply that a majority of shares are on the market and have no reason to be sold lower than what inflated values are presently passing for. It’s like owning a house knowing people need a place to live. People need securities for retirement. There’s only also many that exist. [1]: https://www.andrewmcwatters.org/personal-finance/#investments https://www.andrewmcwatters.org/personal-finance/#investment...
- jmole 4y ago
  The easiest way to make money is to convince the government to print more of it, and then take a percentage cut of the distribution and growth of the money supply. The financial industry has been doing this for decades.
- carabiner 4y ago
  I think it was his biographer, Alice Schroeder, who said that Buffett has never revealed his start to finish analysis for an investment. He's only talked about guidelines and principles, but it's like one thing to know F = m*a vs. how to design a bridge. Also, a lot of his money came from what was basically insider trading as a 20-something (chatting up CEO's and trading from that information).
asdfman123 4y ago
Aren’t you talking about index investing and not the “index effect” the article mentions? > the small rise in share value that once accompanied a stock’s addition to the index has disappeared on average over the past decade.
ant6n 4y ago
Where does one even get the data for a backtesting analysis?
- Galanwe 4y ago
  There's a gazillion market data vendors out there. Quandl and Tiingo are often used for personal/retail algorithmic day traders.
  - ant6n 4y ago
    Thanks for the recommendations. This Quandl company has a terrible website. They don't say what they offer and how much it costs, all is behind either creating an account or talking to their sales department. I don't understand how they suppossedly have 400K users.
- FredPret 4y ago
  It’s either free and hard or easy and expensive!
ye-olde-sysrq 4y ago
To everyone asking for more detail - don't bother. I don't mean to sound mean, OP, but if you have alpha ("several basic Buffett-style value strategies seem to beat the S&P 500 handily"), go start a prop shop and make millions of dollars. You can turn huge profits with even miniscule alpha. The problem is, armchair traders typically commit a few classical blunders: 1. Accidentally used future information to make trading decisions in backtesting. Obviously a crystal ball would make for GREAT alpha. And while you might think it's easy to avoid doing this, there can be some very subtle oopsies that are not apparent at all to even a somewhat sophisticated analyst that will catch you out. This was a common oopsie I would see our analysts do, sometimes only even becoming evident after the research team had vetted and idea and they hand it off to me to implement and I'd be sitting there in the data flow and have to tell them "yeah this data item you used, it's latency sensitive, did you make sure you were using the appropriate network path when you keyed into the dict or did you just hardcode the common one - because if you hardcoded then you're using a crystal ball for all the other network paths". 2. Assuming that if you see a price, you can trade at that price. Even HFTs can't just go get any ole' price they see. 2.5. A tangent to 2 is that your own trading activity can impact prices. Not important if your alpha is long term but if it's short term then this can be significant. 3. Ignored trading fees, and I don't mean brokerage fees - I mean SEC fees, etc that are usually very small for most retail trades so most brokerages don't even show you them (especially because depending on exactly how they executed your trade they might be different than you'd get if you just sent them to an exchange). 4. Not taking into account tail risks. Look at how many retail investors got blown the fuck up when XIV died [0] [0] https://www.forbes.com/sites/jimcollins/2018/02/06/it-works-until-it-doesnt-work-the-death-of-xiv-shows-the-folly-of-gaming-market-volatility/?sh=15659ad43c38 https://www.forbes.com/sites/jimcollins/2018/02/06/it-works-... Trading is hard. There's literally armies of ridiculously sophisticated actors as your market participant peers. I always used to joke that the federal budget is to defense spending as our nation's brainpower is to finance. (We probably throw WAY too much %-wise at it, but damn do we get a world class product as a result.)

jiqiren 4y ago

GitHub link?

satvikpendem 4y ago

Care to share the alpha with the rest of us? ;)

Galanwe 4y ago

That makes no sense. The last 10 to 20 years have seen a bullish round of momentum stocks negatively correlated to value loaded stocks. Value has consistently lost money in the last decade except for short times during momentum crashes in 2018 and 2023. Either your definition of value is inconsistent with the rest of the world, or your analysis is biased. Considering your sentence: > an algorithm to mass-analyze all the stocks on the US market I guess you reinvented some variation of"low vol" or "small vs big" signal. If you want to say you've beaten the S&P, you would have to show me a long short, beta neutral portfolio that does so using the same investment universe in a point in time manner.

jrockway 4y ago
An algorithm could be anything, right? Maybe they analyzed the middle school report cards of executives and majority shareholders, and tied that to value over time in a way that the S&P 500 did. Maybe there's some telling data point that people have never looked at before. I agree that predicting future price based on the current price has likely been tried to death by billions of hedge funds, but who knows, maybe they found something new that nobody has ever noticed. And, you can come up with the best stock trading algorithm in history and still not have the capital necessary to execute it, so it's possible that portfolio results don't matter. I'm not saying anything is likely here, merely that we have nowhere near enough data to even speculate or offer advice.
- Galanwe 4y ago
  > Maybe they analyzed the middle school report cards of executives and majority shareholders, and tied that to value over time in a way that the S&P 500 did. You are talking about ad-hoc signals here. Of course there are a lot of them that beat the benchmark, it's the basis of active management. But we're not talking about any signal here, we're talking about "value" - a signal so well known, studied and persistent we call it a "factor", and often decompose beta/sector/country adjusted residual returns _just_ based on a set of 4/5 of these factors.
carabiner 4y ago
Most finance comment threads on HN don't make sense. Just wait for Jumpcrisscross to drop in and forget the rest.
lordnacho 4y ago
The issue with backtesting this sort of thing is the data: - Was survivorship bias accounted for? If you don't have a list of stocks that includes dead issues (bankrupt, merged, or closed) then all your value stocks will be the ones that had a bad period and then miraculously recovered. Your backtest would buy those and make money, while not seeing the ones that had a bad period and then went to zero. This also means you have to track whether the stock was tradable, whether due to the exchange announcing a freeze, or the market just seizing up as participants decided not to risk much. If your strategy does shorting this is going to cause even more pain as you look for historical hard-to-borrow data. - Was the data backfilled? This is the hardest one to drag out of data sales people, because they often just don't get it. If I have a company that reports its Q2 earnings about six weeks into Q3, is the data labelled as coming out in Q2? Because if it is, I'm backtesting on future information. Also firms that are in trouble may also have timeliness issues in getting their numbers out. The real reason why the index effect is gone is actually simple. For years now there's been a bunch of traders who build their models around the effect. Their methods are common sense: screen the market for firms that are not yet in the index, and rank them according to how likely they are to get included. Buy a lot of what you're certain will get in, a bit less if you're less certain, and so on. The effect of this is to give you a smoothed out effect so that you can't just hop on board right before inclusion anymore. It also seems likely that the big index investors actually hire a few of these experts to optimize their purchases. In fact I had a chat with a guy who did this yesterday. It turns out to follow an index, you don't need to buy the whole index. Stocks tend to move together so you can cover most of the principal components with just a tiny amount of stocks. You're not going to do this by buying the little stocks at the bottom of the index, you buy the big ones at the top.
FredPret 4y ago
Makes perfect sense. There were massive increases in certain SP500 stocks, but the majority goes nowhere. On the whole, if you have the financials of all companies, and apply certain variations of the cigar-butt strategy, you get out ahead. This not only makes sense logically but also works in backtesting. Value investing is a wide term that includes many strategies. Some of them have made quite a lot of money, very consistently.
- Galanwe 4y ago
  > if you have the financials of all companies, and apply certain variations of the cigar-butt strategy, you get out ahead > This not only makes sense logically but also works in backtesting. No, it does not work at all in backtests. That's exactly what I'm talking about: a long short, normalized, rank based on e. g EPS. A typical value signal.
  - FredPret 4y ago
    Doesn’t work at all? Using “a common value signal”? There are many value signals that can be used. Some of which print money.
    - Galanwe 4y ago
      > There are many value signals that can be used. Some of which print money Please share portfolio weights of such a signal, so that I can check the value exposure and adjusted returns.
      - edgyquant 4y ago
        The burden of proof is on the person claiming no form of value investing will have passed a backtest over the last decade. That’s you, not the other person.
        Galanwe 4y ago
        > The burden of proof is on the person claiming no form of value investing will have passed a backtest over the last decade Says who? In my book the burden of proof is on the argument being the less in line with general consensus. Value being a bad investment in the last decade is not an argument, it's a fact. Value is not something you find, it is a defined factor with academic consensus on the definition. You can have some leaway and build a portfolio highly correlated to value, but that only goes so far. Claiming to have found a good value signal is then akin to claiming being able to doing consistently good factor timing on value. This is a feat the top hedge funds on the planet would be weary to claim. Allow me to be skeptical when someone on HN claims to have done so with "basic value indicators" and found something that "prints money".

andrewmcwatters 4y ago

I think he says one thing to the crowd and does another thing in private. I remember him taking about discounted cash flow calculations and what maybe, supposedly, he and Munger use as the discount rate, but as soon as you apply this calculation to all 3000+ publicly traded US securities, you find that most of them are completely overvalued and have been for years. You can adjust the discount rate, but it doesn’t help in practice. Anyone with a computer and the patience to parse financial reports can do this, so of course people have done it. The problem is there aren’t many good investments actually there anymore. Period. If you’ve been investing the last few years and have made returns as all of us collectively have, you know those returns definitely aren’t coming from real gains. They’ve almost entirely been speculative returns. I was so jaded calculating this stuff and seeing marketable securities histories across various sectors that I came to realize all the free lunch has already been eaten. People are searching among the crumbs now. It’s just entirely an entrenched practice. There’s too many people in that game. I’m also a fan of the textbook guidelines from Security Analysis and The Intelligent Investor, but of course using those specific measurements also yield nothing worse investing in at the present date, and haven’t for years.[1] Everything is way more expensive than it should be valued at, and I suspect a part of it, too, is simply that a majority of shares are on the market and have no reason to be sold lower than what inflated values are presently passing for. It’s like owning a house knowing people need a place to live. People need securities for retirement. There’s only also many that exist. [1]: https://www.andrewmcwatters.org/personal-finance/#investments https://www.andrewmcwatters.org/personal-finance/#investment...

jmole 4y ago
The easiest way to make money is to convince the government to print more of it, and then take a percentage cut of the distribution and growth of the money supply. The financial industry has been doing this for decades.
carabiner 4y ago
I think it was his biographer, Alice Schroeder, who said that Buffett has never revealed his start to finish analysis for an investment. He's only talked about guidelines and principles, but it's like one thing to know F = m*a vs. how to design a bridge. Also, a lot of his money came from what was basically insider trading as a 20-something (chatting up CEO's and trading from that information).

asdfman123 4y ago

Aren’t you talking about index investing and not the “index effect” the article mentions? > the small rise in share value that once accompanied a stock’s addition to the index has disappeared on average over the past decade.

ant6n 4y ago

Where does one even get the data for a backtesting analysis?

Galanwe 4y ago
There's a gazillion market data vendors out there. Quandl and Tiingo are often used for personal/retail algorithmic day traders.
- ant6n 4y ago
  Thanks for the recommendations. This Quandl company has a terrible website. They don't say what they offer and how much it costs, all is behind either creating an account or talking to their sales department. I don't understand how they suppossedly have 400K users.
FredPret 4y ago
It’s either free and hard or easy and expensive!

ye-olde-sysrq 4y ago

To everyone asking for more detail - don't bother. I don't mean to sound mean, OP, but if you have alpha ("several basic Buffett-style value strategies seem to beat the S&P 500 handily"), go start a prop shop and make millions of dollars. You can turn huge profits with even miniscule alpha. The problem is, armchair traders typically commit a few classical blunders: 1. Accidentally used future information to make trading decisions in backtesting. Obviously a crystal ball would make for GREAT alpha. And while you might think it's easy to avoid doing this, there can be some very subtle oopsies that are not apparent at all to even a somewhat sophisticated analyst that will catch you out. This was a common oopsie I would see our analysts do, sometimes only even becoming evident after the research team had vetted and idea and they hand it off to me to implement and I'd be sitting there in the data flow and have to tell them "yeah this data item you used, it's latency sensitive, did you make sure you were using the appropriate network path when you keyed into the dict or did you just hardcode the common one - because if you hardcoded then you're using a crystal ball for all the other network paths". 2. Assuming that if you see a price, you can trade at that price. Even HFTs can't just go get any ole' price they see. 2.5. A tangent to 2 is that your own trading activity can impact prices. Not important if your alpha is long term but if it's short term then this can be significant. 3. Ignored trading fees, and I don't mean brokerage fees - I mean SEC fees, etc that are usually very small for most retail trades so most brokerages don't even show you them (especially because depending on exactly how they executed your trade they might be different than you'd get if you just sent them to an exchange). 4. Not taking into account tail risks. Look at how many retail investors got blown the fuck up when XIV died [0] [0] https://www.forbes.com/sites/jimcollins/2018/02/06/it-works-until-it-doesnt-work-the-death-of-xiv-shows-the-folly-of-gaming-market-volatility/?sh=15659ad43c38 https://www.forbes.com/sites/jimcollins/2018/02/06/it-works-... Trading is hard. There's literally armies of ridiculously sophisticated actors as your market participant peers. I always used to joke that the federal budget is to defense spending as our nation's brainpower is to finance. (We probably throw WAY too much %-wise at it, but damn do we get a world class product as a result.)