6 ms·

I think its helpful to realize that the S&P 500 is not some static index of companies whose selection has been on auto-pilot but actually a fairly actively mana

I think its helpful to realize that the S&P 500 is not some static index of companies whose selection has been on auto-pilot but actually a fairly actively managed index whose selection algorithm is continuously being updated and refined. There is a not insignificant human judgement component of how to define the selection/weighting criteria. In some senses, behind the scenes its likely similar to the Twitter/FB/Netflix algorithm - at some point there is some editorial opinion being inserted via criteria and weighing of those criteria. "Some years ago Bill Miller, then a well-known Legg Mason portfolio manager with a stellar record of beating the market, noted that the S&P 500’s track record of being very hard to beat suggested that active management can succeed and that the Index Committee were actually good active managers." https://www.indexologyblog.com/2014/08/07/inside-the-sp-500-an-active-committee/ https://www.indexologyblog.com/2014/08/07/inside-the-sp-500-...

anonu 4y ago
Yes there's an index committee comprised of humans. But the index methodology is public and when there are tie break decisions needed, they're fairly predictable. Another thing to think about is that most of the outperformance of the index is driven by the largest companies. But not because they were added when they were their current size. But because of the buy and hold approach of getting them in when they're still on the small end of the large caps. There was a good recent paper on this concept, showcasing that most of the big index returns are attributable to only a few stocks. Similar to how most of global market returns are mostly just US.
- herostratus101 4y ago
  I think you mean that the committee comprises humans.
- somethoughts 4y ago
  Definitely agree the algorithm is likely pretty well documented but as noted in the blog it is adapted in realtime. Here's the current breakdown: Apple 7.08%, Microsoft 6.20%, Amazon.com 2.62%, NVIDIA 1.87%, Alphabet 1.84% Berkshire 1.65% Alphabet 1.61% Tesla 1.44% which if it was an active manager would be highly opinionated.
  - singhrac 4y ago
    Really? That's essentially just the market cap weights, which is the most default thing you could do. There's a slight deviation because it's float-adjusted and I'm assuming their marks are at some frequency, but those numbers aren't mysteries.
    - pavlov 4y ago
      I think the point is that the divergence in market cap is so large that it’s become equivalent to a highly opinionated active manager. Any fund that’s allocating over 13% to just two companies seems a bit risky, but that’s what you’re getting with Apple and Microsoft here.
      - sokoloff 4y ago
        I don’t think it’s as opinionated nor active as you’re making it out to be. Investing proportionally more in a huge company than you do in the 500th largest eligible company seems much more passive than active to me. If two S&P companies were to merge and everything else remained the same, I’d think that having the combined amount invested in the combined company to be more sensible than cutting your investment proportion in half merely by a merger. Or if a company you’d invested in doubled in value, I’d rather hold the constant number of shares rather than sell half of them to bring my exposure to the winner down. I’d also view that as passive and un-opinionated. Selling half feels more active and opinionated than simply holding.
      - deleted 4y ago
        [deleted]
      - singhrac 4y ago
        I’m sorry, but this is just not true. Active managers are measured by their deviation from the S&P (i.e. a beta 1 manager is extremely passive). This index has also always been concentrated to some extent, so not sure it’s any kind of recent change. I think maybe you are talking about risk, but it’s a completely separate idea.
  - DesiLurker 4y ago
- 88913527 4y ago
  Is the algorithm "select * from companies order by market_cap desc limit 500"? At the risk of oversimplifying, there should be quite little to debate, and when some hair-splitting occurs, I trust a committee of reasonable humans will make a reasonable choice.
  - mandevil 4y ago
    The algorithm is a little more complicated than that, as various judgements have been added in, but always with specific rules. When there was a trend of tech companies going public with dual classes of shares leaving founders in control forever (e.g. Zuck with Facebook) they announced that any company which went public after date X with dual classes to prevent the economic owners from exercising control would not be allowed onto the index. Similarly, there are rules about profitability that, IIRC, were added after the first dotcom bubble to try and keep extreme bubbles from mucking up the index (I think it's that you need to have two consecutive quarters of profitability, but I don't remember the details). Also, the market cap of company 450 and company 550 are close enough that recent market performance can flop ]back and forth fairly regularly, which the S&P has further rules to try and limit, though again I don't remember the details. Again, they try to be fairly rules based, but do exercise some discretion in creating the rules.
  - anonu 4y ago
    You have to adjust for free float market cap. Companies need to be US incorporated. You have to be aware of share classes. For example both GOOG share classes are in the S&P. When it comes to tiebreak decisions, there may be 5 or so potential adds when there is 1 delete. There may be a focus on evening out sector exposures, preferencing underweight sector adds. (Just my guess)
- jackcosgrove 4y ago
  That's interesting. I was contacted by a personal money manager looking for clients and he explained his philosophy as sampling 30-50 stocks from the top end of the S&P 500. He had outperformed the market during a bull run, which I attributed to the increased risk of a smaller portfolio. I didn't invest because I figured his fund would underperform the index during the next bear run for that same reason, canceling out the gains and moreso with fees. Maybe there actually was something to his strategy; I haven't followed up on his performance.
  - robocat 4y ago
    Owning a handful of shares does capture a significant amount of the beta for a market. However with very significant profitability outliers, such as power law distributions, then you have to own all the shares to ensure you don’t miss out on the one or two shares that make up most of the gains. Venture capitalists talk about this a lot. Historically the same issue occurred with market timing: most market gains are made during only a few relatively short unpredictable periods. You need to own shares for a long time to ensure you are in the market when those few unpredictable ups occur.
  - bombcar 4y ago
    IIRC many active managers beat the market/S&P 500/benchmark before fees. It's almost impossible to beat after fees because the fees are too much of a drag.
ransom1538 4y ago
Agreed. I have ruined so many dinner conversations over this. You can't beat the SP500, it is the only free lunch. It is truly depressing when you think about it - you just feed the machine. The American millionare class is a group of 401k holders. [Oh. and they made 54.96% in the last 5 years]
- geysersam 4y ago
  > The American millionare class is a group of 401k holders. What do you mean by this? That pension saving make up the bulk of wealth in the US? Because from what I can gather that's not correct.
  - ransom1538 4y ago
    Your best odds of being an american millionare is pushing all you can into a 401k which purchases sp500. Most american millionares are not business ninjas. They are teachers and firemen pushing into their 401k, they also don't make much money. "Only 31% averaged $100,000 a year over the course of their career." What happens is the ridiculous return of the sp500. Sorry, about this link in advance: https://www.ramseysolutions.com/retirement/the-national-study-of-millionaires-research https://www.ramseysolutions.com/retirement/the-national-stud...
  - sokoloff 4y ago
    Being in the millionaire class in the US isn’t that rare[1] and very often is achieved by real estate and retirement savings. [1]-It’s about 9% of American adults.
    - abdullahkhalids 4y ago
      > It’s about 9% of American adults. How is this counted wrt spouses?
    - geysersam 4y ago
      Yes but the vast bulk of wealth is owned by a very small minority. That's who people are generally talking about when referring to the "millionaire class".
throw0101b 4y ago
> * There is a not insignificant human judgement component of how to define the selection/weighting criteria.* Not wrong, but the rules are relatively fixed and known ahead of time and somewhat more deterministic compared to the decision-making process of most active funds. It should also be noted that the S&P 500 isn't the only index. The Russell 3000 and Wilshire 5000 try to cover "all" publicly trade companies in the US: * https://en.wikipedia.org/wiki/Russell_3000_Index https://en.wikipedia.org/wiki/Russell_3000_Index * https://en.wikipedia.org/wiki/Wilshire_5000 https://en.wikipedia.org/wiki/Wilshire_5000 But "passive" investing does exist on a spectrum: > The terms passive investing and index investing are often intertwined, but they are not exactly the same thing. Today’s guest is Adriana Robertson, the Honourable Justice Frank Iacobucci Chair in Capital Markets Regulation […].Adriana is interested in index investing and, in this episode, we hear her views on whether or not index investing is passive. Hear facts from her paper on the S&P 500 Index fund specifically, and all of the reasons that it's not passive, as well as some of the issues that are potentially arising from the creation of so many indexes or so-called passive investments. A more recent paper by Adriana, published in The Journal of Finance, surveyed a representative sample of U.S. individual investors about how well leading academic theories describe their financial beliefs and decisions, and Adriana shares the differences in something like value growth from an academic perspective versus a real-world perspective. Find out how investors can go about evaluating the performance of their portfolios and what they should be looking for when deciding which index fund to invest in, as well as why index funds aren’t a meaningful category anyway, factors from Adriana’s surveys that might influence investor’s equity allocation, and the trend towards indexing and whether it will overtake active portfolios. Tune in today for all this and more! * https://rationalreminder.ca/podcast/133 https://rationalreminder.ca/podcast/133
rthomas6 4y ago
Yeah but the Russell 3000 tends to slightly beat the S&P 500, and it comprises 98% of the market.
- maest 4y ago
  That's just the size premium.
scotty79 4y ago
> Index Committee were actually good active managers. It's as if researchers complained that placebo is too therapeutic.
ncr100 4y ago
.. and it's "THE" S&P 500 ... popularity surely has an impact on whether an investor will simply choose to invest in a member of the '500.
hartator 4y ago
That’s not true. The SP500 index might be a little more complicated than just the 500 biggest public companies. When Tesla was rejected despite meeting criteria, it was odd. However “edition” impacts less than 1% the actual value. If you can compare VOO (Vanguard sp500 tracking) vs VV (Vanguard 566 large cap), it’s the same: https://www.etf.com/etfanalytics/etf-comparison-tool/VOO-vs-VV https://www.etf.com/etfanalytics/etf-comparison-tool/VOO-vs-... (20 basis-point difference)
noloblo 4y ago
archive unpaywall link: https://archive.is/20230308071523/https://www.ft.com/content/c331b279-00e5-4e7d-954f-468f4ee44c31 https://archive.is/20230308071523/https://www.ft.com/content...
deleted 4y ago
[deleted]
andallthat 4y ago
They are not actually selling stock that's not performing well and buying stock that is, though, are they? It's very hard to beat their performance if they can pick winners at any time and just discard losers without any actual loss.

anonu 4y ago

Yes there's an index committee comprised of humans. But the index methodology is public and when there are tie break decisions needed, they're fairly predictable. Another thing to think about is that most of the outperformance of the index is driven by the largest companies. But not because they were added when they were their current size. But because of the buy and hold approach of getting them in when they're still on the small end of the large caps. There was a good recent paper on this concept, showcasing that most of the big index returns are attributable to only a few stocks. Similar to how most of global market returns are mostly just US.

herostratus101 4y ago
I think you mean that the committee comprises humans.
somethoughts 4y ago
Definitely agree the algorithm is likely pretty well documented but as noted in the blog it is adapted in realtime. Here's the current breakdown: Apple 7.08%, Microsoft 6.20%, Amazon.com 2.62%, NVIDIA 1.87%, Alphabet 1.84% Berkshire 1.65% Alphabet 1.61% Tesla 1.44% which if it was an active manager would be highly opinionated.
- singhrac 4y ago
  Really? That's essentially just the market cap weights, which is the most default thing you could do. There's a slight deviation because it's float-adjusted and I'm assuming their marks are at some frequency, but those numbers aren't mysteries.
  - pavlov 4y ago
    I think the point is that the divergence in market cap is so large that it’s become equivalent to a highly opinionated active manager. Any fund that’s allocating over 13% to just two companies seems a bit risky, but that’s what you’re getting with Apple and Microsoft here.
    - sokoloff 4y ago
      I don’t think it’s as opinionated nor active as you’re making it out to be. Investing proportionally more in a huge company than you do in the 500th largest eligible company seems much more passive than active to me. If two S&P companies were to merge and everything else remained the same, I’d think that having the combined amount invested in the combined company to be more sensible than cutting your investment proportion in half merely by a merger. Or if a company you’d invested in doubled in value, I’d rather hold the constant number of shares rather than sell half of them to bring my exposure to the winner down. I’d also view that as passive and un-opinionated. Selling half feels more active and opinionated than simply holding.
    - deleted 4y ago
      [deleted]
    - singhrac 4y ago
      I’m sorry, but this is just not true. Active managers are measured by their deviation from the S&P (i.e. a beta 1 manager is extremely passive). This index has also always been concentrated to some extent, so not sure it’s any kind of recent change. I think maybe you are talking about risk, but it’s a completely separate idea.
- DesiLurker 4y ago
88913527 4y ago
Is the algorithm "select * from companies order by market_cap desc limit 500"? At the risk of oversimplifying, there should be quite little to debate, and when some hair-splitting occurs, I trust a committee of reasonable humans will make a reasonable choice.
- mandevil 4y ago
  The algorithm is a little more complicated than that, as various judgements have been added in, but always with specific rules. When there was a trend of tech companies going public with dual classes of shares leaving founders in control forever (e.g. Zuck with Facebook) they announced that any company which went public after date X with dual classes to prevent the economic owners from exercising control would not be allowed onto the index. Similarly, there are rules about profitability that, IIRC, were added after the first dotcom bubble to try and keep extreme bubbles from mucking up the index (I think it's that you need to have two consecutive quarters of profitability, but I don't remember the details). Also, the market cap of company 450 and company 550 are close enough that recent market performance can flop ]back and forth fairly regularly, which the S&P has further rules to try and limit, though again I don't remember the details. Again, they try to be fairly rules based, but do exercise some discretion in creating the rules.
- anonu 4y ago
  You have to adjust for free float market cap. Companies need to be US incorporated. You have to be aware of share classes. For example both GOOG share classes are in the S&P. When it comes to tiebreak decisions, there may be 5 or so potential adds when there is 1 delete. There may be a focus on evening out sector exposures, preferencing underweight sector adds. (Just my guess)
jackcosgrove 4y ago
That's interesting. I was contacted by a personal money manager looking for clients and he explained his philosophy as sampling 30-50 stocks from the top end of the S&P 500. He had outperformed the market during a bull run, which I attributed to the increased risk of a smaller portfolio. I didn't invest because I figured his fund would underperform the index during the next bear run for that same reason, canceling out the gains and moreso with fees. Maybe there actually was something to his strategy; I haven't followed up on his performance.
- robocat 4y ago
  Owning a handful of shares does capture a significant amount of the beta for a market. However with very significant profitability outliers, such as power law distributions, then you have to own all the shares to ensure you don’t miss out on the one or two shares that make up most of the gains. Venture capitalists talk about this a lot. Historically the same issue occurred with market timing: most market gains are made during only a few relatively short unpredictable periods. You need to own shares for a long time to ensure you are in the market when those few unpredictable ups occur.
- bombcar 4y ago
  IIRC many active managers beat the market/S&P 500/benchmark before fees. It's almost impossible to beat after fees because the fees are too much of a drag.

ransom1538 4y ago

Agreed. I have ruined so many dinner conversations over this. You can't beat the SP500, it is the only free lunch. It is truly depressing when you think about it - you just feed the machine. The American millionare class is a group of 401k holders. [Oh. and they made 54.96% in the last 5 years]

geysersam 4y ago
> The American millionare class is a group of 401k holders. What do you mean by this? That pension saving make up the bulk of wealth in the US? Because from what I can gather that's not correct.
- ransom1538 4y ago
  Your best odds of being an american millionare is pushing all you can into a 401k which purchases sp500. Most american millionares are not business ninjas. They are teachers and firemen pushing into their 401k, they also don't make much money. "Only 31% averaged $100,000 a year over the course of their career." What happens is the ridiculous return of the sp500. Sorry, about this link in advance: https://www.ramseysolutions.com/retirement/the-national-study-of-millionaires-research https://www.ramseysolutions.com/retirement/the-national-stud...
- sokoloff 4y ago
  Being in the millionaire class in the US isn’t that rare[1] and very often is achieved by real estate and retirement savings. [1]-It’s about 9% of American adults.
  - abdullahkhalids 4y ago
    > It’s about 9% of American adults. How is this counted wrt spouses?
  - geysersam 4y ago
    Yes but the vast bulk of wealth is owned by a very small minority. That's who people are generally talking about when referring to the "millionaire class".

throw0101b 4y ago

> * There is a not insignificant human judgement component of how to define the selection/weighting criteria.* Not wrong, but the rules are relatively fixed and known ahead of time and somewhat more deterministic compared to the decision-making process of most active funds. It should also be noted that the S&P 500 isn't the only index. The Russell 3000 and Wilshire 5000 try to cover "all" publicly trade companies in the US: * https://en.wikipedia.org/wiki/Russell_3000_Index https://en.wikipedia.org/wiki/Russell_3000_Index * https://en.wikipedia.org/wiki/Wilshire_5000 https://en.wikipedia.org/wiki/Wilshire_5000 But "passive" investing does exist on a spectrum: > The terms passive investing and index investing are often intertwined, but they are not exactly the same thing. Today’s guest is Adriana Robertson, the Honourable Justice Frank Iacobucci Chair in Capital Markets Regulation […].Adriana is interested in index investing and, in this episode, we hear her views on whether or not index investing is passive. Hear facts from her paper on the S&P 500 Index fund specifically, and all of the reasons that it's not passive, as well as some of the issues that are potentially arising from the creation of so many indexes or so-called passive investments. A more recent paper by Adriana, published in The Journal of Finance, surveyed a representative sample of U.S. individual investors about how well leading academic theories describe their financial beliefs and decisions, and Adriana shares the differences in something like value growth from an academic perspective versus a real-world perspective. Find out how investors can go about evaluating the performance of their portfolios and what they should be looking for when deciding which index fund to invest in, as well as why index funds aren’t a meaningful category anyway, factors from Adriana’s surveys that might influence investor’s equity allocation, and the trend towards indexing and whether it will overtake active portfolios. Tune in today for all this and more! * https://rationalreminder.ca/podcast/133 https://rationalreminder.ca/podcast/133

rthomas6 4y ago

Yeah but the Russell 3000 tends to slightly beat the S&P 500, and it comprises 98% of the market.

maest 4y ago
That's just the size premium.

scotty79 4y ago

> Index Committee were actually good active managers. It's as if researchers complained that placebo is too therapeutic.

ncr100 4y ago

.. and it's "THE" S&P 500 ... popularity surely has an impact on whether an investor will simply choose to invest in a member of the '500.

hartator 4y ago

That’s not true. The SP500 index might be a little more complicated than just the 500 biggest public companies. When Tesla was rejected despite meeting criteria, it was odd. However “edition” impacts less than 1% the actual value. If you can compare VOO (Vanguard sp500 tracking) vs VV (Vanguard 566 large cap), it’s the same: https://www.etf.com/etfanalytics/etf-comparison-tool/VOO-vs-VV https://www.etf.com/etfanalytics/etf-comparison-tool/VOO-vs-... (20 basis-point difference)

noloblo 4y ago

archive unpaywall link: https://archive.is/20230308071523/https://www.ft.com/content/c331b279-00e5-4e7d-954f-468f4ee44c31 https://archive.is/20230308071523/https://www.ft.com/content...

deleted 4y ago

[deleted]

andallthat 4y ago

They are not actually selling stock that's not performing well and buying stock that is, though, are they? It's very hard to beat their performance if they can pick winners at any time and just discard losers without any actual loss.