12 ms·

Show HN: Find the 10 highest and 10 lowest correlations to any stock

217 points by macco 4y ago

During the start of the year I was thinking how could I bet against certain stocks (in my case mainly Tesla) without using derivatives and the risks that come with them. After I had success betting on the oil price with a highly correlated investment fond, I came to the conclusion that negative correlations could be used to bet against the price of other assets. Unfortunately, it is not easy to find correlations between assets if you don't know which assets to compare in the first place. So I created a website where you can find the 10 highest and 10 lowest correlations of certain assets.

lupire 4y ago
Bet Against? I thought the opposite of buying a single stock (Alpha) would be in service of.... Beta Gains T.
noduerme 4y ago
And the answer is... Nvidia. Cool idea, by the way. I'll have to play with this more.
vortico 4y ago
Very cool, although index funds like S&P 500 and DJI don't seem to work. https://betagainst.fun/asset/_gspc_spy/ https://betagainst.fun/asset/_gspc_spy/ https://betagainst.fun/asset/_dji_dia/ https://betagainst.fun/asset/_dji_dia/
- andrewmcwatters 4y ago
  That doesn't make any sense anyway. You're asking what are the highest and lowest correlations of the market against the market itself. You cannot compare the same or roughly the same basket of assets against itself.
  - scarmig 4y ago
    That still makes sense. Something can be more or less correlated with the broader market. It's very useful, actually: you can use securities that tend to not be correlated with the broader market to build a portfolio with less average variance.
    - andrewmcwatters 4y ago
      Yeah, I understand the concept, its just that regardless that people can graph it, doesn't mean that it widely has any tangible meaning other than being able to play with the numbers that come out of it. It's a concept that fits nicely in the category of technical analysis, but not one that allows you to find any particular market insights. If you took the whole tradable US securities market and said "what doesn't correlate with these price movements?" it doesn't mean anything. If it did, I would ask you, "What explicitly do you _think_ this means?" Just because you get numbers in and out doesn't mean you're working with anything meaningful. The only broadly meaningful concept you can get out of anti correlation with the market would be bonds because interest rates increasing push down the estimated future cash flows of publicly traded companies, affecting their valuations. Everything else can be considered speculation.
      - nickles 4y ago
        > It's a concept that fits nicely in the category of technical analysis, but not one that allows you to find any particular market insights. It's useful for hedging risk > The only broadly meaningful concept you can get out of anti correlation with the market would be bonds because interest rates increasing push down the estimated future cash flows of publicly traded companies, affecting their valuations. This is wrong. Consider pair trading or long/short strategies, both of which rely on estimating correlations.
      - JackFr 4y ago
        The correlation of an asset to the market is the definition of 'beta' and it is not technical analysis (from investopedia): "Beta is a measure used in fundamental analysis to determine the volatility of an asset or portfolio in relation to the overall market." "Technical analysis is a trading discipline employed to evaluate investments and identify trading opportunities in price trends and patterns seen on charts."
        andrewmcwatters 4y ago
        I know what Investopedia says, but there's nothing fundamental about beta. Beta is the "measure of how an individual asset moves (on average) when the overall stock market increases or decreases." That's just looking at pricing. It's the same category as technical analysis.
  - greatwave1 4y ago
    Yes, it makes sense. An asset's correlation are with the broader stock market is quantifiable. See also: https://www.investopedia.com/terms/b/beta.asp https://www.investopedia.com/terms/b/beta.asp
    - andrewmcwatters 4y ago
      No, it doesn't. The S&P 500 basically is the market. Any meaningful difference between it and the other US investable equities is marginally notable. It's literally asking "How do these roughly 500 plus or minus largest stocks correlate with the largely the same basket of securities?" If you took one company and compared it against the total US market, then you have something at least. If you take the market itself and compare it against itself, it makes no sense.
      - nickles 4y ago
        > If you took one company and compared it against the total US market, then you have something at least. This is a common metric referred to as market beta. > If you take the market itself and compare it against itself, it makes no sense. S&P 500 and Russell 2000 are both broad market indices, but they perform differently. Even using the same index constituents with different weightings (e.g. equal vs cap weighted) can produce meaningfully different results. It makes plenty of sense to compare them.
        andrewmcwatters 4y ago
        I don't think you understand why multiple broad market indices exist in the first place. It has nothing to do with "performance" and everything to do with licensing fees when institutional organizations want to track against the market. No one. No one is seriously looking at the Russell 3000 and comparing it to the Wilshire 5000. To the child comment: I'm talking about two total market indices, apples to apples, and you're talking about apples to oranges. NASDAQ is a stock exchange, not an index. Further still, even if you were talking about the NASDAQ composite, they're two entirely different indices. They do not have the same goals. There might be some relative meaning there. There isn't much meaning between comparing two or more indices that have the same goals and constituents. You're only tracking the differences between constituents at that point, and you wouldn't need beta to do that. You could just calculate the difference between the constituents that are not a part of the total set.
daxaxelrod 4y ago
Getting 429 status codes after I hit search
- LinuxBender 4y ago
  And I get a blank page. Maybe because I block the trackers?
omarhaneef 4y ago
I wonder if correlation is the best measure. Cointegration might be more important for this sort of trading. And might be a supplementary measure you want to add since you’re doing all the pairwise comparisons. (If I recall there is a computational shortcut that helps)
deleted 4y ago
[deleted]
tdehnel 4y ago
I’ve often wondered the same thing, good on you for actually doing it. That said, in any set of 20+ variables there will be a 10 highest/10 lowest correlating. Without a good (specific, hard to vary) explanation as to why the correlation happens, I would not use this information to gamble real money.
- onlyrealcuzzo 4y ago
  It's the 10 highest and lowest SO FAR. If you track this over time, you will see that these change (often at times when you really want them not to). Past observances aren't super helpful in predicting fractal futures.
  - somedude895 4y ago
    Would be interesting to see the differences in correlation from say 1980-2000 to 2000-2020. Of course that would exclude many of today's heavyweights.
- hedora 4y ago
  > Step 3: Check
Bostonian 4y ago
"We calculate the correlations between 2 securities on the daily closing values of the last 20 years." It's better to correlate daily returns than daily prices, since the latter are nonstationary, and I suggest using 1 year of daily returns rather than 20 since correlations do change over time. When I worked as a financial quant no one looked at 20 year correlations to measure near-term risk.
- danuker 4y ago
  For a retail trader, the fees are higher, affording fewer trades in a given period. As such the holding period might be on the order of 1 year for some people, so just 1 year of daily returns might invite too much trading if you're rebalancing.
  - nurettin 4y ago
    Retail traders should be realizing gains within 2-3 month intervals. Retail investors will usually hold for years and do average pricing until the next market crash
    - Bostonian 4y ago
      "Retail traders should be realizing gains within 2-3 month intervals." Not in taxable accounts where short term capital gains are taxed at ordinary income tax rates (or at an even higher rate in Massachusetts). It's difficult to beat index funds on a pre-tax basis. On an after-tax basis it is almost impossible with discretionary stock picking.
cosmic_quanta 4y ago
Very cool! Where are you getting the historical data? There's a ton of fun stuff you can do with this kind of long-horizon of data
lesgobrandon 4y ago
runeks 4y ago
Super useful! However, I get a “504 Gateway Time-out” error for https://betagainst.fun/asset/bz__f_bno/ https://betagainst.fun/asset/bz__f_bno/. HN hug of death?
- davidkuennen 4y ago
  I suspect HN hug of death
  - elihu 4y ago
    I think so. It seems to be running a lot smoother now than it was earlier today.
- jedberg 4y ago
  Do you have an Orbi or Netgear router? The service they use to block harmful websites has list this one as harmful.
  - runeks 4y ago
    Nope; I have a TP-Link router.
LudwigNagasena 4y ago
I don't think the method is sound. You should look into finance literature on the topic, maybe something about diversification.
- PaulHoule 4y ago
  It's pretty notorious for not working. With few exceptions, all stocks are correlated to the broad market. https://en.wikipedia.org/wiki/Beta_(finance) https://en.wikipedia.org/wiki/Beta_(finance) One explanation that would be popular right now is that the price of stocks can be calculated based on expected future cash flows and interest rates (e.g. think of what you'd pay for a bond that produces the same csh flow.) Since the interest rate is a factor in that calculation for all stocks, there's a correlation right there. The exceptions tend to be sketchy. At one time it was thought that gold mining stocks would move against the market, but what I heard was that gold mining firms are badly run and if you like gold you should just buy gold.
  - thom 4y ago
    People still do pair trading though, right?
    - PaulHoule 4y ago
      That's kinda the opposite of what that site is advocating since in pair trading you are going long one and short the other. https://en.wikipedia.org/wiki/Pairs_trade https://en.wikipedia.org/wiki/Pairs_trade The most famous pair trade was this: in the morning of a trading day you buy the stocks that performed the worst yesterday and short the ones that performed the best. Because the market reverts to the mean, this strategy made money hand over fist from the 1980s when Morgan Stanley started using it until the summer of 2007 when somebody fat fingered a button, unwinded their position during the day, and crashed everybody else who was in this trade. The profitability of that trade went downhill long before that crash but since then it's been impossible to make money doing it. (When there's a hedge fund bubble the profitability of a trade drops gradually as more people pile into it and the market inefficiency it targets is dispelled, quite different from an equity bubble which goes up dramatically until...)
snake_doc 4y ago
In many ways, compared to a vanilla short position or a synthetic short via derivatives, you are implicitly accepting higher risk to “short” via this manner. This is generally a bad idea as it’s hard enough to arbitrage the same equity on different exchanges[1]. Now imagine trying locate perfect negative substitutes for an equity… [1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=525282 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=525282
- joelthelion 4y ago
  I agree that this is very dubious. However, looking at the data sounds fun, if only I could get past the 50x errors :)
- macco 4y ago
  I highly doubt your assessment. When you have a short position, with underlying values you don't own, you have a leveraged investment. Which is inherently more risky than investments in a base value. The idea is not to have perfect negative correlations, but to find values with some negative correlation. This way you can profit from falling prices of an asset. Not with a 1 to 1 yield, or even higher.
  - snake_doc 4y ago
    In equity markets, risk is measured in volatility or the standard deviation of return. By definition a short and long position of an equity has the same volatility or standard deviation of returns. You may be using the term risk colloquially, but your understanding of leverage under a short sale is also misguided. If you short a stock and have cash to cover, you don’t need leverage. If you short a stock and the position is in the black, you don’t need leverage. You only need leverage if your short position is in the net negative. Yes, you may have to pay borrowing costs for the stock, but that’s just normally known as transaction costs. Until you require leverage to keep the short position open when it’s net negative or you used the cash proceeds, you are not leveraged. If your goal is to manage the maximum loss from a short position, it’s much easier to short the stock and buy a protective put. This gives you perfect exposure to your bet, while allowing you to manage your loss exactly.
marcrosoft 4y ago
Nitpick: you can’t lose over 100% of your investment using a put option. You can only lose the premium.
- akg_67 4y ago
  You can lose up to 100% of premium (which makes up 100% of your investment in that option) if you BOUGHT the put option. You can lose up to Strike Price minus the premium received if you SOLD the put option.
  - sakopov 4y ago
    I know next nothing about option trading, but doesn't betting against a stock have infinite loses?
    - plouffy 4y ago
      Not with options. If you want to bet against a company you with options you can do it by either: 1. Buy a put. The put gives you the right to sell the stock at a certain price. If it goes below that price you make a profit, if it doesn’t your put is worthless. The maximum you can lose is what you paid for the put. 2. Sell a Call. This gives someone the right to buy the stock at a certain price (and you promise to sell at that price). If the stock is below that price your profit is what you sold the call. Your maximum loss is infinite since a stock price has no (theoretical) limit.
      - sigstoat 4y ago
        > > doesn't betting against a stock have infinite loses? > Not with options. you maybe meant to say "not when you buy options": > 2. Sell a Call. This gives someone the right to buy the stock at a certain price (and you promise to sell at that price). If the stock is below that price your profit is what you sold the call. Your maximum loss is infinite since a stock price has no (theoretical) limit.
        4y ago
iso1631 4y ago
I can see the risk in short positions - you theoretically could lose an infinite amount (if you borrow X shares of something, sell them, but then can't find any to buy when it comes to returning them), and it's certainly possible to lose more than you put in (sell 100 short at $10 netting you $1000, price doubles overnight on new news, you have to buy $2000, losing a total of $1000 in the process. If price trippled overnight you'd lose $2000 -- more than you could have ever made even if the company went bankrupt overnight) But put options? Surely all you can do is lose what you bet in the first place?
- nopassrecover 4y ago
  In principle you have the same problem if you’re selling naked put options
  - NoboruWataya 4y ago
    I think you mean selling naked call options. If you're selling put options, you are selling someone else the right to force you to purchase shares from them at a given price. Which means that you are long the shares and also that your downside is limited by the strike price.
  - robocat 4y ago
    The price of a stock can only go to zero, so the value of a put option is bounded. A put option at exercise price of $2 has a maximum value of $2 (ignoring discount rate, conditions, etcetera). There are corner cases because a stock going to zero has other implications (e.g. cannot actually trade in that stock) which can affect delivery. Matt Levine puts it nicely: “Betting on disaster is hard because, if you win, there has been a disaster, and you might not get paid.” and he writes about (under CDS heading[2]) some of the other costly risks of shorting, some of which might also apply to options. The value of a call option is unbounded. This link[1] has some good questions and answers about options, although it certainly isn’t complete (e.g. dividends are mentioned but voting rights are not). [1] https://www.blackwellpublishing.com/content/kolb5thedition/chapter_12_solution.pdf https://www.blackwellpublishing.com/content/kolb5thedition/c... [2] https://news.bloombergtax.com/tax-insights-and-commentary/matt-levines-money-stuff-leaving-russia-for-morals-or-money https://news.bloombergtax.com/tax-insights-and-commentary/ma...
karxxm 4y ago
Are you also considering time lagged correlations? To me I don't care if a certain stock is correlated by something, I would more like to know which stocks do have correlations or if there are correlations with a time lag
- dhosek 4y ago
  I was thinking the same thing. To know that ABC follows DEF by x days with 95% accuracy would be a superpower in investing. Of course, the problem with something like this is that using it would, of course, ultimately change the market so that x→0.
  - betwixthewires 4y ago
    That's not necessarily true, there are positively reinforcing relationships and negative ones too. So, some strategies, everyone does them and the result becomes less predictable, others, everyone does them and the result becomes more predictable. A common modern example of a positive reinforcement is people pumping meme stocks. There are strategies that the more people know about it the stronger it becomes.
lbhdc 4y ago
Does this support all publicly listed tickers? It didn't seem to find the few I searched for.
joshuak 4y ago
Is it just me or do others see a correlation between people who short Tesla, and posting on HN about investment tools that use questionable prediction models based on historical "patterns"? These are starting to look like perpetual motion submissions to the patent office.
dom96 4y ago
This is a really great idea, unfortunately also getting 504s.
natly 4y ago
Random question: Would it be illegal to use insider information to trade on the result of a correlated stock that you suspect will be affected by the actual company you have information about? (I mean I get that it probably is since you're using insider information but wouldn't it be really hard to prove.) Just curious, I'm not intending on doing it myself.
- niklasbuschmann 4y ago
  Yes it is illegal, but happens nonetheless. Matt Levine wrote about this quite often, you can find it by searching „Money Stuff Shadow Trading“ for example
  - natly 4y ago
    Yup this seems to explain it pretty well. (Though he does talk about how my mentioned scenario is a bit of a grey area, and that barely anyone seems to have gotten in trouble for it.) If anyone else is curious: https://web.archive.org/web/20220129121919/https://www.bloomberg.com/opinion/articles/2022-01-26/watch-out-for-shadow-trading https://web.archive.org/web/20220129121919/https://www.bloom...
sk8terboi 4y ago
Betting against only wins with communism and socialism. And that's not really a win, is it?
chychiu 4y ago
Great site! I've been playing around with it and seeing some interesting patterns. One small typo: Highest -> Higest on the graphs
HFguy 4y ago
Couple of random thoughts: - Building things like this is always great. And its a fun site to poke around on. - I would not count on this approach or expect it to be reliable in terms of actually hedging. Correlation, as a measure, has lots of issues. You are boiling down a lot of complex relationships into a single number. While it is convenient for many calculations, there are many problems. For example, many asset classes will go through periods with positive correlation and then later, negative correlation. This is due to a factor driving both securities price becoming more or less volatile compared to the other drivers. E.g., recent increased volatility around inflation expectations driving correlations between rates and equities. Whereas, few years ago, inflation was not driving anything. - One alterative approach is to have a "risk model". Which essentially decomposes a security into drivers. Each security then represents a basket of these drivers. You can then use this model for range of purposes. While not perfect by any means, the model contains more information than a correlation. These too have a range of issues and creation and use is as much art as science. - In general, you won't find many negative (or even very low) correlations across individual equities. Most stocks are driven by a common set shared risk factors that drive much of the risk. But if you can find negatively correlated securities (or lowly correlated), then that is certainly helpful.
- deltree7 4y ago
  tl;dr -- All trading strategies work, until they don't. And most people blow up their account, because their trading strategies worked for a long time.
  - vasco 4y ago
    The amount of engineers that discover finance an the incredible, flawless, never-gonna-go-wrong-strategy of being short volatility and feel great about their results for a long time before blowing up their accounts never ceases to amaze me. Speaking for a friend, of course.
    - riku_iki 4y ago
      Did friend forget to put loss stops when issuing orders..
Aachen 4y ago
How does this submission have both text and a URL? When hitting submit it's shown as either/or. Does the server not enforce it?
- OJFord 4y ago
  Huh, I didn't realise that was possible either. To be fair though, it does say 'or', not 'xor'!
  - pushrax 4y ago
    In casual English “or” means “xor” most of the time.
    - OJFord 4y ago
      I'm not so sure about that.. it certainly can, and in a lot of contexts it would all or almost all of the time; in other contexts it doesn't.
- QuarterReptile 4y ago
  At one point a few months ago I tried to do the same in an 'Ask HN' post (not knowing how it was supposed to work), and the text portion of my submission was accepted as a comment on the post that got buried and led to lots of confusion.
- macco 4y ago
  It happened more by accident. I only used the field to compose the "first" comment, but then I hit submit before cutting the text.
ltbarcly3 4y ago
"any stock"? First two I tried are not available.
work_ta_220503 4y ago
Cool! Little offtopic highjacking: does it make sense to "bootstrap" correlation among stock returns (frankly, any multidimensional time series, but since we're talking about stocks) with different time periods? Say, for any pair of stock a and b, randomly selecting a startint point and a period (N days) and using this as a better estimator for the "true" correlation instead of using all the data points? Or something like this, not this process exactly
deleted 4y ago
[deleted]
nairboon 4y ago
Can you add Gamestop?
- bflesch 4y ago
  Would be really interesting to see for GME
  - benibela 4y ago
    i have heard It correlates with AMC
- elihu 4y ago
  It's there. (Maybe it wasn't earlier today, or the server was too bogged down?)
TradingPlaces 4y ago
HN Hug of Death continues. 503 and 504 all morning. Congrats?
mfrye0 4y ago
Seems like an interesting project, but so far the UX isn't great. The searching UI for companies has no loading or success/failure indicators. A note on performance - waiting an extra second before firing the search request can help take some of the load off the server, along with cancelling requests after I change the search. Some requests time out, and others return no results (e.g. HOOD, TLRY). For other requests that returned results, there's no correlation (Apple). Maybe an example page could be helpful to illustrate what the app can do?
- macco 4y ago
  Thanks for your feedback. I think the server was just overwhelmed. Normally, the search works instantly. Even though, I am located in Germany and the server is located in the US. I debounce the search for 250ms.
throwaway4good 4y ago
Where do you get your data from?
dmitrygr 4y ago
This tool does not correctly handle stock splits and thus all the correlations it produces are wrong for any stock that has ever had a split or a reverse split. It shows a split as a huge price drop, but that is not really how that works, since the number of shares expands, thus anyone holding the stock does not, in fact, face a 4x loss for a 1:4 split.
surfsvammel 4y ago
Correlation between assets change over time. Predicting what the correlation will be in the future is important for asset managers. I am supervising a master thesis project (in fact the second such project on the matter) where we are trying to predict the covariance matrix of a portfolio of assets using machine learning. Results are promising!
matt321 4y ago
I want LAGGING correlation please thanks.
loxias 4y ago
Great project! Even more so, congrats on shipping something! A little tip from someone who dabbles in algorithmic trading, look into cointegration as well as correlation. Also, the cross correlation matrix changes over time, you can have great fun seeing spikes and convergence/divergence as markets tend to get more or less correlated reacting to real life events.
- MuffinFlavored 4y ago
  any success algorithmically trading SPY?
  - loxias 4y ago
    Actually? Yeah. Though "kinda". I don't have a large enough sample size to determine how much is luck and how much isn't. And I'm too cheap about purchasing data. If I could give one condensed tip after a few years of experiments: "never buy a stock, sell a put. never sell a stock, sell a call". The residuals from that options premium, over time, were greater than anything else I made trying to be "more clever using math".
Def_Os 4y ago
Data source? Did you correct for splits and dividends? Why not use returns? Nice idea, but we’re left with so many questions.
- macco 4y ago
  Data source are dailys from alphavantage: https://www.alphavantage.co/documentation/#daily https://www.alphavantage.co/documentation/#daily Prices are not adjusted, as these are not free.
flying_sheep 4y ago
Well this tool does not cater for splits and dividends :-/ very misleading IMO
pmarreck 4y ago
HN hug of death. I didn't see any results. Even without shorting intention, this information would be interesting!
macilacilove 4y ago
Can someone explain why Microsoft has overall the strongest negative correlation with Activision/Blizzard(-0.87) on a 1 year basis. That must be related to the aquistion but I can not figure out how.
- GraemeMeyer 4y ago
  If the stocks fairly reliably went in opposite directions simultaneously I think that would cause it??
- sicromoft 4y ago
  This is standard for all acquisitions. - The acquiring company's share price drops because it often pays a premium for the target company, or incurs debt to finance the acquisition. - The target company's short-term share price tends to rise because the shareholders only agree to the deal if the purchase price exceeds their company's current value. See https://www.investopedia.com/ask/answers/203.asp https://www.investopedia.com/ask/answers/203.asp
- drinkzima 4y ago
  Maybe single day move on acquisition announcement - ATVI up quite a bit on 1/18, MSFT down that day. As an aside, the methodology says "We calculate the correlations between 2 securities on the daily closing values of the last 20 years. If one of the two securities has not been on the market for so long, we use all available prices to calculate the correlation." - hopefully the author means daily changes, not daily values, because otherwise everything is spurious.
  - drinkzima 4y ago
    More here: https://quantdare.com/correlation-prices-returns/ https://quantdare.com/correlation-prices-returns/
throwamon 4y ago
Cool. Not sure how useful it is considering what other commenters said, but is there anything like this for indices or currencies? Also, typo: Higest -> Highest
- macco 4y ago
  I also have some currencies in the database. You can suggest other ones.
jnwatson 4y ago
Searching “By industry” doesn’t make a lot of sense. Do you mean “within the same industry”? Mispelling “higest” in tab. A convenient tool. I’m restricted from trading in my employer’s stock. There are no rules about trading in a highly correlated proxy.
- macco 4y ago
  I changed your suggestions (and the typo). THX.
macco 4y ago
Wow, I was hoping for some feedback. I didn't expect to get on the HN homepage.
betwixthewires 4y ago
I'm interested in knowing how you do your correlation calculations, if there are any criteria for them (such as if a correlation is weighted against if the industries are unrelated). Also interested in your information feeds. I'm not looking to compete or do it for myself, this field just draws my playful interest and I'm curious.
sdcoffey 4y ago
This is pretty easy to do in SQL too! https://factor.fyi/questions/top-10-aapl-correlating-stocks-s7v8xk0nsu https://factor.fyi/questions/top-10-aapl-correlating-stocks-...
c_o_n_v_e_x 4y ago
Nice. I've been stewing on a similar idea but that finds correlations in equities with commodity futures.
vohu43 4y ago
Very interesting results for tesla for example. So can it make sense to buy a stock with very low correlations as "stabilisator" for example?
mountaineer 4y ago
FYI, the link in the copyright/footer is broken.

217 points by macco 4y ago

lupire 4y ago

Bet Against? I thought the opposite of buying a single stock (Alpha) would be in service of.... Beta Gains T.

noduerme 4y ago

And the answer is... Nvidia. Cool idea, by the way. I'll have to play with this more.

vortico 4y ago

Very cool, although index funds like S&P 500 and DJI don't seem to work. https://betagainst.fun/asset/_gspc_spy/ https://betagainst.fun/asset/_gspc_spy/ https://betagainst.fun/asset/_dji_dia/ https://betagainst.fun/asset/_dji_dia/

andrewmcwatters 4y ago
That doesn't make any sense anyway. You're asking what are the highest and lowest correlations of the market against the market itself. You cannot compare the same or roughly the same basket of assets against itself.
- scarmig 4y ago
  That still makes sense. Something can be more or less correlated with the broader market. It's very useful, actually: you can use securities that tend to not be correlated with the broader market to build a portfolio with less average variance.
  - andrewmcwatters 4y ago
    Yeah, I understand the concept, its just that regardless that people can graph it, doesn't mean that it widely has any tangible meaning other than being able to play with the numbers that come out of it. It's a concept that fits nicely in the category of technical analysis, but not one that allows you to find any particular market insights. If you took the whole tradable US securities market and said "what doesn't correlate with these price movements?" it doesn't mean anything. If it did, I would ask you, "What explicitly do you _think_ this means?" Just because you get numbers in and out doesn't mean you're working with anything meaningful. The only broadly meaningful concept you can get out of anti correlation with the market would be bonds because interest rates increasing push down the estimated future cash flows of publicly traded companies, affecting their valuations. Everything else can be considered speculation.
    - nickles 4y ago
      > It's a concept that fits nicely in the category of technical analysis, but not one that allows you to find any particular market insights. It's useful for hedging risk > The only broadly meaningful concept you can get out of anti correlation with the market would be bonds because interest rates increasing push down the estimated future cash flows of publicly traded companies, affecting their valuations. This is wrong. Consider pair trading or long/short strategies, both of which rely on estimating correlations.
    - JackFr 4y ago
      The correlation of an asset to the market is the definition of 'beta' and it is not technical analysis (from investopedia): "Beta is a measure used in fundamental analysis to determine the volatility of an asset or portfolio in relation to the overall market." "Technical analysis is a trading discipline employed to evaluate investments and identify trading opportunities in price trends and patterns seen on charts."
      - andrewmcwatters 4y ago
        I know what Investopedia says, but there's nothing fundamental about beta. Beta is the "measure of how an individual asset moves (on average) when the overall stock market increases or decreases." That's just looking at pricing. It's the same category as technical analysis.
- greatwave1 4y ago
  Yes, it makes sense. An asset's correlation are with the broader stock market is quantifiable. See also: https://www.investopedia.com/terms/b/beta.asp https://www.investopedia.com/terms/b/beta.asp
  - andrewmcwatters 4y ago
    No, it doesn't. The S&P 500 basically is the market. Any meaningful difference between it and the other US investable equities is marginally notable. It's literally asking "How do these roughly 500 plus or minus largest stocks correlate with the largely the same basket of securities?" If you took one company and compared it against the total US market, then you have something at least. If you take the market itself and compare it against itself, it makes no sense.
    - nickles 4y ago
      > If you took one company and compared it against the total US market, then you have something at least. This is a common metric referred to as market beta. > If you take the market itself and compare it against itself, it makes no sense. S&P 500 and Russell 2000 are both broad market indices, but they perform differently. Even using the same index constituents with different weightings (e.g. equal vs cap weighted) can produce meaningfully different results. It makes plenty of sense to compare them.
      - andrewmcwatters 4y ago
        I don't think you understand why multiple broad market indices exist in the first place. It has nothing to do with "performance" and everything to do with licensing fees when institutional organizations want to track against the market. No one. No one is seriously looking at the Russell 3000 and comparing it to the Wilshire 5000. To the child comment: I'm talking about two total market indices, apples to apples, and you're talking about apples to oranges. NASDAQ is a stock exchange, not an index. Further still, even if you were talking about the NASDAQ composite, they're two entirely different indices. They do not have the same goals. There might be some relative meaning there. There isn't much meaning between comparing two or more indices that have the same goals and constituents. You're only tracking the differences between constituents at that point, and you wouldn't need beta to do that. You could just calculate the difference between the constituents that are not a part of the total set.

daxaxelrod 4y ago

Getting 429 status codes after I hit search

LinuxBender 4y ago
And I get a blank page. Maybe because I block the trackers?

omarhaneef 4y ago

I wonder if correlation is the best measure. Cointegration might be more important for this sort of trading. And might be a supplementary measure you want to add since you’re doing all the pairwise comparisons. (If I recall there is a computational shortcut that helps)

deleted 4y ago

[deleted]

tdehnel 4y ago

I’ve often wondered the same thing, good on you for actually doing it. That said, in any set of 20+ variables there will be a 10 highest/10 lowest correlating. Without a good (specific, hard to vary) explanation as to why the correlation happens, I would not use this information to gamble real money.

onlyrealcuzzo 4y ago
It's the 10 highest and lowest SO FAR. If you track this over time, you will see that these change (often at times when you really want them not to). Past observances aren't super helpful in predicting fractal futures.
- somedude895 4y ago
  Would be interesting to see the differences in correlation from say 1980-2000 to 2000-2020. Of course that would exclude many of today's heavyweights.
hedora 4y ago
> Step 3: Check

Bostonian 4y ago

"We calculate the correlations between 2 securities on the daily closing values of the last 20 years." It's better to correlate daily returns than daily prices, since the latter are nonstationary, and I suggest using 1 year of daily returns rather than 20 since correlations do change over time. When I worked as a financial quant no one looked at 20 year correlations to measure near-term risk.

danuker 4y ago
For a retail trader, the fees are higher, affording fewer trades in a given period. As such the holding period might be on the order of 1 year for some people, so just 1 year of daily returns might invite too much trading if you're rebalancing.
- nurettin 4y ago
  Retail traders should be realizing gains within 2-3 month intervals. Retail investors will usually hold for years and do average pricing until the next market crash
  - Bostonian 4y ago
    "Retail traders should be realizing gains within 2-3 month intervals." Not in taxable accounts where short term capital gains are taxed at ordinary income tax rates (or at an even higher rate in Massachusetts). It's difficult to beat index funds on a pre-tax basis. On an after-tax basis it is almost impossible with discretionary stock picking.

cosmic_quanta 4y ago

Very cool! Where are you getting the historical data? There's a ton of fun stuff you can do with this kind of long-horizon of data

lesgobrandon 4y ago

runeks 4y ago

Super useful! However, I get a “504 Gateway Time-out” error for https://betagainst.fun/asset/bz__f_bno/ https://betagainst.fun/asset/bz__f_bno/. HN hug of death?

davidkuennen 4y ago
I suspect HN hug of death
- elihu 4y ago
  I think so. It seems to be running a lot smoother now than it was earlier today.
jedberg 4y ago
Do you have an Orbi or Netgear router? The service they use to block harmful websites has list this one as harmful.
- runeks 4y ago
  Nope; I have a TP-Link router.

LudwigNagasena 4y ago

I don't think the method is sound. You should look into finance literature on the topic, maybe something about diversification.

PaulHoule 4y ago
It's pretty notorious for not working. With few exceptions, all stocks are correlated to the broad market. https://en.wikipedia.org/wiki/Beta_(finance) https://en.wikipedia.org/wiki/Beta_(finance) One explanation that would be popular right now is that the price of stocks can be calculated based on expected future cash flows and interest rates (e.g. think of what you'd pay for a bond that produces the same csh flow.) Since the interest rate is a factor in that calculation for all stocks, there's a correlation right there. The exceptions tend to be sketchy. At one time it was thought that gold mining stocks would move against the market, but what I heard was that gold mining firms are badly run and if you like gold you should just buy gold.
- thom 4y ago
  People still do pair trading though, right?
  - PaulHoule 4y ago
    That's kinda the opposite of what that site is advocating since in pair trading you are going long one and short the other. https://en.wikipedia.org/wiki/Pairs_trade https://en.wikipedia.org/wiki/Pairs_trade The most famous pair trade was this: in the morning of a trading day you buy the stocks that performed the worst yesterday and short the ones that performed the best. Because the market reverts to the mean, this strategy made money hand over fist from the 1980s when Morgan Stanley started using it until the summer of 2007 when somebody fat fingered a button, unwinded their position during the day, and crashed everybody else who was in this trade. The profitability of that trade went downhill long before that crash but since then it's been impossible to make money doing it. (When there's a hedge fund bubble the profitability of a trade drops gradually as more people pile into it and the market inefficiency it targets is dispelled, quite different from an equity bubble which goes up dramatically until...)

snake_doc 4y ago

In many ways, compared to a vanilla short position or a synthetic short via derivatives, you are implicitly accepting higher risk to “short” via this manner. This is generally a bad idea as it’s hard enough to arbitrage the same equity on different exchanges[1]. Now imagine trying locate perfect negative substitutes for an equity… [1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=525282 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=525282

joelthelion 4y ago
I agree that this is very dubious. However, looking at the data sounds fun, if only I could get past the 50x errors :)
macco 4y ago
I highly doubt your assessment. When you have a short position, with underlying values you don't own, you have a leveraged investment. Which is inherently more risky than investments in a base value. The idea is not to have perfect negative correlations, but to find values with some negative correlation. This way you can profit from falling prices of an asset. Not with a 1 to 1 yield, or even higher.
- snake_doc 4y ago
  In equity markets, risk is measured in volatility or the standard deviation of return. By definition a short and long position of an equity has the same volatility or standard deviation of returns. You may be using the term risk colloquially, but your understanding of leverage under a short sale is also misguided. If you short a stock and have cash to cover, you don’t need leverage. If you short a stock and the position is in the black, you don’t need leverage. You only need leverage if your short position is in the net negative. Yes, you may have to pay borrowing costs for the stock, but that’s just normally known as transaction costs. Until you require leverage to keep the short position open when it’s net negative or you used the cash proceeds, you are not leveraged. If your goal is to manage the maximum loss from a short position, it’s much easier to short the stock and buy a protective put. This gives you perfect exposure to your bet, while allowing you to manage your loss exactly.

marcrosoft 4y ago

Nitpick: you can’t lose over 100% of your investment using a put option. You can only lose the premium.

akg_67 4y ago
You can lose up to 100% of premium (which makes up 100% of your investment in that option) if you BOUGHT the put option. You can lose up to Strike Price minus the premium received if you SOLD the put option.
- sakopov 4y ago
  I know next nothing about option trading, but doesn't betting against a stock have infinite loses?
  - plouffy 4y ago
    Not with options. If you want to bet against a company you with options you can do it by either: 1. Buy a put. The put gives you the right to sell the stock at a certain price. If it goes below that price you make a profit, if it doesn’t your put is worthless. The maximum you can lose is what you paid for the put. 2. Sell a Call. This gives someone the right to buy the stock at a certain price (and you promise to sell at that price). If the stock is below that price your profit is what you sold the call. Your maximum loss is infinite since a stock price has no (theoretical) limit.
    - sigstoat 4y ago
      > > doesn't betting against a stock have infinite loses? > Not with options. you maybe meant to say "not when you buy options": > 2. Sell a Call. This gives someone the right to buy the stock at a certain price (and you promise to sell at that price). If the stock is below that price your profit is what you sold the call. Your maximum loss is infinite since a stock price has no (theoretical) limit.
      - 4y ago

iso1631 4y ago

I can see the risk in short positions - you theoretically could lose an infinite amount (if you borrow X shares of something, sell them, but then can't find any to buy when it comes to returning them), and it's certainly possible to lose more than you put in (sell 100 short at $10 netting you $1000, price doubles overnight on new news, you have to buy $2000, losing a total of $1000 in the process. If price trippled overnight you'd lose $2000 -- more than you could have ever made even if the company went bankrupt overnight) But put options? Surely all you can do is lose what you bet in the first place?

nopassrecover 4y ago
In principle you have the same problem if you’re selling naked put options
- NoboruWataya 4y ago
  I think you mean selling naked call options. If you're selling put options, you are selling someone else the right to force you to purchase shares from them at a given price. Which means that you are long the shares and also that your downside is limited by the strike price.
- robocat 4y ago
  The price of a stock can only go to zero, so the value of a put option is bounded. A put option at exercise price of $2 has a maximum value of $2 (ignoring discount rate, conditions, etcetera). There are corner cases because a stock going to zero has other implications (e.g. cannot actually trade in that stock) which can affect delivery. Matt Levine puts it nicely: “Betting on disaster is hard because, if you win, there has been a disaster, and you might not get paid.” and he writes about (under CDS heading[2]) some of the other costly risks of shorting, some of which might also apply to options. The value of a call option is unbounded. This link[1] has some good questions and answers about options, although it certainly isn’t complete (e.g. dividends are mentioned but voting rights are not). [1] https://www.blackwellpublishing.com/content/kolb5thedition/chapter_12_solution.pdf https://www.blackwellpublishing.com/content/kolb5thedition/c... [2] https://news.bloombergtax.com/tax-insights-and-commentary/matt-levines-money-stuff-leaving-russia-for-morals-or-money https://news.bloombergtax.com/tax-insights-and-commentary/ma...

karxxm 4y ago

Are you also considering time lagged correlations? To me I don't care if a certain stock is correlated by something, I would more like to know which stocks do have correlations or if there are correlations with a time lag

dhosek 4y ago
I was thinking the same thing. To know that ABC follows DEF by x days with 95% accuracy would be a superpower in investing. Of course, the problem with something like this is that using it would, of course, ultimately change the market so that x→0.
- betwixthewires 4y ago
  That's not necessarily true, there are positively reinforcing relationships and negative ones too. So, some strategies, everyone does them and the result becomes less predictable, others, everyone does them and the result becomes more predictable. A common modern example of a positive reinforcement is people pumping meme stocks. There are strategies that the more people know about it the stronger it becomes.

lbhdc 4y ago

Does this support all publicly listed tickers? It didn't seem to find the few I searched for.

joshuak 4y ago

Is it just me or do others see a correlation between people who short Tesla, and posting on HN about investment tools that use questionable prediction models based on historical "patterns"? These are starting to look like perpetual motion submissions to the patent office.

dom96 4y ago

This is a really great idea, unfortunately also getting 504s.

natly 4y ago

Random question: Would it be illegal to use insider information to trade on the result of a correlated stock that you suspect will be affected by the actual company you have information about? (I mean I get that it probably is since you're using insider information but wouldn't it be really hard to prove.) Just curious, I'm not intending on doing it myself.

niklasbuschmann 4y ago
Yes it is illegal, but happens nonetheless. Matt Levine wrote about this quite often, you can find it by searching „Money Stuff Shadow Trading“ for example
- natly 4y ago
  Yup this seems to explain it pretty well. (Though he does talk about how my mentioned scenario is a bit of a grey area, and that barely anyone seems to have gotten in trouble for it.) If anyone else is curious: https://web.archive.org/web/20220129121919/https://www.bloomberg.com/opinion/articles/2022-01-26/watch-out-for-shadow-trading https://web.archive.org/web/20220129121919/https://www.bloom...

sk8terboi 4y ago

Betting against only wins with communism and socialism. And that's not really a win, is it?

chychiu 4y ago

Great site! I've been playing around with it and seeing some interesting patterns. One small typo: Highest -> Higest on the graphs

HFguy 4y ago

Couple of random thoughts: - Building things like this is always great. And its a fun site to poke around on. - I would not count on this approach or expect it to be reliable in terms of actually hedging. Correlation, as a measure, has lots of issues. You are boiling down a lot of complex relationships into a single number. While it is convenient for many calculations, there are many problems. For example, many asset classes will go through periods with positive correlation and then later, negative correlation. This is due to a factor driving both securities price becoming more or less volatile compared to the other drivers. E.g., recent increased volatility around inflation expectations driving correlations between rates and equities. Whereas, few years ago, inflation was not driving anything. - One alterative approach is to have a "risk model". Which essentially decomposes a security into drivers. Each security then represents a basket of these drivers. You can then use this model for range of purposes. While not perfect by any means, the model contains more information than a correlation. These too have a range of issues and creation and use is as much art as science. - In general, you won't find many negative (or even very low) correlations across individual equities. Most stocks are driven by a common set shared risk factors that drive much of the risk. But if you can find negatively correlated securities (or lowly correlated), then that is certainly helpful.

deltree7 4y ago
tl;dr -- All trading strategies work, until they don't. And most people blow up their account, because their trading strategies worked for a long time.
- vasco 4y ago
  The amount of engineers that discover finance an the incredible, flawless, never-gonna-go-wrong-strategy of being short volatility and feel great about their results for a long time before blowing up their accounts never ceases to amaze me. Speaking for a friend, of course.
  - riku_iki 4y ago
    Did friend forget to put loss stops when issuing orders..

Aachen 4y ago

How does this submission have both text and a URL? When hitting submit it's shown as either/or. Does the server not enforce it?

OJFord 4y ago
Huh, I didn't realise that was possible either. To be fair though, it does say 'or', not 'xor'!
- pushrax 4y ago
  In casual English “or” means “xor” most of the time.
  - OJFord 4y ago
    I'm not so sure about that.. it certainly can, and in a lot of contexts it would all or almost all of the time; in other contexts it doesn't.
QuarterReptile 4y ago
At one point a few months ago I tried to do the same in an 'Ask HN' post (not knowing how it was supposed to work), and the text portion of my submission was accepted as a comment on the post that got buried and led to lots of confusion.
macco 4y ago
It happened more by accident. I only used the field to compose the "first" comment, but then I hit submit before cutting the text.

ltbarcly3 4y ago

"any stock"? First two I tried are not available.

work_ta_220503 4y ago

Cool! Little offtopic highjacking: does it make sense to "bootstrap" correlation among stock returns (frankly, any multidimensional time series, but since we're talking about stocks) with different time periods? Say, for any pair of stock a and b, randomly selecting a startint point and a period (N days) and using this as a better estimator for the "true" correlation instead of using all the data points? Or something like this, not this process exactly

deleted 4y ago

[deleted]

nairboon 4y ago

Can you add Gamestop?

bflesch 4y ago
Would be really interesting to see for GME
- benibela 4y ago
  i have heard It correlates with AMC
elihu 4y ago
It's there. (Maybe it wasn't earlier today, or the server was too bogged down?)

TradingPlaces 4y ago

HN Hug of Death continues. 503 and 504 all morning. Congrats?

mfrye0 4y ago

Seems like an interesting project, but so far the UX isn't great. The searching UI for companies has no loading or success/failure indicators. A note on performance - waiting an extra second before firing the search request can help take some of the load off the server, along with cancelling requests after I change the search. Some requests time out, and others return no results (e.g. HOOD, TLRY). For other requests that returned results, there's no correlation (Apple). Maybe an example page could be helpful to illustrate what the app can do?

macco 4y ago
Thanks for your feedback. I think the server was just overwhelmed. Normally, the search works instantly. Even though, I am located in Germany and the server is located in the US. I debounce the search for 250ms.

throwaway4good 4y ago

Where do you get your data from?

dmitrygr 4y ago

This tool does not correctly handle stock splits and thus all the correlations it produces are wrong for any stock that has ever had a split or a reverse split. It shows a split as a huge price drop, but that is not really how that works, since the number of shares expands, thus anyone holding the stock does not, in fact, face a 4x loss for a 1:4 split.

surfsvammel 4y ago

Correlation between assets change over time. Predicting what the correlation will be in the future is important for asset managers. I am supervising a master thesis project (in fact the second such project on the matter) where we are trying to predict the covariance matrix of a portfolio of assets using machine learning. Results are promising!

matt321 4y ago

I want LAGGING correlation please thanks.

loxias 4y ago

Great project! Even more so, congrats on shipping something! A little tip from someone who dabbles in algorithmic trading, look into cointegration as well as correlation. Also, the cross correlation matrix changes over time, you can have great fun seeing spikes and convergence/divergence as markets tend to get more or less correlated reacting to real life events.

MuffinFlavored 4y ago
any success algorithmically trading SPY?
- loxias 4y ago
  Actually? Yeah. Though "kinda". I don't have a large enough sample size to determine how much is luck and how much isn't. And I'm too cheap about purchasing data. If I could give one condensed tip after a few years of experiments: "never buy a stock, sell a put. never sell a stock, sell a call". The residuals from that options premium, over time, were greater than anything else I made trying to be "more clever using math".

Def_Os 4y ago

Data source? Did you correct for splits and dividends? Why not use returns? Nice idea, but we’re left with so many questions.

macco 4y ago
Data source are dailys from alphavantage: https://www.alphavantage.co/documentation/#daily https://www.alphavantage.co/documentation/#daily Prices are not adjusted, as these are not free.

flying_sheep 4y ago

Well this tool does not cater for splits and dividends :-/ very misleading IMO

pmarreck 4y ago

HN hug of death. I didn't see any results. Even without shorting intention, this information would be interesting!

macilacilove 4y ago

Can someone explain why Microsoft has overall the strongest negative correlation with Activision/Blizzard(-0.87) on a 1 year basis. That must be related to the aquistion but I can not figure out how.

GraemeMeyer 4y ago
If the stocks fairly reliably went in opposite directions simultaneously I think that would cause it??
sicromoft 4y ago
This is standard for all acquisitions. - The acquiring company's share price drops because it often pays a premium for the target company, or incurs debt to finance the acquisition. - The target company's short-term share price tends to rise because the shareholders only agree to the deal if the purchase price exceeds their company's current value. See https://www.investopedia.com/ask/answers/203.asp https://www.investopedia.com/ask/answers/203.asp
drinkzima 4y ago
Maybe single day move on acquisition announcement - ATVI up quite a bit on 1/18, MSFT down that day. As an aside, the methodology says "We calculate the correlations between 2 securities on the daily closing values of the last 20 years. If one of the two securities has not been on the market for so long, we use all available prices to calculate the correlation." - hopefully the author means daily changes, not daily values, because otherwise everything is spurious.
- drinkzima 4y ago
  More here: https://quantdare.com/correlation-prices-returns/ https://quantdare.com/correlation-prices-returns/

throwamon 4y ago

Cool. Not sure how useful it is considering what other commenters said, but is there anything like this for indices or currencies? Also, typo: Higest -> Highest

macco 4y ago
I also have some currencies in the database. You can suggest other ones.

jnwatson 4y ago

Searching “By industry” doesn’t make a lot of sense. Do you mean “within the same industry”? Mispelling “higest” in tab. A convenient tool. I’m restricted from trading in my employer’s stock. There are no rules about trading in a highly correlated proxy.

macco 4y ago
I changed your suggestions (and the typo). THX.

macco 4y ago

Wow, I was hoping for some feedback. I didn't expect to get on the HN homepage.

betwixthewires 4y ago

I'm interested in knowing how you do your correlation calculations, if there are any criteria for them (such as if a correlation is weighted against if the industries are unrelated). Also interested in your information feeds. I'm not looking to compete or do it for myself, this field just draws my playful interest and I'm curious.

sdcoffey 4y ago

This is pretty easy to do in SQL too! https://factor.fyi/questions/top-10-aapl-correlating-stocks-s7v8xk0nsu https://factor.fyi/questions/top-10-aapl-correlating-stocks-...

c_o_n_v_e_x 4y ago

Nice. I've been stewing on a similar idea but that finds correlations in equities with commodity futures.

vohu43 4y ago

Very interesting results for tesla for example. So can it make sense to buy a stock with very low correlations as "stabilisator" for example?

mountaineer 4y ago