12 ms·

Hedge funds use satellite images to beat Wall Street

278 points by jonbaer 7y ago

bluGill 7y ago
Nothing new here other than satellites. Years ago (1990s?) Peter Lynch - at the time manager of the largest mutual fund followed his daughters into a shopping mall for back to school - his daughters didn't go into GAP stores so he went back to the office and sold as his shared of GAP. His fund beat everyone else by a large margin because GAP stocks dropped a month latter when they announced the earnings drop. This is better of course, but nothing you can't do in some other way with a little work.
- 65934 7y ago
  Where did you get this anecdote from? Googled it, couldn't find anything
  - lionsdan 7y ago
    Similar story from 1993 "Some of Mr. Lynch's best picks came from watching his three teen-age daughters shop at a nearby mall,..." https://www.baltimoresun.com/news/bs-xpm-1993-03-09-1993068075-story.html https://www.baltimoresun.com/news/bs-xpm-1993-03-09-19930680...
  - deleted 7y ago
    [deleted]
  - frakkingcylons 7y ago
    His book, One Up On Wall Street, references this idea of using casual observations to find undervalued stocks. You might find that interesting to look through.
  - bluGill 7y ago
    His Book "One up on Wall Street". (which others have mentioned.)
- blihp 7y ago
  Was going to make the same comment. I remember Peter talking about how he used foot traffic at the mall to size up retailers he was looking at back in the early 90's. These were interviews on channels like CNBC, so pretty widely watched by retail investors. All the satellites do is automate the work and I'm pretty sure hedge funds have been using satellite pics for well over a decade now. (I recall reading about the parking lot use case pre-2010) So really all that's changed is that they're now using ML to automate most of the work that analysts used to do manually.
  - pigscantfly 7y ago
    They've been using ML to automate most of that work for at least five years.
- Diederich 7y ago
  > This is better of course, but nothing you can't do in some other way with a little work. I worked at one of the companies doing machine learning on top of large quantities of satellite imagery and selling the results. It's more than a little work; it's pretty challenging, in a variety of ways. Just the training aspect is surprisingly difficult to get right.
- anthony_barker 7y ago
  Not even the satellites is new. Bloomberg has has oil tanker tracking for a decade as well as oil field sat images.
ISL 7y ago
As an occasional retail trader, it doesn't bother me at all that other market participants are trying to be better informed. In fact, it means that market prices are more likely to be properly-priced. We should not aspire to drag hedge-fund research down to the Main Street level. If there is a place for government in this context, it is to fund the education necessary to help retail investors learn a) how to take beneficial approaches to investing and risk, b) to appreciate the realities of trading against professional investors, and c) to understand how to conceive, test, and implement their own hypotheses in the real world. In general, an informed market is a healthy one.
- logifail 7y ago
  > As an occasional retail trader, it doesn't bother me at all that other market participants are trying to be better informed. In fact, it means that market prices are more likely to be properly-priced. What about companies that find out about your retail trade before it's even executed, and are able to act on that information to profit from your very intent to trade?
  - tryptophan 7y ago
    This is called front running and is super illegal.
    - tanderson92 7y ago
      Not all front-running is illegal, see this article which discusses a few examples, including quote matching and order anticipation: https://blogs.cfainstitute.org/investor/2013/04/24/what-to-do-about-high-frequency-trading/ https://blogs.cfainstitute.org/investor/2013/04/24/what-to-d... Front-running is very common in bond trading.
      - tryptophan 7y ago
        Quote from your article: >HFT greatly reduces average trading costs for investors They are market-making, which is something else entirely than front-running.
dageshi 7y ago
The tone of the article seems like it has an axe to grind. Is it really that much of a surprise that very well capitalised investors have significant advantages of "main street" investors? That's always been the argument for index funds for regular investors, you're never going to compete with hedge funds, why waste your time?
nwah1 7y ago
I remember seeing stories about programmers with access to credit card databases getting indicted for trading on the information. This is a legal version of the same thing, and also probably any attempt to make it illegal would be unenforceable.
- dillonmckay 7y ago
  ‘Outsider information’?
  - nwah1 7y ago
    Precisely. It's unfair for the little guys, but lot's of stuff is unfair... such as most of high speed trading, and the people who use arbitrage to skim from the index funds that use predictable purchasing.
    - eeZah7Ux 7y ago
      ...which doesn't make it any more acceptable.
- piker 7y ago
  One distinction is whether or not there's a duty of confidentiality owed by the possessor of the information with respect to the source of the information (the "non-public" element of "material non-public information"). It doesn't seem here there would be any duty of confidentiality. Presumably the programmers in your example owed a duty of confidentiality to their employer or contract counterparty.
- criley2 7y ago
  I mean, you don't necessarily have to make it illegal. 1. It's from hedge funds, and you can invest in a hedge fund today and reap the benefits of their strategies yourself 2. There exists a gap right now, where a startup or data analysis firm could provide this kind of information to smaller investors through some kind of subscription model. 3. Ultimately the government itself could "level the playing field" by either gathering this information through other channels and releasing it (BLS or SEC style) or by replicating the strategies and releasing it. I think #3 could be viable. We already require businesses to share very important and actionable info with the government, and since everyone does it, the risks of one firm having data leaked and another not are mitigated. (SEC requires public reports, BLS takes private data and aggregates it for higher level views) If the government required businesses to report data to them on a weekly or monthly basis regarding things that are being satellite tracked, they could release this information on a set schedule to basically demolish the ability of institutional investors to gain secret information to bet against the public positions
citilife 7y ago
I use discussions on the internet... https://projectpiglet.com/ https://projectpiglet.com/ If it's "public" it's fair game for the stock market. My favorite example was the company Quandl[1] which was monitoring charted flight patterns of companies to determine if a company in the area was going to be purchased. [1] https://blog.quandl.com/corporate-aviation-intelligence/aircraft https://blog.quandl.com/corporate-aviation-intelligence/airc...
- deleted 7y ago
  [deleted]
- dna_polymerase 7y ago
  Of course you do. You developed that service. Maybe you should mention that in such posts...
rococode 7y ago
I went to a recruiting talk by Two Sigma a while back where they discussed the same thing as an example of "look at this cool stuff we get to work on". They had slides with satellite images of some parking lot and showed how they could segment cars, deal with car-like things on roofs, map regions of parking lots to the right stores, handle different conditions like snow, etc. From a technical perspective it's actually an interesting problem because there are so many things to consider. Really makes you notice how disadvantaged retail traders are, though. I think traditionally a lot of hedge fund strategies are kept pretty secret, but this one is so out of reach and technically difficult for most people that they don't really care if it gets out.
- mc32 7y ago
  How do they deal with underground parking in urban areas especially?
  - mbreese 7y ago
    You wouldn’t. But just because a prediction can’t be 100% accurate because of missing data doesn’t mean that the prediction isn’t useful. Really, what they are likely comparing is the relative changes in parking lot occupancy over time. So it’s not necessary to have complete coverage of all parking lots. You just need enough coverage to make the prediction with a certain level of confidence.
  - bluGill 7y ago
    They don't need to. People who park underground, walk or use public transit are close enough to people who park above ground that they get enough data without that. In most cases just one WalMart is enough to infer everything you need to know about them all - if you select the right one for your question. (you wouldn't select just one because then you need to ask if there is something specific about that one, but a statistical sample is enough). Unless your thesis is people who park underground shop differently. That is a valid idea and you cannot answer it via this study. However you can study it by other means if you believe this is important.
  - floatrock 7y ago
peteretep 7y ago
How is it that Berkeley can't run https on their webservers?!
- secabeen 7y ago
  That site is hosted off-campus, coordinating obtaining the cert from the campus CA and then getting it into their off-campus host is probably just a low-priority. They should still do it, though, being off HTTPS is going to hurt their SEO soon enough.
harryh 7y ago
This article is very wrong to confuse "material non-public information" with hard to acquire information like counting cars in satellite photos. Remember the Matt Levine test: insider trading is about theft, not fairness. When a company insider uses private company information to trade (or colludes with an outside party to do so) they are stealing material non-public information from the company for their own benefit. Nothing of the kind is happening when a hedge fund (or anyone else) acquires and analyzes satellite photos. There is no expectation of fairness in the stock market. All sorts of people work very hard and spend a lot of money to develop proprietary information in the pursuit of stronger returns. There is nothing at all wrong wit this, especially because it means that in the end we all get more accurately priced securities.
- vostok 7y ago
  This is exactly right. If you're upset about this then you should be upset that hedge funds can afford to hire smarter analysts than you, which is just silly.
  - davidw 7y ago
    I agree, but where's this end up? Why would retail investors play a game where they're almost certain to lose?
    - piker 7y ago
      That's the why this isn't a perfect quote. Some proponents of insider trading laws pose that exact question and pursue insider trading violations in the interest of protecting the "fairness" of the markets.
    - mrosett 7y ago
      They shouldn’t. Pick a Vanfuard fund, put your money in it, and check back in a couple decades.
      - davidw 7y ago
erickerr 7y ago
https://secondmeasure.com https://secondmeasure.com is like "counting cars in parking lots" on steroids.
waterside81 7y ago
There's a whole marketplace where companies with so-called "Alternative-data" sell their datasets to quant traders. Satellite imagery of farms, passenger traffic through airports, all sorts of random, seemingly disparate data.
- floatrock 7y ago
  The reason your free weather app asks for your location data isn't for "local weather forecasts". Well, it is, but that's a cost for them. The income comes from keeping that location tracking on all the time and selling which stores and dentists you go to. Someone can link the NYT story from a few months ago when this came to mainstream attention. Hedge funds also fly cesnas over refinery fields to look at the levels of those big tanks you see off the highway... most of them have roofs that move up or down with capacity, and those that don't you look at with an infrared camera to look at relative temperature differences.
  - goodfight 7y ago
    If anyone is profiting off my location I better get a cut of it haha. It's amazing what they can do and I enjoy data-centric trading strategies but we don't have to violate individual privacy.
  - knd775 7y ago
    Some employees of a credit card company got in trouble for this sort of thing a while ago. They were running queries against customer transactions to figure out sales ahead of earnings calls. For example, they figured out that Chipotle was going to beat expectations and bought a ton of stock just before the earnings call. (This very predictably got the attention of the SEC)
    - Thrymr 7y ago
      Previous discussion of that event on HN: https://news.ycombinator.com/item?id=8966817 https://news.ycombinator.com/item?id=8966817
  - 7y ago
lolc 7y ago
I find it somewhat interesting to what lengths traders go for profits. Cue people who explain how this makes the markets more efficient. Something I'm wondering about though: The article makes it sound like this is some new unfair advantage of the big traders. But isn't this an old game? If you can afford to acquire and process the data you're likely going to beat the other participants. That was always true. I don't see what's qualitatively different here. Is there?
- roywiggins 7y ago
  It increases the information that makes its way into the markets and makes it more likely that the price reflects the what's happening in the real world. A world in which nobody bothered to check whether a company was actually producing anything would turn the stock market into a collective guessing game.
  - sjg007 7y ago
    >A world in which nobody bothered to check whether a company was actually producing anything would turn the stock market into a collective guessing game. This is kind of a thing in China.
  - ryandrake 7y ago
    The stock market is a collective guessing game. These stories suffer from survivorship bias. For every investor A you hear about who did X and was successful, there are investors B, C, and D who also did X and were not successful, and don’t get articles written about them. There are also investors E, F, and G who don’t do X and may be successful or unsuccessful. Finding someone successful at stock picking and asking him what he did to be successful is like finding that 1 out of 1024 person who flipped a coin heads 10 times in a row and asking him what makes him such a great coin flipper.
    - roywiggins 7y ago
      By that argument, there'd be no successful insider traders, but clearly you can beat make money if you're an insider. You know that the company is (doing well/doing bad) before everyone else does, you trade on that, you make money from the people who are in the dark. The hard part is not regressing to the mean, and consistently beating the market, which sure, nobody can do. But if I learn today that a stock is going to rise tomorrow, and I buy as much as I can, the stock will rise a bit today. So the market has become a little bit more efficient. That I happen to then base all my subsequent trades on overconfidence and tossing darts doesn't retroactively invalidate that trade. You can even make a market more efficient on losing trades- you short a stock, it goes down a bit, but you spend more maintaining the short than the stock goes down. Then, the stock crashes after your short expires. Oops! But your prediction was still partly right, and that information was integrated into the price a bit earlier than it would have been without you.
client4 7y ago
This article reminds me of tracking corporate jets and seeing if they were doing tarmac M&A talks. There are a lot of interesting opportunities for 'corporate information side-channel' attacks and it would be a great opportunity for a new information based corporation to corral this information and sell it.
yial 7y ago
I remember reading years ago about similar data being used to monitor and track the recession (in the US) and potential recovery from. Especially as it related to big-box retailers.
xtacy 7y ago
Reminds me of this article from Thasos Group that analysed smart phone signals at a Tesla factory to infer overtime work: http://thasosgroup.com/blog/thasos-data-tesla-wsj/ http://thasosgroup.com/blog/thasos-data-tesla-wsj/
paradox1234 7y ago
Honestly, I'm not particularly impressed. There are a lot of other trading edges that could be found for a lot less hassle and money, but I guess that's also the beautiful thing about the market - more ways to skin a cat than anyone can imagine, and it's also not a zero sum game.
- kami8845 7y ago
  > There are a lot of other trading edges that could be found for a lot less hassle What are you thinking about?
  - TomMarius 7y ago
    :^)
- eeZah7Ux 7y ago
  > also not a zero sum game Indeed, it's negative. While the big funds steal money from small investors the waste of energy and the pollution caused by launching satellites remains.
sj3k 7y ago
A company in my town, https://www.descarteslabs.com/ https://www.descarteslabs.com/ Is becoming a popular choice for Hedge Funds seeking this kind of data.
jandrewrogers 7y ago
The same thing has been done with mobile data for many years, with higher fidelity. Unlike with satellite imagery, the measurement is almost continuous, you can accurately characterize which subpopulations are changing behavior, and you can sometimes characterize the change in behavior e.g. shopping more/less, spending more time at different retailers/competitors, etc. Remote sensing like satellite is excellent for other similar types financial modeling applications, but I wouldn't use it for modeling retail as it is much too coarse.
opportune 7y ago
I almost worked for a satellite imaging company that does this stuff. It kind of bothered me that rather than providing farmers or the government with this extremely rich agricultural data, they were just contracted out to specific hedge funds that would use the data to give them an edge in trading. I feel like that's not the best use of that kind of data to society
- driverdan 7y ago
  If the farmers or governments were willing to pay the same as hedge funds I'm sure they would sell it to them too.
Kapura 7y ago
I think the most important part is this quote: >“What we found is that it’s a gain for large sophisticated investors who can afford the substantial costs of acquiring and processing big alternative data” This is yet another example of wealth accumulating, of rich actors getting richer faster than poor ones. Without some controls on the behaviours that allow wealth to accumulate, it will further concentrate among the wealthy. I would argue that is a bad outcome.
- airstrike 7y ago
  Just invest in a Vanguard fund and you'll be fine. You don't need to start your own fund, just like you don't need to be your own doctor.
- noir_lord 7y ago
  Shrugs. Any game where you get more of an advantage the more points you accumulate from a fixed pool of points is going to rapidly result in an unequal share of the points going to the first person to acquire the advantage. Monopoly was created as a game to warn of the pitfalls of capitalism (it's a simplified form of the winner takes if not all a disproportionate share scenario that is life)[1] http://www.bbc.com/capital/story/20170728-monopoly-was-invented-to-demonstrate-the-evils-of-capitalism http://www.bbc.com/capital/story/20170728-monopoly-was-inven... [1] Of course that I think this puts me at odds with many people, I like capitalism, I like the social pluses of it but I believe that capitalism should be sub-ordinate to society directly (more what you see in the north of europe) and less the other way around (pretty much most other places and arguably the US).
notlukesky 7y ago
Nothing unfair at all. Savvy investors have used many tools in the past as well. One analyst at Tiger Fund had got a job at Macy’s in NYC in the cosmetics department before making a decision on whether to have a position on L’Oreal. Unless they have time to do their homework etc.. main street investors should stick to index funds or bonds.
jetti 7y ago
I was listening to a podcast on trading where there was a quant trader as a guest. He mentioned that quant firms have been doing this for years. The guest mentioned that this is no longer an advantage in quant trading as it is common knowledge and can be done by other firms.
sib 7y ago
Something that is visible from space - the number of cars in a parking lot - seems to be the very antithesis of "non-public" information. I think Berkeley missed the thread on this one.
- whymsicalburito 7y ago
  Yes, but it's not like the cars are ONLY visible from space. You are welcome to organize a large group of people to travel around to parking lots and count cars and then privately share your findings. What the funds are doing is just a more efficient version of that. It's all public information.
  - smountjoy 7y ago
    OP agrees with you: antithesis of non-public = public.
  - harimau777 7y ago
    If the information is only shared privately, then it seems to me that it is by definition not public information; regardless of what the law says.
    - ironSkillet 7y ago
      What is your definition of "public information"? It's trickier than you may think to define it precisely. For example, is any processed version of "public information" also public information?
  - sib 7y ago
    Completely agree. My point is only that there's no real argument here that this is "non-public information," which is what insider trading laws are about.
- keiru 7y ago
  So snooping on your neighbour gets more morally sound the higher your ladder is?
joshu 7y ago
i was pitched a dataset of this almost 20 years ago. not new.
maitredusoi 7y ago
A soooooo good idea, I wouldn't have thought about it ...
cgy1 7y ago
I think this was a storyline in the TV show Billions.
- noir_lord 7y ago
  It was, something to do with chip fabrication iirc. This is definitely a case of art imitating life though.
ForHackernews 7y ago
Do hedge funds really consider themselves distinct from 'Wall Street"? I know they aren't always located in lower Manhattan, but as a metonym, Wall Street would commonly be taken to include all of high finance.
module0000 7y ago
Not to duplicate other opinions, but I'll add my similar one.. This is GOOD for the market(the practice, not the article). Price manipulation is a real thing, and fact-based research such as this means prices are more likely to reflect reality. This type of analysis is what we need more of! It's not opinion-based drivel from a talk show or blog, it's empirical data, analyzed impartially. edit: typo
rmah 7y ago
Um... hedge funds are part of "wall street". Silly clickbait headline... sigh
lil-scamp 7y ago
This example of counting cars has been around the geo industry forever, and satellite imagery is pretty cost prohibitive to get on a regular basis. I'd be surprised if hedge funds haven't already moved on to purchasing location-based data (ie mobile phone data) to get even more accurate counts of people in box stores.
- 50656E6973 7y ago
  That seems to be the case: https://news.ycombinator.com/item?id=17081684 https://news.ycombinator.com/item?id=17081684
  - lil-scamp 7y ago
    Wonder what the pricing in on that location data is for financial sector customers, and if it's cheaper than a regular purchase of satellite data. It'd be interesting to compare the mobile location data to the satellite data and find the deltas— that would make it a lot easier to build efficient models w/ existing satellite if it's cheaper.
ris 7y ago
So of course the next big business idea is large unrollable printed vinyl top-views of cars. Get your employees to move them around a bit every morning, company's stocks start to rise. Anyone who ever considers this sort of thing to be "unfair" or "cheating" should also consider how comparatively easy it is to make a nonsense of. Who wants to go halfs with me on a large-format printer?
anonu 7y ago
They've been using satellites for over a decade now. The fact that you're hearing about it means there's not much value in this type of data anymore.... I reserve some room for being wrong here. This type of data is ultimately used to predict revenue for consumer companies. Cellphone data and wifi data is much better for that type of prédiction.
- goodfight 7y ago
  The information about the state of hedge funds takes time to travel, anyone that has been in the quant field knows about these practices. Now that data science and machine learning are becoming more accessible to the public, I'm not surprised that articles are being made on it. There is value in this data now that we have better data processing abilities. There was less technical value in the past and now there is less competitive value in the present.
hnburnsy 7y ago
This was a better story from 2016 on how they are deploying the satellites... "A company called Planet Labs Inc. has launched a small constellation of what it calls “cubesats” that can deliver much more frequent imagery of economically sensitive spots than traditional satellites. Those spots include retailers’ parking lots, oil-storage tanks or farmland" https://www.wsj.com/articles/satellites-hedge-funds-eye-in-the-sky-1471207062 https://www.wsj.com/articles/satellites-hedge-funds-eye-in-t...
- eeZah7Ux 7y ago
  sigh
totaldude87 7y ago
I think i saw the same thing in the showtime series "Billions"? where Taylor uses this information , thought this is another level of technical analysis :) Moral ethics aside, this looks legal?!, even if the shorts tank shares based on images of car parking lots..
- alphagrep12345 7y ago
  In Billions, Bobby shrugs off saying - everyone does that, tell me something new. So I thought this is already pretty popular.
- tryptophan 7y ago
  Watching Billions to learn about what hedgefunds do is like watching Silicon Valley to learn how to make a startup.
  - alecco 7y ago
    But it's a good way to learn old cliches in both cases.
pmarreck 7y ago
This was apparently a plot point in Billions. There's market opportunity here. Off the top of my head: 1) Drone photos instead of satellite photos to get at the data cheaper 2) ML counting of cars 3) Some way to legally sense the number of cellphones within a given radius (can possibly be combined with a drone) 4) Simply aiming a network cam at the entrance/exit from across the street and counting the number of people that enter and exit via ML All of these could get you at public data that highly correlates with retail sales
- deepnotderp 7y ago
  So all of that already exists and is already widely used. I'm a bit confused as to why this is news considering that this has been going on for quite some time.
  - pmarreck 7y ago
    Maybe because I'm a pure-software back-end guy who has no idea what the hell institutional investors do in the hardware space to make their targets, because that's not remotely my wheelhouse? ¯\_(ツ)_/¯
- jrochkind1 7y ago
  Why would drones be cheaper than satellite photos? If they were, wouldn't the hedge funds already be doing that instead? They clearly seem not to me. The issue, if there is one, is that it is not cheap to get this data on a large enough scale to be useful. I suppose it's possible a sampling of just a few parking lots would be representative enough to be useful? In which case you don't need drones, you can just go to the parking lots. Which is apparently what Sam Walton did? I dunno. If that were good enough, again I'm not sure why the hedge funds would be paying for sattelite photos instead.
  - csteubs 7y ago
    You're correct that drones are more expensive than satellite imagery at scale--I'm currently working on a startup in this space that cuts the cost of imaging by between 100x and 10,000x depending on the target location.
alphagrep12345 7y ago
There's also a YC company, (https://secondmeasure.com/ https://secondmeasure.com/), that sells anonymized credit card data to hedgefunds so that the funds can directly check the volume of trade happened at walmart, etc.
est31 7y ago
Hedge funds do a ton of crazy research like this. I remember reading about hedge funds buying new cars only to sell them immediately after. They were only interested in their serial numbers which they used to estimate the count of sold cars of a specific model using the german tanks method. Before the sales numbers are out, information like this is very valuable.
- gist 7y ago
  > about hedge funds buying new cars only to sell them immediately after To get this data you wouldn't need to buy the cars. So it sounds like they did something that sounded like that as opposed to what you remember. Even w/o social engineering you could quite easily get some dealer to give you that type of information (or someone working there or otherwise).
  - sokoloff 7y ago
    You could walk the lot and look at the corner of the windshields...
- sokoloff 7y ago
  We sell customized products. A little over a decade ago, we noticed a few orders going out that were not at all customized or were customized in a way that suggested the customer wouldn't be happy when they got them. We tracked down the account (based in NYC) and they'd been ordering products about every 45 days (mid quarter and end of quarter) and they'd never complained nor asked for a refund. Seems they were after the shipping identifier, as when we changed the shipping identifier to not leak shipment volume information, the orders soon stopped.
Bakary 7y ago
"I [suspect] that we are throwing more and more of our resources, including the cream of our youth, into financial activities remote from the production of goods and services, into activities that generate high private rewards disproportionate to their social productivity. I suspect that the immense power of the computer is being harnessed to this 'paper economy', not to do the same transactions more economically but to balloon the quantity and variety of financial exchanges." --James Tobin, July 1984
fyz 7y ago
Some of the comments conflate retail traders with regular Joes and Janes. A lot of hedge fund investor money comes from pension funds who are placing retirees' money either directly or indirectly into hedge funds. HF compensation is different can of worms.
aiyodev 7y ago
Retailers should try painting images of cars in their parking lots.
- kennyloginNTH 7y ago
  They could splurge for some covered multi-level parking. That would be nice.
238sjni9 7y ago
what surprises me is how sat images are more cost effective than digital dragnets.
tamaharbor 7y ago
Can someone with access to very recent satellite photography tell me how much corn they got planted in the Midwest this year (compared to last)? Thank you.
socialist_coder 7y ago
Theoretically you could fake a bunch of cars moving around in the parking lots of a failing store. Wait for the hedge funds to bite and move the stock price up... then boom, dump your shares.

278 points by jonbaer 7y ago

Nothing new here other than satellites. Years ago (1990s?) Peter Lynch - at the time manager of the largest mutual fund followed his daughters into a shopping mall for back to school - his daughters didn't go into GAP stores so he went back to the office and sold as his shared of GAP. His fund beat everyone else by a large margin because GAP stocks dropped a month latter when they announced the earnings drop. This is better of course, but nothing you can't do in some other way with a little work.

65934 7y ago
Where did you get this anecdote from? Googled it, couldn't find anything
- lionsdan 7y ago
  Similar story from 1993 "Some of Mr. Lynch's best picks came from watching his three teen-age daughters shop at a nearby mall,..." https://www.baltimoresun.com/news/bs-xpm-1993-03-09-1993068075-story.html https://www.baltimoresun.com/news/bs-xpm-1993-03-09-19930680...
- deleted 7y ago
  [deleted]
- frakkingcylons 7y ago
  His book, One Up On Wall Street, references this idea of using casual observations to find undervalued stocks. You might find that interesting to look through.
- bluGill 7y ago
  His Book "One up on Wall Street". (which others have mentioned.)
blihp 7y ago
Was going to make the same comment. I remember Peter talking about how he used foot traffic at the mall to size up retailers he was looking at back in the early 90's. These were interviews on channels like CNBC, so pretty widely watched by retail investors. All the satellites do is automate the work and I'm pretty sure hedge funds have been using satellite pics for well over a decade now. (I recall reading about the parking lot use case pre-2010) So really all that's changed is that they're now using ML to automate most of the work that analysts used to do manually.
- pigscantfly 7y ago
  They've been using ML to automate most of that work for at least five years.
Diederich 7y ago
> This is better of course, but nothing you can't do in some other way with a little work. I worked at one of the companies doing machine learning on top of large quantities of satellite imagery and selling the results. It's more than a little work; it's pretty challenging, in a variety of ways. Just the training aspect is surprisingly difficult to get right.
anthony_barker 7y ago
Not even the satellites is new. Bloomberg has has oil tanker tracking for a decade as well as oil field sat images.

ISL 7y ago

As an occasional retail trader, it doesn't bother me at all that other market participants are trying to be better informed. In fact, it means that market prices are more likely to be properly-priced. We should not aspire to drag hedge-fund research down to the Main Street level. If there is a place for government in this context, it is to fund the education necessary to help retail investors learn a) how to take beneficial approaches to investing and risk, b) to appreciate the realities of trading against professional investors, and c) to understand how to conceive, test, and implement their own hypotheses in the real world. In general, an informed market is a healthy one.

logifail 7y ago
> As an occasional retail trader, it doesn't bother me at all that other market participants are trying to be better informed. In fact, it means that market prices are more likely to be properly-priced. What about companies that find out about your retail trade before it's even executed, and are able to act on that information to profit from your very intent to trade?
- tryptophan 7y ago
  This is called front running and is super illegal.
  - tanderson92 7y ago
    Not all front-running is illegal, see this article which discusses a few examples, including quote matching and order anticipation: https://blogs.cfainstitute.org/investor/2013/04/24/what-to-do-about-high-frequency-trading/ https://blogs.cfainstitute.org/investor/2013/04/24/what-to-d... Front-running is very common in bond trading.
    - tryptophan 7y ago
      Quote from your article: >HFT greatly reduces average trading costs for investors They are market-making, which is something else entirely than front-running.

dageshi 7y ago

The tone of the article seems like it has an axe to grind. Is it really that much of a surprise that very well capitalised investors have significant advantages of "main street" investors? That's always been the argument for index funds for regular investors, you're never going to compete with hedge funds, why waste your time?

nwah1 7y ago

I remember seeing stories about programmers with access to credit card databases getting indicted for trading on the information. This is a legal version of the same thing, and also probably any attempt to make it illegal would be unenforceable.

dillonmckay 7y ago
‘Outsider information’?
- nwah1 7y ago
  Precisely. It's unfair for the little guys, but lot's of stuff is unfair... such as most of high speed trading, and the people who use arbitrage to skim from the index funds that use predictable purchasing.
  - eeZah7Ux 7y ago
    ...which doesn't make it any more acceptable.
piker 7y ago
One distinction is whether or not there's a duty of confidentiality owed by the possessor of the information with respect to the source of the information (the "non-public" element of "material non-public information"). It doesn't seem here there would be any duty of confidentiality. Presumably the programmers in your example owed a duty of confidentiality to their employer or contract counterparty.
criley2 7y ago
I mean, you don't necessarily have to make it illegal. 1. It's from hedge funds, and you can invest in a hedge fund today and reap the benefits of their strategies yourself 2. There exists a gap right now, where a startup or data analysis firm could provide this kind of information to smaller investors through some kind of subscription model. 3. Ultimately the government itself could "level the playing field" by either gathering this information through other channels and releasing it (BLS or SEC style) or by replicating the strategies and releasing it. I think #3 could be viable. We already require businesses to share very important and actionable info with the government, and since everyone does it, the risks of one firm having data leaked and another not are mitigated. (SEC requires public reports, BLS takes private data and aggregates it for higher level views) If the government required businesses to report data to them on a weekly or monthly basis regarding things that are being satellite tracked, they could release this information on a set schedule to basically demolish the ability of institutional investors to gain secret information to bet against the public positions

citilife 7y ago

I use discussions on the internet... https://projectpiglet.com/ https://projectpiglet.com/ If it's "public" it's fair game for the stock market. My favorite example was the company Quandl[1] which was monitoring charted flight patterns of companies to determine if a company in the area was going to be purchased. [1] https://blog.quandl.com/corporate-aviation-intelligence/aircraft https://blog.quandl.com/corporate-aviation-intelligence/airc...

deleted 7y ago
[deleted]
dna_polymerase 7y ago
Of course you do. You developed that service. Maybe you should mention that in such posts...

rococode 7y ago

I went to a recruiting talk by Two Sigma a while back where they discussed the same thing as an example of "look at this cool stuff we get to work on". They had slides with satellite images of some parking lot and showed how they could segment cars, deal with car-like things on roofs, map regions of parking lots to the right stores, handle different conditions like snow, etc. From a technical perspective it's actually an interesting problem because there are so many things to consider. Really makes you notice how disadvantaged retail traders are, though. I think traditionally a lot of hedge fund strategies are kept pretty secret, but this one is so out of reach and technically difficult for most people that they don't really care if it gets out.

mc32 7y ago
How do they deal with underground parking in urban areas especially?
- mbreese 7y ago
  You wouldn’t. But just because a prediction can’t be 100% accurate because of missing data doesn’t mean that the prediction isn’t useful. Really, what they are likely comparing is the relative changes in parking lot occupancy over time. So it’s not necessary to have complete coverage of all parking lots. You just need enough coverage to make the prediction with a certain level of confidence.
- bluGill 7y ago
  They don't need to. People who park underground, walk or use public transit are close enough to people who park above ground that they get enough data without that. In most cases just one WalMart is enough to infer everything you need to know about them all - if you select the right one for your question. (you wouldn't select just one because then you need to ask if there is something specific about that one, but a statistical sample is enough). Unless your thesis is people who park underground shop differently. That is a valid idea and you cannot answer it via this study. However you can study it by other means if you believe this is important.
- floatrock 7y ago

peteretep 7y ago

How is it that Berkeley can't run https on their webservers?!

secabeen 7y ago
That site is hosted off-campus, coordinating obtaining the cert from the campus CA and then getting it into their off-campus host is probably just a low-priority. They should still do it, though, being off HTTPS is going to hurt their SEO soon enough.

harryh 7y ago

This article is very wrong to confuse "material non-public information" with hard to acquire information like counting cars in satellite photos. Remember the Matt Levine test: insider trading is about theft, not fairness. When a company insider uses private company information to trade (or colludes with an outside party to do so) they are stealing material non-public information from the company for their own benefit. Nothing of the kind is happening when a hedge fund (or anyone else) acquires and analyzes satellite photos. There is no expectation of fairness in the stock market. All sorts of people work very hard and spend a lot of money to develop proprietary information in the pursuit of stronger returns. There is nothing at all wrong wit this, especially because it means that in the end we all get more accurately priced securities.

vostok 7y ago
This is exactly right. If you're upset about this then you should be upset that hedge funds can afford to hire smarter analysts than you, which is just silly.
- davidw 7y ago
  I agree, but where's this end up? Why would retail investors play a game where they're almost certain to lose?
  - piker 7y ago
    That's the why this isn't a perfect quote. Some proponents of insider trading laws pose that exact question and pursue insider trading violations in the interest of protecting the "fairness" of the markets.
  - mrosett 7y ago
    They shouldn’t. Pick a Vanfuard fund, put your money in it, and check back in a couple decades.
    - davidw 7y ago

erickerr 7y ago

https://secondmeasure.com https://secondmeasure.com is like "counting cars in parking lots" on steroids.

waterside81 7y ago

There's a whole marketplace where companies with so-called "Alternative-data" sell their datasets to quant traders. Satellite imagery of farms, passenger traffic through airports, all sorts of random, seemingly disparate data.

floatrock 7y ago
The reason your free weather app asks for your location data isn't for "local weather forecasts". Well, it is, but that's a cost for them. The income comes from keeping that location tracking on all the time and selling which stores and dentists you go to. Someone can link the NYT story from a few months ago when this came to mainstream attention. Hedge funds also fly cesnas over refinery fields to look at the levels of those big tanks you see off the highway... most of them have roofs that move up or down with capacity, and those that don't you look at with an infrared camera to look at relative temperature differences.
- goodfight 7y ago
  If anyone is profiting off my location I better get a cut of it haha. It's amazing what they can do and I enjoy data-centric trading strategies but we don't have to violate individual privacy.
- knd775 7y ago
  Some employees of a credit card company got in trouble for this sort of thing a while ago. They were running queries against customer transactions to figure out sales ahead of earnings calls. For example, they figured out that Chipotle was going to beat expectations and bought a ton of stock just before the earnings call. (This very predictably got the attention of the SEC)
  - Thrymr 7y ago
    Previous discussion of that event on HN: https://news.ycombinator.com/item?id=8966817 https://news.ycombinator.com/item?id=8966817
- 7y ago

lolc 7y ago

I find it somewhat interesting to what lengths traders go for profits. Cue people who explain how this makes the markets more efficient. Something I'm wondering about though: The article makes it sound like this is some new unfair advantage of the big traders. But isn't this an old game? If you can afford to acquire and process the data you're likely going to beat the other participants. That was always true. I don't see what's qualitatively different here. Is there?

roywiggins 7y ago
It increases the information that makes its way into the markets and makes it more likely that the price reflects the what's happening in the real world. A world in which nobody bothered to check whether a company was actually producing anything would turn the stock market into a collective guessing game.
- sjg007 7y ago
  >A world in which nobody bothered to check whether a company was actually producing anything would turn the stock market into a collective guessing game. This is kind of a thing in China.
- ryandrake 7y ago
  The stock market is a collective guessing game. These stories suffer from survivorship bias. For every investor A you hear about who did X and was successful, there are investors B, C, and D who also did X and were not successful, and don’t get articles written about them. There are also investors E, F, and G who don’t do X and may be successful or unsuccessful. Finding someone successful at stock picking and asking him what he did to be successful is like finding that 1 out of 1024 person who flipped a coin heads 10 times in a row and asking him what makes him such a great coin flipper.
  - roywiggins 7y ago
    By that argument, there'd be no successful insider traders, but clearly you can beat make money if you're an insider. You know that the company is (doing well/doing bad) before everyone else does, you trade on that, you make money from the people who are in the dark. The hard part is not regressing to the mean, and consistently beating the market, which sure, nobody can do. But if I learn today that a stock is going to rise tomorrow, and I buy as much as I can, the stock will rise a bit today. So the market has become a little bit more efficient. That I happen to then base all my subsequent trades on overconfidence and tossing darts doesn't retroactively invalidate that trade. You can even make a market more efficient on losing trades- you short a stock, it goes down a bit, but you spend more maintaining the short than the stock goes down. Then, the stock crashes after your short expires. Oops! But your prediction was still partly right, and that information was integrated into the price a bit earlier than it would have been without you.

client4 7y ago

This article reminds me of tracking corporate jets and seeing if they were doing tarmac M&A talks. There are a lot of interesting opportunities for 'corporate information side-channel' attacks and it would be a great opportunity for a new information based corporation to corral this information and sell it.

yial 7y ago

I remember reading years ago about similar data being used to monitor and track the recession (in the US) and potential recovery from. Especially as it related to big-box retailers.

xtacy 7y ago

Reminds me of this article from Thasos Group that analysed smart phone signals at a Tesla factory to infer overtime work: http://thasosgroup.com/blog/thasos-data-tesla-wsj/ http://thasosgroup.com/blog/thasos-data-tesla-wsj/

paradox1234 7y ago

Honestly, I'm not particularly impressed. There are a lot of other trading edges that could be found for a lot less hassle and money, but I guess that's also the beautiful thing about the market - more ways to skin a cat than anyone can imagine, and it's also not a zero sum game.

kami8845 7y ago
> There are a lot of other trading edges that could be found for a lot less hassle What are you thinking about?
- TomMarius 7y ago
  :^)
eeZah7Ux 7y ago
> also not a zero sum game Indeed, it's negative. While the big funds steal money from small investors the waste of energy and the pollution caused by launching satellites remains.

sj3k 7y ago

A company in my town, https://www.descarteslabs.com/ https://www.descarteslabs.com/ Is becoming a popular choice for Hedge Funds seeking this kind of data.

jandrewrogers 7y ago

The same thing has been done with mobile data for many years, with higher fidelity. Unlike with satellite imagery, the measurement is almost continuous, you can accurately characterize which subpopulations are changing behavior, and you can sometimes characterize the change in behavior e.g. shopping more/less, spending more time at different retailers/competitors, etc. Remote sensing like satellite is excellent for other similar types financial modeling applications, but I wouldn't use it for modeling retail as it is much too coarse.

opportune 7y ago

I almost worked for a satellite imaging company that does this stuff. It kind of bothered me that rather than providing farmers or the government with this extremely rich agricultural data, they were just contracted out to specific hedge funds that would use the data to give them an edge in trading. I feel like that's not the best use of that kind of data to society

driverdan 7y ago
If the farmers or governments were willing to pay the same as hedge funds I'm sure they would sell it to them too.

Kapura 7y ago

I think the most important part is this quote: >“What we found is that it’s a gain for large sophisticated investors who can afford the substantial costs of acquiring and processing big alternative data” This is yet another example of wealth accumulating, of rich actors getting richer faster than poor ones. Without some controls on the behaviours that allow wealth to accumulate, it will further concentrate among the wealthy. I would argue that is a bad outcome.

airstrike 7y ago
Just invest in a Vanguard fund and you'll be fine. You don't need to start your own fund, just like you don't need to be your own doctor.
noir_lord 7y ago
Shrugs. Any game where you get more of an advantage the more points you accumulate from a fixed pool of points is going to rapidly result in an unequal share of the points going to the first person to acquire the advantage. Monopoly was created as a game to warn of the pitfalls of capitalism (it's a simplified form of the winner takes if not all a disproportionate share scenario that is life)[1] http://www.bbc.com/capital/story/20170728-monopoly-was-invented-to-demonstrate-the-evils-of-capitalism http://www.bbc.com/capital/story/20170728-monopoly-was-inven... [1] Of course that I think this puts me at odds with many people, I like capitalism, I like the social pluses of it but I believe that capitalism should be sub-ordinate to society directly (more what you see in the north of europe) and less the other way around (pretty much most other places and arguably the US).

notlukesky 7y ago

Nothing unfair at all. Savvy investors have used many tools in the past as well. One analyst at Tiger Fund had got a job at Macy’s in NYC in the cosmetics department before making a decision on whether to have a position on L’Oreal. Unless they have time to do their homework etc.. main street investors should stick to index funds or bonds.

jetti 7y ago

I was listening to a podcast on trading where there was a quant trader as a guest. He mentioned that quant firms have been doing this for years. The guest mentioned that this is no longer an advantage in quant trading as it is common knowledge and can be done by other firms.

sib 7y ago

Something that is visible from space - the number of cars in a parking lot - seems to be the very antithesis of "non-public" information. I think Berkeley missed the thread on this one.

whymsicalburito 7y ago
Yes, but it's not like the cars are ONLY visible from space. You are welcome to organize a large group of people to travel around to parking lots and count cars and then privately share your findings. What the funds are doing is just a more efficient version of that. It's all public information.
- smountjoy 7y ago
  OP agrees with you: antithesis of non-public = public.
- harimau777 7y ago
  If the information is only shared privately, then it seems to me that it is by definition not public information; regardless of what the law says.
  - ironSkillet 7y ago
    What is your definition of "public information"? It's trickier than you may think to define it precisely. For example, is any processed version of "public information" also public information?
- sib 7y ago
  Completely agree. My point is only that there's no real argument here that this is "non-public information," which is what insider trading laws are about.
keiru 7y ago
So snooping on your neighbour gets more morally sound the higher your ladder is?

joshu 7y ago

i was pitched a dataset of this almost 20 years ago. not new.

maitredusoi 7y ago

A soooooo good idea, I wouldn't have thought about it ...

cgy1 7y ago

I think this was a storyline in the TV show Billions.

noir_lord 7y ago
It was, something to do with chip fabrication iirc. This is definitely a case of art imitating life though.

ForHackernews 7y ago

Do hedge funds really consider themselves distinct from 'Wall Street"? I know they aren't always located in lower Manhattan, but as a metonym, Wall Street would commonly be taken to include all of high finance.

module0000 7y ago

Not to duplicate other opinions, but I'll add my similar one.. This is GOOD for the market(the practice, not the article). Price manipulation is a real thing, and fact-based research such as this means prices are more likely to reflect reality. This type of analysis is what we need more of! It's not opinion-based drivel from a talk show or blog, it's empirical data, analyzed impartially. edit: typo

rmah 7y ago

Um... hedge funds are part of "wall street". Silly clickbait headline... sigh

lil-scamp 7y ago

This example of counting cars has been around the geo industry forever, and satellite imagery is pretty cost prohibitive to get on a regular basis. I'd be surprised if hedge funds haven't already moved on to purchasing location-based data (ie mobile phone data) to get even more accurate counts of people in box stores.

50656E6973 7y ago
That seems to be the case: https://news.ycombinator.com/item?id=17081684 https://news.ycombinator.com/item?id=17081684
- lil-scamp 7y ago
  Wonder what the pricing in on that location data is for financial sector customers, and if it's cheaper than a regular purchase of satellite data. It'd be interesting to compare the mobile location data to the satellite data and find the deltas— that would make it a lot easier to build efficient models w/ existing satellite if it's cheaper.

ris 7y ago

So of course the next big business idea is large unrollable printed vinyl top-views of cars. Get your employees to move them around a bit every morning, company's stocks start to rise. Anyone who ever considers this sort of thing to be "unfair" or "cheating" should also consider how comparatively easy it is to make a nonsense of. Who wants to go halfs with me on a large-format printer?

anonu 7y ago

They've been using satellites for over a decade now. The fact that you're hearing about it means there's not much value in this type of data anymore.... I reserve some room for being wrong here. This type of data is ultimately used to predict revenue for consumer companies. Cellphone data and wifi data is much better for that type of prédiction.

goodfight 7y ago
The information about the state of hedge funds takes time to travel, anyone that has been in the quant field knows about these practices. Now that data science and machine learning are becoming more accessible to the public, I'm not surprised that articles are being made on it. There is value in this data now that we have better data processing abilities. There was less technical value in the past and now there is less competitive value in the present.

hnburnsy 7y ago

This was a better story from 2016 on how they are deploying the satellites... "A company called Planet Labs Inc. has launched a small constellation of what it calls “cubesats” that can deliver much more frequent imagery of economically sensitive spots than traditional satellites. Those spots include retailers’ parking lots, oil-storage tanks or farmland" https://www.wsj.com/articles/satellites-hedge-funds-eye-in-the-sky-1471207062 https://www.wsj.com/articles/satellites-hedge-funds-eye-in-t...

eeZah7Ux 7y ago
sigh

totaldude87 7y ago

I think i saw the same thing in the showtime series "Billions"? where Taylor uses this information , thought this is another level of technical analysis :) Moral ethics aside, this looks legal?!, even if the shorts tank shares based on images of car parking lots..

alphagrep12345 7y ago
In Billions, Bobby shrugs off saying - everyone does that, tell me something new. So I thought this is already pretty popular.
tryptophan 7y ago
Watching Billions to learn about what hedgefunds do is like watching Silicon Valley to learn how to make a startup.
- alecco 7y ago
  But it's a good way to learn old cliches in both cases.

pmarreck 7y ago

This was apparently a plot point in Billions. There's market opportunity here. Off the top of my head: 1) Drone photos instead of satellite photos to get at the data cheaper 2) ML counting of cars 3) Some way to legally sense the number of cellphones within a given radius (can possibly be combined with a drone) 4) Simply aiming a network cam at the entrance/exit from across the street and counting the number of people that enter and exit via ML All of these could get you at public data that highly correlates with retail sales

deepnotderp 7y ago
So all of that already exists and is already widely used. I'm a bit confused as to why this is news considering that this has been going on for quite some time.
- pmarreck 7y ago
  Maybe because I'm a pure-software back-end guy who has no idea what the hell institutional investors do in the hardware space to make their targets, because that's not remotely my wheelhouse? ¯\_(ツ)_/¯
jrochkind1 7y ago
Why would drones be cheaper than satellite photos? If they were, wouldn't the hedge funds already be doing that instead? They clearly seem not to me. The issue, if there is one, is that it is not cheap to get this data on a large enough scale to be useful. I suppose it's possible a sampling of just a few parking lots would be representative enough to be useful? In which case you don't need drones, you can just go to the parking lots. Which is apparently what Sam Walton did? I dunno. If that were good enough, again I'm not sure why the hedge funds would be paying for sattelite photos instead.
- csteubs 7y ago
  You're correct that drones are more expensive than satellite imagery at scale--I'm currently working on a startup in this space that cuts the cost of imaging by between 100x and 10,000x depending on the target location.

alphagrep12345 7y ago

There's also a YC company, (https://secondmeasure.com/ https://secondmeasure.com/), that sells anonymized credit card data to hedgefunds so that the funds can directly check the volume of trade happened at walmart, etc.

est31 7y ago

Hedge funds do a ton of crazy research like this. I remember reading about hedge funds buying new cars only to sell them immediately after. They were only interested in their serial numbers which they used to estimate the count of sold cars of a specific model using the german tanks method. Before the sales numbers are out, information like this is very valuable.

gist 7y ago
> about hedge funds buying new cars only to sell them immediately after To get this data you wouldn't need to buy the cars. So it sounds like they did something that sounded like that as opposed to what you remember. Even w/o social engineering you could quite easily get some dealer to give you that type of information (or someone working there or otherwise).
- sokoloff 7y ago
  You could walk the lot and look at the corner of the windshields...
sokoloff 7y ago
We sell customized products. A little over a decade ago, we noticed a few orders going out that were not at all customized or were customized in a way that suggested the customer wouldn't be happy when they got them. We tracked down the account (based in NYC) and they'd been ordering products about every 45 days (mid quarter and end of quarter) and they'd never complained nor asked for a refund. Seems they were after the shipping identifier, as when we changed the shipping identifier to not leak shipment volume information, the orders soon stopped.

Bakary 7y ago

"I [suspect] that we are throwing more and more of our resources, including the cream of our youth, into financial activities remote from the production of goods and services, into activities that generate high private rewards disproportionate to their social productivity. I suspect that the immense power of the computer is being harnessed to this 'paper economy', not to do the same transactions more economically but to balloon the quantity and variety of financial exchanges." --James Tobin, July 1984

fyz 7y ago

Some of the comments conflate retail traders with regular Joes and Janes. A lot of hedge fund investor money comes from pension funds who are placing retirees' money either directly or indirectly into hedge funds. HF compensation is different can of worms.

aiyodev 7y ago

Retailers should try painting images of cars in their parking lots.

kennyloginNTH 7y ago
They could splurge for some covered multi-level parking. That would be nice.

238sjni9 7y ago

what surprises me is how sat images are more cost effective than digital dragnets.

tamaharbor 7y ago

Can someone with access to very recent satellite photography tell me how much corn they got planted in the Midwest this year (compared to last)? Thank you.

socialist_coder 7y ago

Theoretically you could fake a bunch of cars moving around in the parking lots of a failing store. Wait for the hedge funds to bite and move the stock price up... then boom, dump your shares.