5 ms·

Citadel and Two Sigma both have had large teams working on this for years. They are making a lot of money doing it but barriers to entry are very high. You need

Citadel and Two Sigma both have had large teams working on this for years. They are making a lot of money doing it but barriers to entry are very high. You need to collect all the data in a way such that you can reconstruct how it looked at any point in time. The vendors can’t be trusted to not make retroactive corrections, so you have to collect it for years before it becomes useful. Doing that takes a lot of time and money.

sseveran 7y ago
Dealing with the corrections is just another product. When I worked with IBES data, which collects all these estimates, you needed to buy a package called something like "As Was" to recreate any arbitrary point in time estimate.
corporateslave5 7y ago
It’s overrated. The alpha gets stripped out of the data sets very fast. Datasets are useful for about 1 year before the whole industry catches on.
- hendzen 7y ago
  The two funds I mentioned can afford exclusivity deals. When they expire the rest of the street catches on, by then they are already on to new ones. At any given time they have multiple in the pipeline.
  - corporateslave5 7y ago
    I worked on an exclusive dataset with citadel. It’s never as valuable, the last few years were a special case where most funds were lagging on the analytics. At this point it’s nearly democratized. Also, even if you know exactly how a company is doing that doesn’t mean the stock will follow the companies real value. The whole thing is way more complex. Two sigma also doesn’t make as much money as they claim, they’re more of a market maker.
    - throwawaymath 7y ago
      I can also confirm this. I used to curate data and develop equities forecasts professionally for about 30 or so funds, including Citadel and Two Sigma. It’s getting harder to build a successful trading strategy based on “quantamental” analysis alone (“alternative data”) each year. A lot of fundamental hedge funds turned to this in the early 2010s as awareness of big data became a thing, thinking they could close the performance gap with the quant funds. It didn’t work. The quant funds that purchase this data use it as only one dimension of analysis to confirm a hypothesis which has already been empirically tested across many other inputs. I have a specific example I can talk about, because my old firm abandoned the data: I found a reliable method for predicting exactly how many Model X and Model S vehicles Tesla sold well before earnings each quarter of 2017, including complete configuration data for each vehicle. Even with that KPI in hand, I couldn’t successfully forecast where the stock would go after each earnings call.
      - moneywoes 7y ago
        Incredible. What strategies are hedge funds using now vs quants?
      - defen 7y ago
        > I found a reliable method for predicting exactly how many Model X and Model S vehicles Tesla sold well before earnings each quarter of 2017 Were they leaking VINs somehow?
        omgwtfbyobbq 7y ago
        I think everyone leaks VINs via the NHTSA's site. https://www.nhtsa.gov/vehicle https://www.nhtsa.gov/vehicle
        texasbigdata 7y ago
        How would that give configurations? Genuine question.
        7y ago
dsfyu404ed 7y ago
>Citadel and Two Sigma both have had large teams working on this for years. They are making a lot of money doing it but barriers to entry are very high. You need to collect all the data in a way such that you can reconstruct how it looked at any point in time. The vendors can’t be trusted to not make retroactive corrections, so you have to collect it for years before it becomes useful. Doing that takes a lot of time and money. As someone who works in this space I assure you the barriers to entry are not that high. If I were to sort the various lists of accounts alphabetically you'd see names like Citadel and Two Sigma flanked by hoards of tiny funds you've never heard of. Many of these funds you've never heard of are 5-10man businesses or teams within larger funds. Making money is much more of an algorithms problem than a resources problem.
- throwawaymath 7y ago
  Can also confirm this - the research team I used to work on was just three people. We worked directly with some 30 or so funds, successfully, including Citadel and Two Sigma.
  - bananaquant 7y ago
    This is fascinating. Have you been working for a bigger company back then or was it an independent gig?
  - blevin 7y ago
    Did you find the work to be intellectually satisfying and creative, or was it somewhat rote in the sense of having to spend lots of energy on data hygiene?
    - throwawaymath 7y ago
      I found it to be very intellectually satisfying and creative! But there were parts to the job which were also very boring, including cleaning data. Cleaning data took up probably about as much time as all the fun analysis and exploration.
_sword 7y ago
One of the wildest things I found while working on Wall Street was that ad blockers are wholesale selling data to analysts and funds. For example, I reached out to Ghostery asking if they had data about the data analytics tech / ad tech vendors used by industry, website, etc. to estimate market share and they responded back with a sales person telling me I could buy that data from them for five figures per year and that a number of my peers and clients were doing the same.
- pythux 7y ago
  Hi, I work on the Ghostery extension. Ghostery used to belong to a company named Evidon, which had this business model of collecting data to sell it in some form (the feature was called "Ghostrank"). In 2017, Ghostery was acquired by Cliqz (which builds an independent and private search engine from Germany), and since then a few things happened: (1) the extension was open-sourced, (2) Ghostrank was completely removed and (3) Ghostery is now developing paid products as a business model. For example we launched Ghostery Insights[1] and Ghostery Midnight[2] recently, both of which are subscription-based. [1]: https://www.ghostery.com/insights/ https://www.ghostery.com/insights/ [2]: https://www.ghostery.com/midnight/ https://www.ghostery.com/midnight/
  - _sword 7y ago
    Thank you for the update! I contacted Ghostery back in 2017, though I don't recall when exactly, and at that time I received a response from a salesperson. I was working on a project to try and understand market share of digital marketing software across the web at the time.
- stubish 7y ago
  Ghostery seems to have fallen completely out of favor for this behavior, replaced by the EFF's Privacy Badger, actual ad-blockers, and even some browser built in functionality. It would be interesting to know which other privacy plugins that sell data. Current recommendations seem to be pretty unanimously favoring plugins which do not, but it is always a moving target.

sseveran 7y ago

Dealing with the corrections is just another product. When I worked with IBES data, which collects all these estimates, you needed to buy a package called something like "As Was" to recreate any arbitrary point in time estimate.

corporateslave5 7y ago

It’s overrated. The alpha gets stripped out of the data sets very fast. Datasets are useful for about 1 year before the whole industry catches on.

hendzen 7y ago
The two funds I mentioned can afford exclusivity deals. When they expire the rest of the street catches on, by then they are already on to new ones. At any given time they have multiple in the pipeline.
- corporateslave5 7y ago
  I worked on an exclusive dataset with citadel. It’s never as valuable, the last few years were a special case where most funds were lagging on the analytics. At this point it’s nearly democratized. Also, even if you know exactly how a company is doing that doesn’t mean the stock will follow the companies real value. The whole thing is way more complex. Two sigma also doesn’t make as much money as they claim, they’re more of a market maker.
  - throwawaymath 7y ago
    I can also confirm this. I used to curate data and develop equities forecasts professionally for about 30 or so funds, including Citadel and Two Sigma. It’s getting harder to build a successful trading strategy based on “quantamental” analysis alone (“alternative data”) each year. A lot of fundamental hedge funds turned to this in the early 2010s as awareness of big data became a thing, thinking they could close the performance gap with the quant funds. It didn’t work. The quant funds that purchase this data use it as only one dimension of analysis to confirm a hypothesis which has already been empirically tested across many other inputs. I have a specific example I can talk about, because my old firm abandoned the data: I found a reliable method for predicting exactly how many Model X and Model S vehicles Tesla sold well before earnings each quarter of 2017, including complete configuration data for each vehicle. Even with that KPI in hand, I couldn’t successfully forecast where the stock would go after each earnings call.
    - moneywoes 7y ago
      Incredible. What strategies are hedge funds using now vs quants?
    - defen 7y ago
      > I found a reliable method for predicting exactly how many Model X and Model S vehicles Tesla sold well before earnings each quarter of 2017 Were they leaking VINs somehow?
      - omgwtfbyobbq 7y ago
        I think everyone leaks VINs via the NHTSA's site. https://www.nhtsa.gov/vehicle https://www.nhtsa.gov/vehicle
      - texasbigdata 7y ago
        How would that give configurations? Genuine question.
        7y ago

dsfyu404ed 7y ago

>Citadel and Two Sigma both have had large teams working on this for years. They are making a lot of money doing it but barriers to entry are very high. You need to collect all the data in a way such that you can reconstruct how it looked at any point in time. The vendors can’t be trusted to not make retroactive corrections, so you have to collect it for years before it becomes useful. Doing that takes a lot of time and money. As someone who works in this space I assure you the barriers to entry are not that high. If I were to sort the various lists of accounts alphabetically you'd see names like Citadel and Two Sigma flanked by hoards of tiny funds you've never heard of. Many of these funds you've never heard of are 5-10man businesses or teams within larger funds. Making money is much more of an algorithms problem than a resources problem.

throwawaymath 7y ago
Can also confirm this - the research team I used to work on was just three people. We worked directly with some 30 or so funds, successfully, including Citadel and Two Sigma.
- bananaquant 7y ago
  This is fascinating. Have you been working for a bigger company back then or was it an independent gig?
- blevin 7y ago
  Did you find the work to be intellectually satisfying and creative, or was it somewhat rote in the sense of having to spend lots of energy on data hygiene?
  - throwawaymath 7y ago
    I found it to be very intellectually satisfying and creative! But there were parts to the job which were also very boring, including cleaning data. Cleaning data took up probably about as much time as all the fun analysis and exploration.

_sword 7y ago

One of the wildest things I found while working on Wall Street was that ad blockers are wholesale selling data to analysts and funds. For example, I reached out to Ghostery asking if they had data about the data analytics tech / ad tech vendors used by industry, website, etc. to estimate market share and they responded back with a sales person telling me I could buy that data from them for five figures per year and that a number of my peers and clients were doing the same.

pythux 7y ago
Hi, I work on the Ghostery extension. Ghostery used to belong to a company named Evidon, which had this business model of collecting data to sell it in some form (the feature was called "Ghostrank"). In 2017, Ghostery was acquired by Cliqz (which builds an independent and private search engine from Germany), and since then a few things happened: (1) the extension was open-sourced, (2) Ghostrank was completely removed and (3) Ghostery is now developing paid products as a business model. For example we launched Ghostery Insights[1] and Ghostery Midnight[2] recently, both of which are subscription-based. [1]: https://www.ghostery.com/insights/ https://www.ghostery.com/insights/ [2]: https://www.ghostery.com/midnight/ https://www.ghostery.com/midnight/
- _sword 7y ago
  Thank you for the update! I contacted Ghostery back in 2017, though I don't recall when exactly, and at that time I received a response from a salesperson. I was working on a project to try and understand market share of digital marketing software across the web at the time.
stubish 7y ago
Ghostery seems to have fallen completely out of favor for this behavior, replaced by the EFF's Privacy Badger, actual ad-blockers, and even some browser built in functionality. It would be interesting to know which other privacy plugins that sell data. Current recommendations seem to be pretty unanimously favoring plugins which do not, but it is always a moving target.