11 ms·
Nasdaq Acquires Quandl to Advance the Use of Alternative Data
- anonu 8y agoWe're in the midst of a data gold rush. People who have data are struggling to monetize it. If you're a data buyer, you're probably swamped with the quantity and breadth of data providers out there. AI/ML techniques to make sense of this data are still only scratching the surface. I think this is where there is a lot of low-hanging fruit: creating services or tools that allow non-CS/non-Quant people to extract insights from TBs of data... On the exchange side: these guys are always on the prowl for hot new properties to scoop up. The traditional business model of simply earning fees on exchange trading is slowly eroding away (for the last 10 years). So they need to branch out into services and other data plays...
- pplonski86 8y agoYou are right, extracting insights from data is a low-hanging fruit. From what I observe there is huge lack for proper services and tools that can automatically produce insights. There are of course automatic machine learning solutions, but they focus more on machine learning model tuning (in the kaggle style) rather that giving users understanding and awareness of the their data.
- rainboiboi 8y agoI think data scientists needs to produce more actionable insights as oppose to living in their own world. I suspect there will be an rising group of people who can understand data science techniques and communicate them effectively to drive business decisions. These people will be the ones who can clinch the top posts.
- pplonski86 8y agoI'm running automatic machine learning SaaS for 2 years, and after this time, I can tell you that it is huge problem that data scientist are living in their own world (including me! and including data science tools). I had such situation: my user created 50 ML models (xgboost, lightgbm, NN, rf) and ensemble of them. Let's say that single best model was 5% better than single best model with default hyper-params and ensemble was 2% better than tuned best single model. For me it was a huge success, but the user didn't care about model performance. He wants to have insights about data, not tuned blackbox.
- rainboiboi 8y agoI understand every single word you said and fully agree with you. Good point!
- cpb 8y agoIn an interview, Ryan Caldbeck from Circle Up describes two categories of models: brainy models and brawny models. The ensemble described above sounds like a brawny model: you don't care how it made the decision, you're glad it did the heavy lifting and you might even double check the result. However, the user's concern about the black box suggests they wanted what Ryan refers to as a brainy model, one with explicable decisions. Even within the features of the model there could be things to learn about the data. How else are data scientists stuck within their own world?
- inputcoffee 8y agoAlternative take: there isn't that much low hanging fruit there. Hear me out. "To the person who only has a hammer, everything looks like a nail." The data in front of your is the data you want to analyze, but it doesn't follow that that is the data you ought to analyze. I predict that most of the data you look at will result in nothing. The null hypothesis will not be rejected in the vast majority of cases. I think we -- machine learning learners -- have a fantasy that the signal is lurking and if we just employ that one very clever technique it will emerge. Sure random forests failed, and neural nets failed and the SVR failed but if I reduce the step size, plug the output of the SVR into the net and change the kernel... Let me put an example: suppose you want to analyze the movement of the stock market using the movement of the stars. Adding more information on the stars, and more techniques may feel like you're making progress but it isn't. Conversely, even a simple piece of simple information that requires minimal analysis (this companies sales are way up and no one else but you know it) would be very useful in making that prediction. The first data set is rich, but simply doesn't have the required signal. The second is simple, but has the required signal. The data that is widely available is unlikely to have unextracted signal left in it.
- rdlecler1 8y agoThis is where you might use a genetic algorithm or to learn which data to use on a particular prediction. Good AI won’t use all data just trim down to signal.
- pplonski86 8y agoI would like to see use case when AI selects data source to use that humans will never consider.
- jfoutz 8y agoI read a neat criticism of ai techniques. The author pointed out humans can pick out a strong signal as well or better than ai. Humans could pick out signal from an array of weak sources. Ai would identify that case with fewer weak signals required, but it was hard to trust because it was sometimes wrong. I wish I could remember the source. I’m sure it was an article here a few years ago. I want to say it was medical diagnosis based on charts. Anyway, the point was there is a very narrow valley where ai is useful beyond an expert. And that valley is expensive to explore. And, there might not be anything there.
- momentmaker 8y agoThere is Ocean Protocol (https://oceanprotocol.com/ https://oceanprotocol.com/) that lets you sell your data. There is ChainLink (https://chain.link/ https://chain.link/) that lets you sell your data via API service through decentralized oracle nodes. https://blog.goodaudience.com/the-four-biggest-use-cases-for-chainlink-a0245bd07b66 https://blog.goodaudience.com/the-four-biggest-use-cases-for... Monetization is coming soon... in a big way.
- hobofan 8y agoWe also pitched something in that direction with Rlay at Techcrunch Disrupt Berlin: https://techcrunch.com/2018/11/29/rlay-startup-battlefield/ https://techcrunch.com/2018/11/29/rlay-startup-battlefield/
- abakker 8y agoI'm calling it here: the most useful data is private, or, can't be sold due to confidentiality. The fact that data is confidential is great evidence that we know it is useful, but also that we hope others won't use that signal.
- momentmaker 8y agoWhat about banking initiatives in EU causing every bank to open up their data via API. That seems pretty private and confidential. https://www.evry.com/en/news/articles/psd2-the-directive-that-will-change-banking-as-we-know-it/ https://www.evry.com/en/news/articles/psd2-the-directive-tha... ChainLink has an option to use Intel SGX along with TownCrier to provide trusted execution at the processor level. That ensures confidentiality without exposing the data at all. http://www.town-crier.org/ http://www.town-crier.org/
- Immortalin 8y agoFwiw that's what I have been trying to do for the past few years, infrastructure for easier access to algorithmic trading. Shameless plug: https://KloudTrader.com/narwhal https://KloudTrader.com/narwhal
- pmart123 8y agoNasdaq already makes more money on data licensing than on trading fees or IPO's. Each time a professional in the financial services industry wants real-time display data, for example, they have to pay Nasdaq a monthly fee. Nasdaq and NYSE compete for listings less so now for trading fees, but because it makes its data licensing package more valuable.
- claytonjy 8y agoFor finance in particular, I'd say we're drowning in a massive volume of shitty data. A client of mine purchases several fundamental feeds from Quandl, and I email them regularly to point out errors. Not weird, hard, tricky errors, but stuff like "why are all these volumes missing" or "there's a 1-day closing price increase of 1200%" or "you divided when you should have multiplied". This tells me neither Quandl nor the original provider (e.g. Zacks) do any serious data validation, despite claiming to. If the companies people have been paying for decades for this data get it wrong this often, how can I trust any weirder data they're trying to sell me? I thought the point of buying these feeds was to let the seller worry about quality assurance.
- vasilipupkin 8y agothis doesn't matter - any sophisticated user will have their own software to clean the data anyway. Their concern is getting the data, they know how to clean it once they have it.
- claytonjy 8y agoWe're not talking about data cleaning, but about data validation. I can fix a weirdly formatted field (cleaning), but I can't reliably impute most kinds of missing data. I can detect errors, but can't fix most of them without additional information...which is exactly what I'm paying the provider for.
- vasilipupkin 8y agoyou can, there are ways to do it. interpolation, etc. Sometimes the data is missing just because it's not available, you still have to handle that case. proper way of filling in this missing data will depend on what you are using it for - so for provider to do it would be kind of wrong actually.
- claytonjy 8y agoI think we're talking past eachother here. I don't expect the provider to do imputation for me, but I shouldn't have to bug them to get the best version of the data they have. Sure, sometimes missing is missing, but in my experience with Quandl/Zacks, its usually an error on their end. The price jumps are sometimes because they conflated two different tickers. If they divided instead of multiplying (split factors), I have to have external information to even detect the error! Same goes if they get a date wrong somewhere.
- epapsiou 8y agoAny idea as to its valuation?
- minimaxir 8y agoGiven that its last raise was $15M (Canadian) in 2016 (https://www.crunchbase.com/organization/quandl#section-funding-rounds https://www.crunchbase.com/organization/quandl#section-fundi...), and I haven't heard anything about Quandl since then, I'm guessing it's not a 10x exit.
- pmart123 8y agoThat's probably fair. Quandl started by offering the "everyday" investor API access. I know the typical VC approach is to first get users and then scale, but often in investing/financial data products, it seems better to price high and then move down market. If you study the companies with the most success in the past (Bloomberg, CapIQ, MSCI, Eze, Advent, Factset, Morningstar, etc.), none of them started by trying to cater to the DIY investor.
- peterbraden 8y agoIt's crazy how poor the financial data provider offerings out there are. Most financial data is riddled with inconsistencies, wildly overpriced, and in esoteric formats. Simply ingesting financial data in a reliable manner requires significant engineering. For something so important to the economy, it's amazing that there isn't a better solution, or that an open standard hasn't been mandated.
- erichurkman 8y agoFor my current job, we wanted to get a mapping of stock tickers and exchanges to CUSIPs. Every provider we looked at — and this is fundamental trade data — was full of errors and missing values. Couple that with the extortion that is CUSIP (you can't use CUSIP values without a license from them, and licenses start at $xx,xxx+). It's criminally inept. And when you do fix it up, you don't want to publish it, because you spent all your time and resources fixing it… and it becomes a trade secret.
- JaggerFoo 8y agoI agree, CUSIP is also a problem for the privateer (meaning all data needs to be free to use). While I have found a way to find a mapping online, I have no idea of the accuracy and have to trust that the unaware provider QA's the data.
- mlthoughts2018 8y agoThis is why finance is lucrative, similar to esoteric codes in various types of law. Nothing to do with math models or superior prediction, just paying for someone else to fight through identifier hell, exchange protocol hell, etc., and be able to do some mickey mouse math at the end of it. Honestly, this stuff is so bad that the headache of it might fully justify huge finance compensation, and I’ve had colleagues who turned down huge bonuses and raises to leave finance companies solely to avoid this type of stuff and seek a career where the headaches bother them less and they are paid less.
- hendzen 8y ago
- kaybe 8y agoWhat is 'alternative data'? The text only says > 'The company offers a global database of alternative, financial and public data, including information on capital markets, energy, shipping, healthcare, education, demography, economics and society.' which doesn't really answer the question.
- Desustorm 8y agoAlternative data is non-financial data which can be tied to various securities. Financial data, for example, would be EUR USD spot prices. Non-financial data (i.e. "alternative data") could be healthcare reports which you could theoretically couple to e.g. pharma stocks.
- bostik 8y agoThere are quite a few, and I can think of these off the top of my head: - Real-time weather data from major ports and across the main shipping lines - Telemetry from crop and soil report systems - Up-to-date satellite imagery of basically anything large under construction (solar farms, factories, ...) Provide information like that in a machine-readable, consistent format and you have a business. Btw... Using satellite images to track car manufacturers' inventory levels is an old idea, used for more than a decade.
- barbecue_sauce 8y agoThe way I think of alternative data is data about a business or industry that is obtained, collated, and analyzed through non-traditional communication channels, and helps to provide a better picture of how a company or industry is doing than just relying on trade data and financial statements. The best example of this I can think of is companies scraping AliBaba at certain frequencies, trying to ascertain the movement of certain products or raw materials. This data is then sold to investment firms and hedge funds, because they feel it gives them an edge. One company that operates in this space is YipitData. From what I've been told, they started as something similar to GroupOn but then pivoted to this space after scraping for their own competitive intelligence reasons.
- pmart123 8y ago
- monkeydust 8y agoSpoke to these guys a while back. Asked for examples of real alternative data they had...one intertesting was flight data for private jets labeled against which company owned them. Theory being if ceo of company x keeps visiting a place near company y there may be an acquisition or merger in play.
- jdironman 8y agoOut of curiosity, at what point could things be considered insider trading / insider data?
- claytonjy 8y agoIn America, this kind of research in explicitly encouraged, and very much NOT insider trading according to the SEC. Insider trading has to involved _theft_, not just insider knowledge. If you overhear someone talking about an impending acquisition in a coffee shop, and you trade on that information, you're quite safe in the US. European countries can and do consider that insider trading, though.
- jdironman 8y agoReally? I did not know that at all. I say legal insider information is a very, very interesting market then.
- claytonjy 8y agoWell, the whole point of markets is to incorporate information into the price. So if I've done research and think company X is undervalued, I'm going to buy its stock, the price is going to go up, and now that research of mine is better incorporated into the stock price of X. As long as I didn't _take_ that info from anyone (e.g. I can't hack emails, or bribe an accountant, etc. etc.), this is markets behaving exactly as we want them to.
- jdironman 8y ago
- garysahota93 8y agoI had no idea Nasdaq did acquisitions as well. Maybe that's just the engineer in me..
- Maven911 8y agoI've been researching this topic for some time now, alternative data, and not surprised since Nasdaq is a large provider of software (e.g. market-making sw amongst dozens of other sw): QUANDL SPECIFIC: -Quandl has a pretty decent blog that I would check out, you never know what new large corporate policy enacted might get rid of it: https://blog.quandl.com/ https://blog.quandl.com/ GENERAL NOTES: -More and more asset managers are using it and there is some worry that everyone is making the same conclusions off the same data set, and thus no money to be made. Though most practitioners say this is a none-issue, there is more and more alt. data sets out there to chose from, cleaning the data is tricky and testing the veracity of the data provided and knowing how to combine it with others sets is a key competitive advantage that not every asset manager is good at. -The ROI is something that is top of mind but not always easily attributable throughout the year, e.g. one large insight very late in the financial year can bring +100x returns on what was paid for a data provider's software. -Hugely successful funds like Renaissance's Medallion has likely been doing this for a long long time, coupled with top PhDs looking for a lot of statistical correlation with traditional data as well. -More and more data sets that are being created and thrown into a self-learning financial model (aka AI) have a lot of people excited, and certainly there are a lot of small funds being created, though seems to be mostly by young people or not-so-great hedge fund managers. Getting large investors to lay down significant capital has a huge trust component to it, aka want to bet only on succesful grey-haired largely-male dominated folks -A lot of alternative data can be found directly from the Bloomberg terminal e.g. MAPS <Go> function. However my understanding is that it's not that deep, quality is an issue, and everyone has access to it (no real competitive advantage).