3 ms·
More than you would imagine. The problem has always been awareness, access, noise and definitions. Most people aren't aware data relevant to them is collected a
by malandrew 14y ago
More than you would imagine. The problem has always been awareness, access, noise and definitions. Most people aren't aware data relevant to them is collected and when they discover data they want, it's in a format that is difficult to use or full of noise. And even if you get clean data in a format you want, it can be hard to find official definitions and criteria describing the data, so it's not uncommon for two agencies to appear to collect the same data, but it's not really the same data because those agencies are working under their own varied definition or criteria.
When I worked as a financial analyst in Brazil, I was amazed at the data that was available from IBGE and various other governmental agencies. Despite the availability of data, it was rarely in a format that was useful without constantly having to writing a special scraper/parser. And even then the same type of data was inconsistent between sources.