3 ms·
> We've witnessed more than exponential growth in the volume and breadth of that data. In fact, by our estimate, we've seen an increase by a factor of about 1x1
by amenod 6y ago
> We've witnessed more than exponential growth in the volume and breadth of that data. In fact, by our estimate, we've seen an increase by a factor of about 1x10^12 over the past decade.
If 10 years ago we had X data collected, now it is 1,000,000,000,000 * X data??? I find this highly surprising and wonder if it is a typo, and if so, what the real numbers are.
- jandrewrogers 6y agoI can't speak for Theo's number (it does seem high) but in my own experience working on similar data models, the typical machine-generated data model size roughly doubles every 6 months on average and has been growing at this rate for two decades. This implies that average data models today are a million times larger than data models a decade ago. Exabyte scale working data models have been something I've needed to consider in designs for at least a few years. We are surprisingly close to overflowing 64-bit integers in real systems.
- amenod 6y agoInteresting - thank you for the insight!