3 ms·
It depends on the kind of data you work with. Many kinds of important data models -- geospatial, sensing, telemetry, et al -- can hit petabyte volumes at "hello
by jandrewrogers 2y ago
It depends on the kind of data you work with. Many kinds of important data models -- geospatial, sensing, telemetry, et al -- can hit petabyte volumes at "hello world".
Data models generated by intentional human action e.g. clicking a link, sending a message, buying something, etc are universally small. There is a limit on the number of humans and the number of intentional events they can generate per second regardless of data model.
Data models generated by machines, on the other hand, can be several orders of magnitude higher velocity and higher volume, and the data model size is unbounded. These are often some of the most interesting and under-utilized data models that exist because they can get at many facts about the world that are not obtainable from the intentional human data models.