3 ms·
The spark is already here. I work in a rapidly emerging domain where the trends clearly indicate that traditional concepts of centralized infrastructure cannot
by jandrewrogers 7y ago
The spark is already here. I work in a rapidly emerging domain where the trends clearly indicate that traditional concepts of centralized infrastructure cannot serve the required workloads: operational sensor/geospatial data models. Basically, machinery measuring and reasoning about the physical world at scale, often in real-time. Several aspects of these data models (technical, economic, regulatory) strongly indicate a globally federated implementation that allows for fast, decentralized, ad hoc cooperation of storage-dense compute elements at the edge. The aggregate data velocity is so high that the physics of data model centralization is untenable, so there is a certain near-term inevitability about it even though you can make a centralized solution work today.
There is active research into the theoretical and practical design of systems and protocols that will make this plausible. It has no precedent in literature and it is a very non-trivial problem but the sense is that a practical workable design is achievable in the not too distant future.
It is worth noting that effectively managing climate change requires implementing the same kind of data model with similar theoretical constraints. Building data models of physical reality at scale breaks just about every part of classic data infrastructure architecture.