3 ms·
Not dead, it's just having it's winter time not unlike AI winter and once it has its similar "chatbot" moment, all will be well. My take on the killer applicat
by teleforce 2y ago
Not dead, it's just having it's winter time not unlike AI winter and once it has its similar "chatbot" moment, all will be well.
My take on the killer application is the climate change for example earthquakes monitoring. For a case study China has just finished building world's largest earthquake monitoring system with the cost of around USD1 Billion across the country with 15K stations [1]. Somehow at the moment is just monitoring existing earthquakes. But let's say there is a big data analytics technique can reliably predicts impending earthquake within a few days, that can probably safe many people and China now still hold the records of the largest mortality and casualty numbers due to earthquakes. Is it probable, the answer is a positive yes based on our work and initial results it's already practical but in order to do that we need integration with comprehensive in-situ IoT networks with regular and frequent data sampling similar to that of China.
Secondly, China also has the largest radio astronomy telescopes and these telescopes together with other radio telescopes collaborate in real-time through e-VLBI to form a virtual giant radio telescopes as big as the earth to monitor distance stars and galaxy. This is how the black hole got its first image but at the time due to logistics one of the telescope remote disks cannot be shipped to the main processing centers in US [2]. At that moment they are not using real-time e-VLBI onky VLBI, and it tooks them several months just to get the complete sets of the black holes observation data. With e-VLBI everything is real-time and with automatic processing it will be hours instead of month. These radio telescopes can also be used for other purposes like monitoring climate change in addition to imaging black holes, their data is astronomical pardon the pun [3].
[1] Chinese Nationwide Earthquake Early Warning System and Its Performance in the 2022 Lushan M6.1 Earthquake:
https://www.mdpi.com/2072-4292/14/17/4269 https://www.mdpi.com/2072-4292/14/17/4269
[2] How Scientists Captured the First Image of a Black Hole:
https://www.jpl.nasa.gov/edu/news/2019/4/19/how-scientists-captured-the-first-image-of-a-black-hole/ https://www.jpl.nasa.gov/edu/news/2019/4/19/how-scientists-c...
[3] Alarmed by Climate Change, Astronomers Train Their Sights on Earth:
https://www.nytimes.com/2024/05/14/science/astronomy-climate-change.html https://www.nytimes.com/2024/05/14/science/astronomy-climate...
- Shrezzing 2y agoI think these examples still loosely fits the author's argument: > There are some cases where big data is very useful. The number of situations where it is useful is limited Even though there are some great use-cases, the overwhelming majority organisations, institutions, and projects will never have a "let's query ten petabytes" scenario that forces them away from platforms like Postgres. Most datasets, even at very large companies, fit comfortably into RAM on a server - which is now cost-effective, even in the dozens of terabytes.
- teleforce 2y agoBig data is not only about storage size but also about processing e.g. RAM. In the next coming years the trend is that there will be more (read exponential) IoT sensor devices than we can ever imagine and the nature of their data will be mostly big data in the sense of size (storage) and the analysis (RAM). Just check the latest cars or EV cars, many of them have hundreds of sensor devices and these devices are already connected to the Internet. Another upcoming example is the latest 5G DECT NR+ standards (the first non-cellular 5G), it will only fuels these massive accumulation of datasets and these monitoring sensor devices do not even get connected to the Internet (think of private factory networks) [1]. Apparently there are limited number of human using or having sensors but for non-human based devices the sky is the limit. For human based communication the data is very limited, we rarely communicate with each others and most of our data now is based on our intermittent media consumptions while streaming audio/video [2]. For IoT sensors devices, they mainly have regular and frequent interval sampling that probably in the ranges of every seconds, minutes, hours, etc. Some if not most of this data is not clean data, there are raw data, and raw data is inherently big and huge compared to the data, for example raw image data vs JPEG data, where the former can be several time bigger in size and processing requirements. [1] DECT NR+: A technical dive into non-cellular 5G: https://news.ycombinator.com/item?id=39905644 https://news.ycombinator.com/item?id=39905644 [2] 50 Video Statistics You Can’t Ignore In 2024: https://www.synthesia.io/post/video-statistics https://www.synthesia.io/post/video-statistics