3 ms·
Show HN: Streaming thousands of data points in real time with redis and webgl
- sillysaurusx 6y agoBeing able to stream thousands of data points per second was challenging but satisfying. All of this data is being generated in real time, on a server. You can watch your training runs, to keep a pulse on them. I had to fix webdis websockets; no one streamed so much data before. All the data is also stored. Redis can handle as much data as you can throw at it. Theoretically I can view the entire history of every one of these streams. Every data point, at every step, with zero dropouts. You lose so much information when you average losses over a batch. Since this link will die eventually, you can find a video here: https://m.youtube.com/watch?v=vxwjh9gcc2U https://m.youtube.com/watch?v=vxwjh9gcc2U