3 ms·
After leaving Google I founded scalyr.com. To get an idea of the type of work we're doing, this old blog post holds up: https://www.scalyr.com/blog/searching-1t
by snewman 5y ago
After leaving Google I founded scalyr.com. To get an idea of the type of work we're doing, this old blog post holds up: https://www.scalyr.com/blog/searching-1tb-sec-systems-engineering-before-algorithms/ https://www.scalyr.com/blog/searching-1tb-sec-systems-engine....
- rossjudson 5y agoKids these days usually don't understand exactly how fast hardware actually is. ;) "I have 200 million entries in a table I need to compress. I'm gonna write a flume job! I estimate it will take 5 minutes to start the job and a half hour to run! Then I'll spend a few days figuring out how to shard it so it actually finishes." "Sounds good. But I also have this bit of Java code here that does the same compression on my desktop in about 30 seconds. Would you like that instead? You could convert it to C++ if that would make you feel better." yellowbrick.com did some neat stuff pushing the query algebra down into the flash storage firmware.