4 ms·
> This kind of sounds like you've never managed a production system, sorry. Don't be sorry. Instead, challange me with specific problems you have and I will sh
by obulpathi 9y ago
> This kind of sounds like you've never managed a production system, sorry.
Don't be sorry. Instead, challange me with specific problems you have and I will show you how to build solutions.
> how do you process and manage lifecycle for an exabyte of classified/sensitive data
The infrastructure for storing/processing data is in place. If you doubt me, please go ahead and give the above-mentioned tech stack a try and let me know if you still have any problems. Or give me a detailed description of what you have to do and I will tell you how to do it.
> People who talk about "data platforms" in terms of size are like people who judge coder productivity in lines of code
Let me be more clear. I know how to build the infrastructure for that scales of data. I am not saying that I will write the code for processing data. That depends on what kind of data you got and what you want to do with it.
> I guarantee your 10pb "data platform" will choke and die the minute someone shows up with 10gb of historical data and you have to get the timezones right.
See my above comment for the answer.