3 ms·
>telling themselves that it's okay for it to take 5 minutes to sum a million records because it's Big Data If it's the kind of query you run every week, it pro
by crdoconnor 11y ago
>telling themselves that it's okay for it to take 5 minutes to sum a million records because it's Big Data
If it's the kind of query you run every week, it probably is okay.
I've not optimized tons of queries like that before - and prevented other developers from doing so - not because it's "big data", but simply because the risk introduced by changing the code and the effort involved in mitigating that risk simply didn't make the optimization worthwhile.
In most cases I never would have built it myself from the ground up in such a way that it would take 5 minutes (e.g. without indexes), but somebody else's dumb system that's running in production and solving real human problems in an entirely acceptable time frame is still code that's solving people's problems. And, changes always come with a higher cost and risk than most developers imagine - especially in big systems.