3 ms·
This post tells the "History of massive-scale sorting experiments at Google" - https://cloud.google.com/blog/big-data/2016/02/history-of-massive-scale-sorting-
by fhoffa 10y ago
This post tells the "History of massive-scale sorting experiments at Google"
- https://cloud.google.com/blog/big-data/2016/02/history-of-massive-scale-sorting-experiments-at-google https://cloud.google.com/blog/big-data/2016/02/history-of-ma...
When I asked why BigQuery doesn't do these sorts, the answer came straight from the post "Nobody really wants a huge globally-sorted output. We haven’t found a single use case for the problem as stated."
These accomplishments are awesome nevertheless!
Disclaimer: I'm Felipe Hoffa, and I work for Google (http://twitter.com/felipehoffa http://twitter.com/felipehoffa).
- andrioni 10y agoDo you think you could ask someone and find out the cluster sizes they used for those sorts? They mention "With the largest cluster at Google under our control", but it would be more interesting to have an idea of actual numbers, even if just an order of magnitude.
- fhoffa 10y agoI could ask - but then I wouldn't be able to publish unpublished numbers on my own (if I want to keep my job). :)