5 ms·
Most people do not run their data centers like Google. The data centers used the clos network to provide massive bandwidth between very many very similar machi
by benlivengood 4y ago
Most people do not run their data centers like Google. The data centers used the clos network to provide massive bandwidth between very many very similar machines, and so spreading storage across all of them made a lot of sense. Most algorithms were very parallel/distributed in nature and so sharding data non-locally made a lot of sense v.s. running databases or caches on single machines which favors fast localized storage (15K RPM RAIDs and then SSD and NVME, sometimes in SAN chasses to aggregate more IOPS/bandwidth over fast dedicated storage networks). As others pointed out, it couldn't use the traditional file system interface and so doesn't work with commodity software either. Rewriting all the open source software to use an open version of colossus would be a huge task, and without tooling the usefulness of such a thing is very small.
Most companies don't need to process hundreds of terabytes per second to and from disk and so they can centralize storage behind a few network interfaces. Centralized storage is conceptually simpler and more straightforward to manage.
In short, very few folks have the architecture to run something like Colossus, or the need for it, so it doesn't attract open-source replication. Gluster, Ceph, etc. are the closest.