3 ms·
Are you suggesting that it would have been better for Google to use off-the-shelf databases etc? Because at their scale it seems clearly necessary to bring that
by jeffbee 3y ago
Are you suggesting that it would have been better for Google to use off-the-shelf databases etc? Because at their scale it seems clearly necessary to bring that in-house.
- jacquesm 3y agoAt Google's scale a lot of the ordinary limitations do not apply. Unfortunately many companies believe that because Google does it it must be good. It's cargo cult reasoning and the result is endless NIH projects. As for Google's 'Falcon' project: it smacks of NIH to me, but maybe their use cases are specific enough that none of the off-the-shelf bits were usable.
- jeffbee 3y agoWhich part of falcon, specifically, do you think duplicates off-the-shelf software that accomplishes the same things? Swift? Carousel? PLB?
- jacquesm 3y agoI think you are reading something into my comment that I did not actually write.
- jabl 3y agoThere seems to be quite a lot of overlap between Falcon and what the Ultra Ethernet Consortium is ostensibly working on. As well as Amazon's Scalable Reliable Datagram (SRD) thing. All of them, in a way, are about addressing deficiencies in RoCEv2 for large scale latency sensitive networking that you see in HPC and DL training. But none of these are things you can buy today. Well, there's InfiniBand, but if you're wed to Ethernet..
- jeffbee 3y agoI think it's a mistake to assume that an organization this large and sophisticated simply failed to try RoCE. They probably gave it a go but it didn't work out for some technical or economic reason.
- jabl 3y agoI'm sure all the big players in this space have tried RoCE, and found it wanting. Hence all these new approaches.