4 ms·
That link doesn't work for me, so here's the relevant bit: Rapid Storage: A new Cloud Storage zonal bucket that enables you to colocate your primary storage wi
by dang 2y ago
That link doesn't work for me, so here's the relevant bit:
Rapid Storage: A new Cloud Storage zonal bucket that enables you to colocate your primary storage with your TPUs or GPUs for optimal utilization. It provides up to 20x faster random-read data loading than a Cloud Storage regional bucket.
(Normally we wouldn't allow a post like this which cherry-picks one bit of a larger article, but judging by the community response it's clear that you've put your finger on something important, so thanks! We're always game to suspend the rules when doing so is interesting.)
- alobrah 2y agoApologies! First time making a post on hacker news, and I thought this was really exciting news. FWIW, I talked to the presenter after this was revealed during the NEXT conference today, and he seems to have implied that zonal storage is quite close to what Google seems to have with Colossus.
- dang 2y agoOh no, don't apologize - this was a case where you did exactly the right thing and I'm glad you posted! (I was just adding some explanation for more seasoned users who might wonder why we were treating this a bit differently.) Also, welcome to posting on HN and we hope you'll continue!
- noahl 2y agoThere's now another blog post about Rapid storage specifically: https://cloud.google.com/blog/products/storage-data-transfer/how-the-colossus-stateful-protocol-benefits-rapid-storage https://cloud.google.com/blog/products/storage-data-transfer... . (That wasn't up yet when the original post was made.)
- dang 2y agoAh excellent—that's what we were waiting for. I've changed the URL to that from https://cloud.google.com/blog/products/compute/whats-new-with-ai-hypercomputer https://cloud.google.com/blog/products/compute/whats-new-wit... above. Thanks!