3 ms·

Thanks for replying, is there any performance numbers for gcsfuse. We use small files and large files. We write small files at high rate. (Jupyter Notebooks) an

• boulos 6y ago
  More than anything, gcsfuse and similar projects that don’t put a separate metadata / data cache in between, reflect GCS’s latency and throughput (with a bit of an extra burden for being done through fuse). GCS has pretty high latency. So even if you write a single byte, expect it to be like 50+ms. This is slower than a spinning disk in an old computer. If you’re just updating a single person’s notebook, they’ll feel it a bit on save (but obviously each person and file is independent). But you can also do about 1 Gbps per network stream (and putting several of those together, 32+ Gbps per VM) such that even a 1 MB file is also probably done in about that much time. I think for streaming writes (save model) gcsfuse may still do a local copy until some threshold before writing out. I’d probably put your models directly on GCS though. That’s what we do in colab and with TensorFlow (sadly, it seems from a quick search that PyTorch doesn’t have this out of the box). Filestore and multi-writer PD will naturally improve over time. But I’m guessing you need something “today”. Feel free to send me a note (contact in profile) if you want to share specifics.