6 ms·

FYI this was unveiled at the 2025 Google Next conference, and they're apparently unveiling a gRPC client for Rapid Storage, which appears to be a very thin wrap

FYI this was unveiled at the 2025 Google Next conference, and they're apparently unveiling a gRPC client for Rapid Storage, which appears to be a very thin wrapper over Colossus itself, as this is just zonal storage.

jauntywundrkind 2y ago
Struggling to find a definition, but seemingly zonal just means there's a massive instance per cluster. Did find some interesting recent (March 28th, 2025) reads though! Colossus under the hood: How we deliver SSD performance at HDD prices https://cloud.google.com/blog/products/storage-data-transfer/how-colossus-optimizes-data-placement-for-performance https://cloud.google.com/blog/products/storage-data-transfer... I kind of thought you meant ZNS / https://zonedstorage.io/ https://zonedstorage.io/ at first, or it's more recent better awesomer counterpart Host Directed Placement (HDP). I wish someone would please please advertize support for HDP, sounds like such a free win, tackling so many write amplification issues for so little extra complexity: just say which stream you want to write to, and writes to that stream will go onto the same superblock. Duh, simple, great.
- re-thc 2y ago
  > Struggling to find a definition, but seemingly zonal just means there's a massive instance per cluster. There are a number of zones in a region. Region usually means city. Zone can mean data center. Rarely just means some sort of isolation (separate power / network).
- bushbaba 2y ago
  It’s GCP’s answer to AWS S3 express zone https://aws.amazon.com/s3/storage-classes/express-one-zone/ https://aws.amazon.com/s3/storage-classes/express-one-zone/
- moandcompany 2y ago
  "Zonal" relates to the concept of "availability zones" which are the next-smallest unit below a (physical) "region." Most instances of a cloud ___ created in a region are allocated and exist at the zonal level (i.e. a specific zone of a region). A physical "region" usually consists of three or more availability zones, and each zone is physically separated from other zones, limiting the potential for foreseeable disaster events from affecting multiple zones simultaneously. Zones are close enough networking-wise to have high throughput and low latency interconnection, but not as fast as same-rack, same-cluster communications. Systems requiring high availability (or replication) generally attain this by placing instances (or replicas) in multiple availability zones. Systems requiring high-availability generally start with multi-zone replication, and Systems with even higher availability requirements may use multi-region replication, which comes at greater cost.
- Dylan16807 2y ago
  Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10% for local parity, 15% free space, 5% in spare drives, and $2000/rack/month overhead, then triple everything for redundancy purposes, then over a 3 year period the price of using your own hard drives is $115/TB and google's price is $720. Over 5 years it's $145 versus $1200. And that's before they charge you massive bandwidth fees.
  - saagarjha 2y ago
    You forgot paying yourself to set that up.
    - Dylan16807 2y ago
      That's covered by the build and overhead numbers. But if you want more on the build side, an extra $10k of labor per rack of 9 servers only increases the cost per TB by about $4.
    - concerndc1tizen 2y ago
      You're paying the same for "cloud engineers". Also, don't forget the hidden cost/risk of giving a third party full access to your data.
      - Sonnigeszeug 2y ago
        Clicking yourself a Bucket takes 5 Minutes. Building a Server and keeping it secure and up-to-date and fixing hardware issues, takes relevant time
        coredog64 2y ago
        Not to mention that I can: - Create a bucket and store 1MB in it without any overhead - Create 50 buckets with strong perimeters around them such that someone deleting the entire account doesn’t bring down the other 49 - Create a bucket and fill it with terabytes of data within seconds and don’t need to wait for hardware to be racked and stacked - Create a bucket, fill it with 2TB of data, and delete it tomorrow Cloud is more than bare metal, but plenty of folks discount the cost benefits of elasticity.
- derefr 2y ago
  In Google Cloud parlance, "regional" usually means "transparently master-master replicated across the availability zones within a region", while "zonal" means "not replicated, it just is where it is."
  - noahl 2y ago
    Slight nit: "zonal" doesn't necessarily mean "not replicated", it means that the replicas could all be within the same zone. That means they can share more points of failure. (I don't know if there's an official definition of zonal.) NB: I am on the rapid storage team.
jeffbee 2y ago
What on this page gives you that impression? Do I have to watch the 2-hour video to learn this?
- korkybuchek 2y ago
  Of course not. Gemini can summarize it for you.
  - CobrastanJorji 2y ago
    I mean, sure, it can easily provide quick text summaries of this sort of thing, but I only consume ML summaries in the forms of podcast discussions between two simulated pundits, as God intended.

jauntywundrkind 2y ago

Struggling to find a definition, but seemingly zonal just means there's a massive instance per cluster. Did find some interesting recent (March 28th, 2025) reads though! Colossus under the hood: How we deliver SSD performance at HDD prices https://cloud.google.com/blog/products/storage-data-transfer/how-colossus-optimizes-data-placement-for-performance https://cloud.google.com/blog/products/storage-data-transfer... I kind of thought you meant ZNS / https://zonedstorage.io/ https://zonedstorage.io/ at first, or it's more recent better awesomer counterpart Host Directed Placement (HDP). I wish someone would please please advertize support for HDP, sounds like such a free win, tackling so many write amplification issues for so little extra complexity: just say which stream you want to write to, and writes to that stream will go onto the same superblock. Duh, simple, great.

re-thc 2y ago
> Struggling to find a definition, but seemingly zonal just means there's a massive instance per cluster. There are a number of zones in a region. Region usually means city. Zone can mean data center. Rarely just means some sort of isolation (separate power / network).
bushbaba 2y ago
It’s GCP’s answer to AWS S3 express zone https://aws.amazon.com/s3/storage-classes/express-one-zone/ https://aws.amazon.com/s3/storage-classes/express-one-zone/
moandcompany 2y ago
"Zonal" relates to the concept of "availability zones" which are the next-smallest unit below a (physical) "region." Most instances of a cloud ___ created in a region are allocated and exist at the zonal level (i.e. a specific zone of a region). A physical "region" usually consists of three or more availability zones, and each zone is physically separated from other zones, limiting the potential for foreseeable disaster events from affecting multiple zones simultaneously. Zones are close enough networking-wise to have high throughput and low latency interconnection, but not as fast as same-rack, same-cluster communications. Systems requiring high availability (or replication) generally attain this by placing instances (or replicas) in multiple availability zones. Systems requiring high-availability generally start with multi-zone replication, and Systems with even higher availability requirements may use multi-region replication, which comes at greater cost.
Dylan16807 2y ago
Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10% for local parity, 15% free space, 5% in spare drives, and $2000/rack/month overhead, then triple everything for redundancy purposes, then over a 3 year period the price of using your own hard drives is $115/TB and google's price is $720. Over 5 years it's $145 versus $1200. And that's before they charge you massive bandwidth fees.
- saagarjha 2y ago
  You forgot paying yourself to set that up.
  - Dylan16807 2y ago
    That's covered by the build and overhead numbers. But if you want more on the build side, an extra $10k of labor per rack of 9 servers only increases the cost per TB by about $4.
  - concerndc1tizen 2y ago
    You're paying the same for "cloud engineers". Also, don't forget the hidden cost/risk of giving a third party full access to your data.
    - Sonnigeszeug 2y ago
      Clicking yourself a Bucket takes 5 Minutes. Building a Server and keeping it secure and up-to-date and fixing hardware issues, takes relevant time
      - coredog64 2y ago
        Not to mention that I can: - Create a bucket and store 1MB in it without any overhead - Create 50 buckets with strong perimeters around them such that someone deleting the entire account doesn’t bring down the other 49 - Create a bucket and fill it with terabytes of data within seconds and don’t need to wait for hardware to be racked and stacked - Create a bucket, fill it with 2TB of data, and delete it tomorrow Cloud is more than bare metal, but plenty of folks discount the cost benefits of elasticity.
derefr 2y ago
In Google Cloud parlance, "regional" usually means "transparently master-master replicated across the availability zones within a region", while "zonal" means "not replicated, it just is where it is."
- noahl 2y ago
  Slight nit: "zonal" doesn't necessarily mean "not replicated", it means that the replicas could all be within the same zone. That means they can share more points of failure. (I don't know if there's an official definition of zonal.) NB: I am on the rapid storage team.

jeffbee 2y ago

What on this page gives you that impression? Do I have to watch the 2-hour video to learn this?

korkybuchek 2y ago
Of course not. Gemini can summarize it for you.
- CobrastanJorji 2y ago
  I mean, sure, it can easily provide quick text summaries of this sort of thing, but I only consume ML summaries in the forms of podcast discussions between two simulated pundits, as God intended.