7 ms·

AlloyDB for PostgreSQL

87 points by mhoad 4y ago

drewda 4y ago
I'd be curious to hear if anyone can speak to how this compares to: - Citus columnar storage for Postgres, now owned by Microsoft - Amazon Aurora PostgreSQL compatibility layer - Timescale hypertables
- mattashii 4y ago
  both of the mentioned features from Citus and Timescale are extensions on PostgreSQL, so they do not alter the way storage works in PostgreSQL in any meaningful manner. AlloyDB Seems better compared to Aurora, and they seem to share a significant amount of design principles with log-shipping, but now with an extra caching layer at or near the database instance. As for the 'columnar' features, from the release post it seems like AlloyDB does not specifically store the data at rest in a columnar format, but transforms the rows into columnar for analytical queries (but I'm not sure about that). That indeed improves performance of certain classes of queries.
  - manigandham 4y ago
    Both Citus and Timescale offer different storage (columnar) storage layouts.
- manigandham 4y ago
  Citus and Timescale are postgres extensions that focus on partitioning data primarily. Citus focused on scaling out across multiple nodes based on whatever primary key you wanted. Timescale focused on single-node and then added multiple nodes later, focused on time (or other integer-based values) and more utilities around time-based analysis. The Citus team also had a columnar data storage extension that's finally more production ready and Timescale created their own implementation by using columnar data but still storing it in the default rowstore and handling the differences in the query layer. Aurora Postgres and AlloyDB are fundamentally the same thing and involve taking the "top" portion of actual PostgreSQL (wire protocol, parser, query planner, etc) and attaching it to their own rebuilt storage layer. Since storage is the bottleneck, they can scale that out using their cloud architecture and make it seamless to the DB compute layer on top. Other open-source databases like Yugabyte also follow this approach with their own data layer implementation to add distribution and replication. There are still some fundamental limitations with this approach which is why you still have the concept of instances and primary/replicas instead of one massive distributed instance like Spanner, but most customers want faster/scalable Postgres rather than shifting to a proprietary DB.
Ataraxy 4y ago
Unfortunately this seems to be expensive for arbitrary use.
- PantaloonFlames 4y ago
  Did they publish cost estimates?
  - nhumrich 4y ago
    Yes, there is a fully fleshed out pricing page.
  - topicseed 4y ago
    it is available at – https://cloud.google.com/alloydb/pricing https://cloud.google.com/alloydb/pricing vCPUs $48.24 / vCPU month Memory $8.18 / GB month Regional cluster storage $0.3 / GB month + network pricing
    - phamilton 4y ago
      For anyone looking to compare to Aurora: Aurora r6g instances are 1 vCPU per 8GB memory at $0.12978 per vCPU hr. The equivalent on AlloyDB is $0.15568 Aurora x2g instances are 1 vCPU per 16GB memory at $0.1885 per vCPU hr. The equivalent on AlloyDB is $0.24528 Given the additional memory/cache characteristics and the 2x performance claims, I'm guessing you'll want to go memory heavy, so it _might_ not be terrible to say that a 2 vCPU/16GB Aurora instance ($0.26/hr) may perform similarly to a 1 vCPU/16GB AlloyDB instance ($0.24528/hr). Storage is cheaper on AlloyDB and data transfer likely is as well.
      - mattashii 4y ago
        > storage is cheaper on AlloyDB AlloyDB is 0.30$/GB/month Aurora is 0.10$/GB/month Where does Aurora gain 0.20$/GB/month in storage costs?
        4y ago
sieabahlpark 4y ago
- anaganisk 4y ago
  I give it 3 months :p But seriously GCP products are rarely discontinued if not ever.
- MikeDelta 4y ago
  Or block all your accounts.
sa46 4y ago
What does 100% compatibility refer to? Does alloydb support triggers, partitions, and extensions?
- sa46 4y ago
  I started a cluster and initial testing shows that alloydb is Postgres, so everything should just work. The blog post [1] has more details on the implementation [1]. [1]: https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage https://cloud.google.com/blog/products/databases/alloydb-for...
- hans_castorp 4y ago
  From what I have read, it sounds to me as if they implemented a new storage engine. The needed hooks/interface were introduced gradually over the last two or three releases.
klysm 4y ago
Open source it or upstream it.
- manigandham 4y ago
  Not much about this is useful to open-source or upstream.
e2e4 4y ago
supabase is a good alternative
- anaganisk 4y ago
  When did supabase start supporting analytical worklods?
  - e2e4 4y ago
    via Postgres it supports the analytical workloads; here is an example: https://jitsu.com/blog/supabase https://jitsu.com/blog/supabase
    - anaganisk 4y ago
      You got the wrong impression here I guess but, its a Google Analytics alternative not for OLAP workloads, those both are a lot different.
- manigandham 4y ago
  Supabase is a frontend and API wrapper around a normal Postgres instance. It does nothing to the database itself and is not an alternative to AlloyDB.
  - burggraf 4y ago
    Supabase developer here. That's entirely correct. But in addition to an API wrapper around Postgres, Supabase also offers Storage, Auth, and Edge Functions as a complete baas -- it's not just a database backend.
ZeroCool2u 4y ago
"We’d like all customers to experience AlloyDB for PostgreSQL. That's why AlloyDB is available at no cost during the preview period. The no cost usage during preview is governed by fair usage limits, after which the full pricing shown on this page applies. Below are the monthly fair usage limits for each cost component: Compute: $15,000 of usage, equivalent to at least 125 vCPUs and 1,000 GBs of RAM in us-central1 Storage: $650 of usage, equivalent to at least 2 TB in us-central1" Hadn't seen this mentioned anywhere yet, but looks like it's basically free for most use cases while it's in preview. The pricing otherwise does seem a bit excessive. For the absolute minimum compute/memory resources in an HA setup, you're looking at a minimum ~$120/month. Non-HA is half that. But that's exactly 1 vCPU and 8 GB RAM per instance. If you're curious, max cost per month based on the constraints of the VM's that run the service is about $15k/month in a minimum HA setup, not excluding storage and egress costs. If you need a transactional DB that's fine, but idk that still seems kinda pricey to me. If you can get away with it, it still makes much more sense to try and use BigQuery, append mostly, and keep your queries efficient.

87 points by mhoad 4y ago

I'd be curious to hear if anyone can speak to how this compares to: - Citus columnar storage for Postgres, now owned by Microsoft - Amazon Aurora PostgreSQL compatibility layer - Timescale hypertables

mattashii 4y ago
both of the mentioned features from Citus and Timescale are extensions on PostgreSQL, so they do not alter the way storage works in PostgreSQL in any meaningful manner. AlloyDB Seems better compared to Aurora, and they seem to share a significant amount of design principles with log-shipping, but now with an extra caching layer at or near the database instance. As for the 'columnar' features, from the release post it seems like AlloyDB does not specifically store the data at rest in a columnar format, but transforms the rows into columnar for analytical queries (but I'm not sure about that). That indeed improves performance of certain classes of queries.
- manigandham 4y ago
  Both Citus and Timescale offer different storage (columnar) storage layouts.
manigandham 4y ago
Citus and Timescale are postgres extensions that focus on partitioning data primarily. Citus focused on scaling out across multiple nodes based on whatever primary key you wanted. Timescale focused on single-node and then added multiple nodes later, focused on time (or other integer-based values) and more utilities around time-based analysis. The Citus team also had a columnar data storage extension that's finally more production ready and Timescale created their own implementation by using columnar data but still storing it in the default rowstore and handling the differences in the query layer. Aurora Postgres and AlloyDB are fundamentally the same thing and involve taking the "top" portion of actual PostgreSQL (wire protocol, parser, query planner, etc) and attaching it to their own rebuilt storage layer. Since storage is the bottleneck, they can scale that out using their cloud architecture and make it seamless to the DB compute layer on top. Other open-source databases like Yugabyte also follow this approach with their own data layer implementation to add distribution and replication. There are still some fundamental limitations with this approach which is why you still have the concept of instances and primary/replicas instead of one massive distributed instance like Spanner, but most customers want faster/scalable Postgres rather than shifting to a proprietary DB.

Ataraxy 4y ago

Unfortunately this seems to be expensive for arbitrary use.

PantaloonFlames 4y ago
Did they publish cost estimates?
- nhumrich 4y ago
  Yes, there is a fully fleshed out pricing page.
- topicseed 4y ago
  it is available at – https://cloud.google.com/alloydb/pricing https://cloud.google.com/alloydb/pricing vCPUs $48.24 / vCPU month Memory $8.18 / GB month Regional cluster storage $0.3 / GB month + network pricing
  - phamilton 4y ago
    For anyone looking to compare to Aurora: Aurora r6g instances are 1 vCPU per 8GB memory at $0.12978 per vCPU hr. The equivalent on AlloyDB is $0.15568 Aurora x2g instances are 1 vCPU per 16GB memory at $0.1885 per vCPU hr. The equivalent on AlloyDB is $0.24528 Given the additional memory/cache characteristics and the 2x performance claims, I'm guessing you'll want to go memory heavy, so it _might_ not be terrible to say that a 2 vCPU/16GB Aurora instance ($0.26/hr) may perform similarly to a 1 vCPU/16GB AlloyDB instance ($0.24528/hr). Storage is cheaper on AlloyDB and data transfer likely is as well.
    - mattashii 4y ago
      > storage is cheaper on AlloyDB AlloyDB is 0.30$/GB/month Aurora is 0.10$/GB/month Where does Aurora gain 0.20$/GB/month in storage costs?
      - 4y ago

sieabahlpark 4y ago

anaganisk 4y ago
I give it 3 months :p But seriously GCP products are rarely discontinued if not ever.
MikeDelta 4y ago
Or block all your accounts.

sa46 4y ago

What does 100% compatibility refer to? Does alloydb support triggers, partitions, and extensions?

sa46 4y ago
I started a cluster and initial testing shows that alloydb is Postgres, so everything should just work. The blog post [1] has more details on the implementation [1]. [1]: https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage https://cloud.google.com/blog/products/databases/alloydb-for...
hans_castorp 4y ago
From what I have read, it sounds to me as if they implemented a new storage engine. The needed hooks/interface were introduced gradually over the last two or three releases.

klysm 4y ago

Open source it or upstream it.

manigandham 4y ago
Not much about this is useful to open-source or upstream.

e2e4 4y ago

supabase is a good alternative

anaganisk 4y ago
When did supabase start supporting analytical worklods?
- e2e4 4y ago
  via Postgres it supports the analytical workloads; here is an example: https://jitsu.com/blog/supabase https://jitsu.com/blog/supabase
  - anaganisk 4y ago
    You got the wrong impression here I guess but, its a Google Analytics alternative not for OLAP workloads, those both are a lot different.
manigandham 4y ago
Supabase is a frontend and API wrapper around a normal Postgres instance. It does nothing to the database itself and is not an alternative to AlloyDB.
- burggraf 4y ago
  Supabase developer here. That's entirely correct. But in addition to an API wrapper around Postgres, Supabase also offers Storage, Auth, and Edge Functions as a complete baas -- it's not just a database backend.

ZeroCool2u 4y ago

"We’d like all customers to experience AlloyDB for PostgreSQL. That's why AlloyDB is available at no cost during the preview period. The no cost usage during preview is governed by fair usage limits, after which the full pricing shown on this page applies. Below are the monthly fair usage limits for each cost component: Compute: $15,000 of usage, equivalent to at least 125 vCPUs and 1,000 GBs of RAM in us-central1 Storage: $650 of usage, equivalent to at least 2 TB in us-central1" Hadn't seen this mentioned anywhere yet, but looks like it's basically free for most use cases while it's in preview. The pricing otherwise does seem a bit excessive. For the absolute minimum compute/memory resources in an HA setup, you're looking at a minimum ~$120/month. Non-HA is half that. But that's exactly 1 vCPU and 8 GB RAM per instance. If you're curious, max cost per month based on the constraints of the VM's that run the service is about $15k/month in a minimum HA setup, not excluding storage and egress costs. If you need a transactional DB that's fine, but idk that still seems kinda pricey to me. If you can get away with it, it still makes much more sense to try and use BigQuery, append mostly, and keep your queries efficient.