8 ms·

AlloyDB Omni – run AlloyDB anywhere

144 points by forrestbrazeal 4y ago

lorec0re 4y ago
wow
CharlieDigital 4y ago
Google Cloud gets a lot of heat because of Google's legacy of killing things off (the exact opposite of what enterprise decision makers want when selecting a cloud platform). But I have to applaud Google for the excellent first party emulators and local tooling that they provide (Alloy Omni doesn't seem like an emulator on first readthrough, but enabling fast local iteration has the same effect). The Firebase emulator suite makes development soooo fast because it cuts out a deployment step to a shared resource that has to be coordinated. Meanwhile, Microsoft as been dragging their feet on supporting CosmosDB emulation on Arm hardware [0]. I was a big fan of CosmosDB, but kind of gave up on it after switching to an M1 MBP because it was unwieldy to work with without a local emulator. [0] https://github.com/Azure/azure-cosmos-db-emulator-docker/issues/54 https://github.com/Azure/azure-cosmos-db-emulator-docker/iss...
- yafbum 4y ago
  Right now it looks just like an emulator - not based on the description, but based on it being only available right now as a free developer edition "not suitable for production use". Wondering how pricing will work for the real deal, and also whether it'll be horizontally scalable like their cloud based solutions.
- GabeWeiss_ 4y ago
  Definitely not an emulator. It's fully AlloyDB...in a container. We'll be working moving forward to smooth out the install process (tech preview, focus was on getting it in y'alls hands). In my mind, it's a proper Postgres drop-in replacement (Yes, of course I'm biased because I work at Google) but the tech is legit.
  - fnordpiglet 4y ago
    I’ve gotta say without unambiguous and long term unencumbered licensing, preferably with open source attached, I can’t foresee recommending any place I’ve got influence at to even look at it yet. Hopefully you can push them in a direction that is plausibly usable by anyone other than hobbyists. Even as a local dev tool it leaves too many holes open in the licensing and distribution model. Letting a team build a process and workflow around a time bomb would be negligence. It’s sad to characterize this as such, but we’ve all been burned too many times by globo mega corp whims.
    - GabeWeiss_ 4y ago
      100% hear you. I realize this may sound hollow coming from a Google employee, but stay tuned on specifics around licensing and costs. Being a tech preview, there's been virtually no discussion on specifics yet. We just wanted to get the tech in folks' hands. We're pushing hard for a free tier of usage (and the product team isn't pushing back currently).
      - fnordpiglet 4y ago
        Not hollow! Good luck.
  - CharlieDigital 4y ago
    > it's a proper Postgres drop-in replacement The only downside I see is that going through the deployment process, it seems that the smallest instance I can provision is 2vCPU/16GB which if I'm looking at the pricing table correctly, equates to $230/mo. for a single instance. The web console only lets me create a cluster with minimum 2 instances so it seems like I'm looking at $460 minimum to deploy Alloy. That feels a bit excessive.
    - GabeWeiss_ 4y ago
      You're looking at Cloud AlloyDB sounds like? If you deploy AlloyDB to GCE (instead of locally, etc) you can provision a smaller instance than that. 2cpu(shared core)/4GB memory with a 20GB boot disk (AlloyDB takes up ~13) would run you $26/month before network/storage costs.
- evntdrvn 4y ago
  At least there is a Cosmos emulator... There has been a ticket open requesting an Azure Service Bus emulator for FIVE YEARS lolsob https://github.com/Azure/azure-service-bus/issues/223 https://github.com/Azure/azure-service-bus/issues/223 Which is too bad, because ASB is awesome, IMO the best Azure service.
  - CharlieDigital 4y ago
    Especially head scratching given that Google's Pub/Sub has an emulator as does SNS/SQS via LocalStack. I am also a big fan of ASB and I cannot fathom how a multi-billion dollar company that bills itself on developer productivity can't ship emulators for their cloud suite. If even for their own internal testing and release validation.
  - yellowfool 4y ago
    Really, then again the bar is low in Azure.
skunkworker 4y ago
That’s incredible. I wasn’t expecting them to allow you to run AlloyDB on premise but this is a potential game changer for on-prem Postgres instances, especially if you are doing analytical queries on the same data set.
- GabeWeiss_ 4y ago
  That's how I've been thinking/talking about it. It should be a drop-in replacement for Postgres. Obviously the install process isn't as smooth yet, but, technical preview...
- ellisv 4y ago
  Eh, there are other options for this that are more mature and more likely to be supported in the long run. The Citus columnar store extension has been around for almost a decade.
  - adapa 4y ago
    I looked at the citus columnar however lack of updates and deletes even occasionally kept me away.
emptysea 4y ago
I wonder how they make transactional workloads 2x faster vs normal Postgres The analytics workloads improvements seem pretty straightforward (or at least there is prior art like timescale)
- ddorian43 4y ago
  > I wonder how they make transactional workloads 2x faster vs normal Postgres There can be improvements like what OrioleDB is trying to do: https://github.com/orioledb/orioledb/ https://github.com/orioledb/orioledb/
- ellisv 4y ago
  My best guess is that transactional workloads are mostly improved by automatically adding the right indexes and parameter tuning. For example, OtterTune also advertises "2x on price/performance over unoptimized DBs".
stevesimmons 4y ago
> AlloyDB Omni provides full compatibility with PostgreSQL extensions
- ellisv 4y ago
  Makes sense since it's not in their managed infra
GiorgioG 4y ago
> The free developer edition of AlloyDB Omni is currently available as a technology preview. Check back for full product pricing information. Thanks, but no thanks!
wokwokwok 4y ago
> 3. Use of the Software. > 3.1. Use. You may authorize employees, agents, and subcontractors to use the Software in accordance with this Section 3, so long as you remain responsible for them. You may make a reasonable number of copies of the Software for back-up and archival purposes. > You acknowledge that the Software is a preview offering not intended for production environments, and you agree that you will only use the Software in non-production environments. Anyway, all that stuff in the docs is lovely, but if you just want to have a look: pip install gsutil gsutil cp -r gs://alloydb-omni-install/$(gsutil cat gs://alloydb-omni-install/latest) . The install scripts are only 16K, look at `installer/scripts/start_alloydb.sh` for more, but basically it just runs the two docker containers listed in https://cloud.google.com/alloydb/docs/omni/install#install https://cloud.google.com/alloydb/docs/omni/install#install Seems kind of weird, having a one-time install script to prep a machine (but only a specific type of machine!) that you then run a pair of docker containers on to me, honestly. Eventually consistent deployment states? eh. whatever...
- mdaniel 4y ago
  For those who want to skip the gsutil part, that bucket and its keys are public: curl -vJLO https://storage.googleapis.com/alloydb-omni-install/$(curl -fs https://storage.googleapis.com/alloydb-omni-install/latest)alloydb_omni_installer.tar.gz The `latest)alloy` is not a typo, the contents of the "latest" file ends with a slash and GCS doesn't tolerate `//` (presumably just like S3 wouldn't)
arecurrence 4y ago
Has Google contributed some of these changes back to Postgres? I went looking and even a search for "alloydb" in some postgres mailing lists return no results. It has some very exciting features, I was a bit surprised to see nobody talking about it.
- Cockbrand 4y ago
  If I understand correctly, AlloyDB is a Postgres compatible re-implementation, so there's no shared code. So this is a bit like Wine compared to Windows, only with the big corp re-implementing the OSS software instead of the other way round.
  - ppg677 4y ago
    You can't really be 100% postgres compatible without borrowing a lot of implementation
    - aeyes 4y ago
      Query and storage layer are nicely separated in the Postgres codebase, you can absolutely rip out the complete backend if you really want it. There is a Postgres interface to Spanner which probably did that.
  - GabeWeiss_ 4y ago
    OMG no, we didn't reimplement Postgres. Definitely started from the PG kernel and made modifications from there.
    - Cockbrand 4y ago
      I often reference Cunningham‘s Law, and I’m glad it’s in full effect :) Thank you for the clarification!
- GabeWeiss_ 4y ago
  Not as of yet, no. I've gently nudged the eng/product teams in this direction, but for now it's all being kept in-house.
lysecret 4y ago
Hugely disappointed by Alloydb. Price and performance for a write heavy workload where off the charts horrible. It didn’t even support disabling an instance and it did only storage autoscaling much worse overall experience than using Aurora we moved back to cloudsql. Does someone have good experience with Alloydb?
- GabeWeiss_ 4y ago
  I have many thoughts here...but let's start with: Do you have your process of how you tested performance? My guess is the eng team would love to look at what you tested.
  - evntdrvn 4y ago
    thank you for the measured response rather than immediately becoming defensive :)
    - GabeWeiss_ 4y ago
      WHAT DO YOU MEAN?! RAAAAAGE!! ;) I mean, if someone legitimately finds workloads that aren't performing well, our engineering team SHOULD want to know about it, right?
      - gtirloni 4y ago
        Do they?
        GabeWeiss_ 4y ago
        Okay, I can't speak for ALL engineering teams, but at the very least I know THIS engineering team absolutely does.
  - lysecret 4y ago
    As a general feedback I have written it down here before: https://news.ycombinator.com/item?id=34304376 https://news.ycombinator.com/item?id=34304376 Sorry for being a bit negative but I was very excited for AlloyDB as a potential serverless offering for Postgres. Especially after having good experience with Aurora. And we ended up wasting a decent amount of resources migrating to and then away from it, thus my frustration. In our use-case we use Postgres essentially as a cache. We have many cloud runs (approx 100) writing a lot of data in parallel which we then on a schedule query in certain ways and put it to GCP. Its quite a bit of data, about 1 TB a day. It was a very CPU bound effort and to get acceptable performance we had to rely on the 8 core configuration, and we couldn't reduce the memory (from 64). We were getting similar performance with a 4 core 8gb Cloud Sql instance. Probably not the most representitive workload but this had exactly the opposite effect to what we wanted (e.g. much more wasted resources and less serverless). Also the fact you couldn't disable it was an absolute joke (we have prod dev and staging env and if we have to have a big always running DB for each of them, you can see how that is unacceptable). We are happy with CloudSql though.
fnordpiglet 4y ago
I see, not open source, ambiguous long term licensing, or even a commitment to a forever free. I did notice the release reads just like an internal AWS PR with Andy’s preferred structure… guess all those aws folks they recruited are making an impact.
brightball 4y ago
This looks really sharp. I wonder if Alloy gives some of the data compression benefits of columnar databases with that hybrid approach?
- roncesvalles 4y ago
  The columnar form is only in-memory and derived from the row-oriented form on disk. Like a denormalized cache of sorts.
deleted 4y ago
[deleted]
TheTaO 4y ago
Why would anyone use this as opposed to using Postgres? The value prop of run-anywhere applies to Postgres as well. I see column store and index advisor as the two features but if I don't need these, is there any reason?
- GabeWeiss_ 4y ago
  It's twice as fast as out of the box PG for most things, and up to 100x faster for reads, depending on what you're doing. So there's that. Also, from a manageability, on top of the index advisor, there's also vacuum management, so it will figure out when the best time to do the garbage cleanup while minimizing impact on performance.
  - dx034 4y ago
    The 100x analytics seems due to columnar storage like with TimescaleDB, but how do you get improvements on the other things? Is it really faster on a database with good indexes?
    - GabeWeiss_ 4y ago
      It really is, yes. I can't go into a lot of detail on the "why" because it's not open source and the product team would murder me...but I highly encourage folks to try it for themselves. Nothing else convinced me until I did it for myself. :)
- virogenesis 4y ago
  also how does it compare to PGHero for index advising?
  - GabeWeiss_ 4y ago
    We encourage folks to draw their own conclusions, but we have done comparisons and ours is more efficient and produces faster results. I only say test yourself because I can't give specific details of how/why, so don't take my word for it. :)
janpio 4y ago
Tried running it on the cloud dev environments I usually use (as I am on Windows myself). Gitpod: 2023-03-29 18:13:54.044 UTC: [alloydb_util.sh:90] FATAL: Docker service must be active to run AlloyDB Omni GitHub Codespaces (after increasing memory): 2023-03-29 18:37:55.236 UTC: [alloydb_util.sh:76] FATAL: AlloyDB Omni requires cgroups V2 to run. GitHub Actions: 2023-03-29 18:53:52.766 UTC: [alloydb_util.sh:44] AlloyDB requires at least 16GB of RAM to run. Only 7 GB available. Please increase available RAM and retry Not today it seems. Shame, sounded super interesting.
- GabeWeiss_ 4y ago
  Yeah, I know. Apologies. Stay tuned though! For the tech preview launch we really focused on a golden path (narrow). We'll be expanding compatibility as we march towards GA.
qeternity 4y ago
Is there any HA built into this? Performance claims look great.
- GabeWeiss_ 4y ago
  Not for Omni no. End-users are responsible for any HA/DR/Replication needs. Cloud AlloyDB has all of it built in, but since it was so tightly tied to the Cloud infrastructure it was simplified for Omni.
inscrutable 4y ago
This looks great! For the 2x general improvement, how much of that is due to setting/modifying postgres settings and adding good indexes, versus improvements to the code itself?
- GabeWeiss_ 4y ago
  Code improvements. Indices aren't taken into account at all since that's so workload dependent. Having said that, the index advisor can make finding the right indices to improve performance there more easily as well. Vacuum management handles figuring out the right time to do garbage collection for you, etc. None of that is part of that 2x improvement.
  - inscrutable 4y ago
    That's impressive, surprised there's that much juice that could be squeezed without changing architecture. I ask because AWS Aurora is flattered in comparison to RDS due to e.g. some dumb oob pg settings that a lot of people don't change. From seeing the cloud alloydb I had imagined most of the improvements were due to the wal-shipping and cloud native aspects.
antman 4y ago
What does it mean in terms of license?
- GabeWeiss_ 4y ago
  We don't have specific details yet about licensing or pricing. Stay tuned as we get closer to GA. Having said that, we're pushing hard for a free usage tier and the product team isn't pushing back. :)
ethicalsmacker 4y ago
Maybe I'm old.. but everytime I read Alloy I think of the old modeling language out of MIT to prove a program's correctness. Gives me PTSD thinking of grad school. https://alloytools.org/ https://alloytools.org/
chadbr 4y ago
A quick demo: AlloyDB Omni Columnar Engine Fast Analytics Demo - YouTube https://www.youtube.com/watch?v=f_dvdKMq6og https://www.youtube.com/watch?v=f_dvdKMq6og
vira28 4y ago
Since we run Vanilla PG on on-prem, this is interesting. Will try once we get clarity on licensing. (Probably we can't use if the code is not open, so let's see)
- GabeWeiss_ 4y ago
  As a heads up, we're unlikely to make the codebase for it open. I might be able to convince the product team to open source some components of it (our GM has talked about this before in a couple articles), the whole thing won't be open sourced.
  - yandie 4y ago
    What's the plan for the license for this? Especially path to production?
    - GabeWeiss_ 4y ago
      I'm assuming you mean "I start using this under a free dev license, and want to shift up to the paid production version"? And what that looks like? If that's the case, I don't know yet. That's literally being discussed/hammered as we speak, and we likely won't have a good answer until we're ready to go public preview (not sure on timeline right now, depends heavily on how the tech preview goes). This stage of the game is literally a "We built an awesome thing, please poke at it and tell us where it does/doesn't work!".
jefflynn 4y ago
Redrock Postgres - Runing Cloud Native Databse Anywhere Network Attached Tablespace https://doc.rockdata.net/admin/network-tablespace/ https://doc.rockdata.net/admin/network-tablespace/ PostgreSQL Multitenant https://doc.rockdata.net/features/multitenant/ https://doc.rockdata.net/features/multitenant/

wow

Google Cloud gets a lot of heat because of Google's legacy of killing things off (the exact opposite of what enterprise decision makers want when selecting a cloud platform). But I have to applaud Google for the excellent first party emulators and local tooling that they provide (Alloy Omni doesn't seem like an emulator on first readthrough, but enabling fast local iteration has the same effect). The Firebase emulator suite makes development soooo fast because it cuts out a deployment step to a shared resource that has to be coordinated. Meanwhile, Microsoft as been dragging their feet on supporting CosmosDB emulation on Arm hardware [0]. I was a big fan of CosmosDB, but kind of gave up on it after switching to an M1 MBP because it was unwieldy to work with without a local emulator. [0] https://github.com/Azure/azure-cosmos-db-emulator-docker/issues/54 https://github.com/Azure/azure-cosmos-db-emulator-docker/iss...

yafbum 4y ago
Right now it looks just like an emulator - not based on the description, but based on it being only available right now as a free developer edition "not suitable for production use". Wondering how pricing will work for the real deal, and also whether it'll be horizontally scalable like their cloud based solutions.
GabeWeiss_ 4y ago
Definitely not an emulator. It's fully AlloyDB...in a container. We'll be working moving forward to smooth out the install process (tech preview, focus was on getting it in y'alls hands). In my mind, it's a proper Postgres drop-in replacement (Yes, of course I'm biased because I work at Google) but the tech is legit.
- fnordpiglet 4y ago
  I’ve gotta say without unambiguous and long term unencumbered licensing, preferably with open source attached, I can’t foresee recommending any place I’ve got influence at to even look at it yet. Hopefully you can push them in a direction that is plausibly usable by anyone other than hobbyists. Even as a local dev tool it leaves too many holes open in the licensing and distribution model. Letting a team build a process and workflow around a time bomb would be negligence. It’s sad to characterize this as such, but we’ve all been burned too many times by globo mega corp whims.
  - GabeWeiss_ 4y ago
    100% hear you. I realize this may sound hollow coming from a Google employee, but stay tuned on specifics around licensing and costs. Being a tech preview, there's been virtually no discussion on specifics yet. We just wanted to get the tech in folks' hands. We're pushing hard for a free tier of usage (and the product team isn't pushing back currently).
    - fnordpiglet 4y ago
      Not hollow! Good luck.
- CharlieDigital 4y ago
  > it's a proper Postgres drop-in replacement The only downside I see is that going through the deployment process, it seems that the smallest instance I can provision is 2vCPU/16GB which if I'm looking at the pricing table correctly, equates to $230/mo. for a single instance. The web console only lets me create a cluster with minimum 2 instances so it seems like I'm looking at $460 minimum to deploy Alloy. That feels a bit excessive.
  - GabeWeiss_ 4y ago
    You're looking at Cloud AlloyDB sounds like? If you deploy AlloyDB to GCE (instead of locally, etc) you can provision a smaller instance than that. 2cpu(shared core)/4GB memory with a 20GB boot disk (AlloyDB takes up ~13) would run you $26/month before network/storage costs.
evntdrvn 4y ago
At least there is a Cosmos emulator... There has been a ticket open requesting an Azure Service Bus emulator for FIVE YEARS lolsob https://github.com/Azure/azure-service-bus/issues/223 https://github.com/Azure/azure-service-bus/issues/223 Which is too bad, because ASB is awesome, IMO the best Azure service.
- CharlieDigital 4y ago
  Especially head scratching given that Google's Pub/Sub has an emulator as does SNS/SQS via LocalStack. I am also a big fan of ASB and I cannot fathom how a multi-billion dollar company that bills itself on developer productivity can't ship emulators for their cloud suite. If even for their own internal testing and release validation.
- yellowfool 4y ago
  Really, then again the bar is low in Azure.

skunkworker 4y ago

That’s incredible. I wasn’t expecting them to allow you to run AlloyDB on premise but this is a potential game changer for on-prem Postgres instances, especially if you are doing analytical queries on the same data set.

GabeWeiss_ 4y ago
That's how I've been thinking/talking about it. It should be a drop-in replacement for Postgres. Obviously the install process isn't as smooth yet, but, technical preview...
ellisv 4y ago
Eh, there are other options for this that are more mature and more likely to be supported in the long run. The Citus columnar store extension has been around for almost a decade.
- adapa 4y ago
  I looked at the citus columnar however lack of updates and deletes even occasionally kept me away.

emptysea 4y ago

I wonder how they make transactional workloads 2x faster vs normal Postgres The analytics workloads improvements seem pretty straightforward (or at least there is prior art like timescale)

ddorian43 4y ago
> I wonder how they make transactional workloads 2x faster vs normal Postgres There can be improvements like what OrioleDB is trying to do: https://github.com/orioledb/orioledb/ https://github.com/orioledb/orioledb/
ellisv 4y ago
My best guess is that transactional workloads are mostly improved by automatically adding the right indexes and parameter tuning. For example, OtterTune also advertises "2x on price/performance over unoptimized DBs".

stevesimmons 4y ago

> AlloyDB Omni provides full compatibility with PostgreSQL extensions

ellisv 4y ago
Makes sense since it's not in their managed infra

GiorgioG 4y ago

> The free developer edition of AlloyDB Omni is currently available as a technology preview. Check back for full product pricing information. Thanks, but no thanks!

wokwokwok 4y ago

> 3. Use of the Software. > 3.1. Use. You may authorize employees, agents, and subcontractors to use the Software in accordance with this Section 3, so long as you remain responsible for them. You may make a reasonable number of copies of the Software for back-up and archival purposes. > You acknowledge that the Software is a preview offering not intended for production environments, and you agree that you will only use the Software in non-production environments. Anyway, all that stuff in the docs is lovely, but if you just want to have a look: pip install gsutil gsutil cp -r gs://alloydb-omni-install/$(gsutil cat gs://alloydb-omni-install/latest) . The install scripts are only 16K, look at `installer/scripts/start_alloydb.sh` for more, but basically it just runs the two docker containers listed in https://cloud.google.com/alloydb/docs/omni/install#install https://cloud.google.com/alloydb/docs/omni/install#install Seems kind of weird, having a one-time install script to prep a machine (but only a specific type of machine!) that you then run a pair of docker containers on to me, honestly. Eventually consistent deployment states? eh. whatever...

mdaniel 4y ago
For those who want to skip the gsutil part, that bucket and its keys are public: curl -vJLO https://storage.googleapis.com/alloydb-omni-install/$(curl -fs https://storage.googleapis.com/alloydb-omni-install/latest)alloydb_omni_installer.tar.gz The `latest)alloy` is not a typo, the contents of the "latest" file ends with a slash and GCS doesn't tolerate `//` (presumably just like S3 wouldn't)

arecurrence 4y ago

Has Google contributed some of these changes back to Postgres? I went looking and even a search for "alloydb" in some postgres mailing lists return no results. It has some very exciting features, I was a bit surprised to see nobody talking about it.

Cockbrand 4y ago
If I understand correctly, AlloyDB is a Postgres compatible re-implementation, so there's no shared code. So this is a bit like Wine compared to Windows, only with the big corp re-implementing the OSS software instead of the other way round.
- ppg677 4y ago
  You can't really be 100% postgres compatible without borrowing a lot of implementation
  - aeyes 4y ago
    Query and storage layer are nicely separated in the Postgres codebase, you can absolutely rip out the complete backend if you really want it. There is a Postgres interface to Spanner which probably did that.
- GabeWeiss_ 4y ago
  OMG no, we didn't reimplement Postgres. Definitely started from the PG kernel and made modifications from there.
  - Cockbrand 4y ago
    I often reference Cunningham‘s Law, and I’m glad it’s in full effect :) Thank you for the clarification!
GabeWeiss_ 4y ago
Not as of yet, no. I've gently nudged the eng/product teams in this direction, but for now it's all being kept in-house.

lysecret 4y ago

Hugely disappointed by Alloydb. Price and performance for a write heavy workload where off the charts horrible. It didn’t even support disabling an instance and it did only storage autoscaling much worse overall experience than using Aurora we moved back to cloudsql. Does someone have good experience with Alloydb?

GabeWeiss_ 4y ago
I have many thoughts here...but let's start with: Do you have your process of how you tested performance? My guess is the eng team would love to look at what you tested.
- evntdrvn 4y ago
  thank you for the measured response rather than immediately becoming defensive :)
  - GabeWeiss_ 4y ago
    WHAT DO YOU MEAN?! RAAAAAGE!! ;) I mean, if someone legitimately finds workloads that aren't performing well, our engineering team SHOULD want to know about it, right?
    - gtirloni 4y ago
      Do they?
      - GabeWeiss_ 4y ago
        Okay, I can't speak for ALL engineering teams, but at the very least I know THIS engineering team absolutely does.
- lysecret 4y ago
  As a general feedback I have written it down here before: https://news.ycombinator.com/item?id=34304376 https://news.ycombinator.com/item?id=34304376 Sorry for being a bit negative but I was very excited for AlloyDB as a potential serverless offering for Postgres. Especially after having good experience with Aurora. And we ended up wasting a decent amount of resources migrating to and then away from it, thus my frustration. In our use-case we use Postgres essentially as a cache. We have many cloud runs (approx 100) writing a lot of data in parallel which we then on a schedule query in certain ways and put it to GCP. Its quite a bit of data, about 1 TB a day. It was a very CPU bound effort and to get acceptable performance we had to rely on the 8 core configuration, and we couldn't reduce the memory (from 64). We were getting similar performance with a 4 core 8gb Cloud Sql instance. Probably not the most representitive workload but this had exactly the opposite effect to what we wanted (e.g. much more wasted resources and less serverless). Also the fact you couldn't disable it was an absolute joke (we have prod dev and staging env and if we have to have a big always running DB for each of them, you can see how that is unacceptable). We are happy with CloudSql though.

fnordpiglet 4y ago

I see, not open source, ambiguous long term licensing, or even a commitment to a forever free. I did notice the release reads just like an internal AWS PR with Andy’s preferred structure… guess all those aws folks they recruited are making an impact.

brightball 4y ago

This looks really sharp. I wonder if Alloy gives some of the data compression benefits of columnar databases with that hybrid approach?

roncesvalles 4y ago
The columnar form is only in-memory and derived from the row-oriented form on disk. Like a denormalized cache of sorts.

deleted 4y ago

[deleted]

TheTaO 4y ago

Why would anyone use this as opposed to using Postgres? The value prop of run-anywhere applies to Postgres as well. I see column store and index advisor as the two features but if I don't need these, is there any reason?

GabeWeiss_ 4y ago
It's twice as fast as out of the box PG for most things, and up to 100x faster for reads, depending on what you're doing. So there's that. Also, from a manageability, on top of the index advisor, there's also vacuum management, so it will figure out when the best time to do the garbage cleanup while minimizing impact on performance.
- dx034 4y ago
  The 100x analytics seems due to columnar storage like with TimescaleDB, but how do you get improvements on the other things? Is it really faster on a database with good indexes?
  - GabeWeiss_ 4y ago
    It really is, yes. I can't go into a lot of detail on the "why" because it's not open source and the product team would murder me...but I highly encourage folks to try it for themselves. Nothing else convinced me until I did it for myself. :)
virogenesis 4y ago
also how does it compare to PGHero for index advising?
- GabeWeiss_ 4y ago
  We encourage folks to draw their own conclusions, but we have done comparisons and ours is more efficient and produces faster results. I only say test yourself because I can't give specific details of how/why, so don't take my word for it. :)

janpio 4y ago

Tried running it on the cloud dev environments I usually use (as I am on Windows myself). Gitpod: 2023-03-29 18:13:54.044 UTC: [alloydb_util.sh:90] FATAL: Docker service must be active to run AlloyDB Omni GitHub Codespaces (after increasing memory): 2023-03-29 18:37:55.236 UTC: [alloydb_util.sh:76] FATAL: AlloyDB Omni requires cgroups V2 to run. GitHub Actions: 2023-03-29 18:53:52.766 UTC: [alloydb_util.sh:44] AlloyDB requires at least 16GB of RAM to run. Only 7 GB available. Please increase available RAM and retry Not today it seems. Shame, sounded super interesting.

GabeWeiss_ 4y ago
Yeah, I know. Apologies. Stay tuned though! For the tech preview launch we really focused on a golden path (narrow). We'll be expanding compatibility as we march towards GA.

qeternity 4y ago

Is there any HA built into this? Performance claims look great.

GabeWeiss_ 4y ago
Not for Omni no. End-users are responsible for any HA/DR/Replication needs. Cloud AlloyDB has all of it built in, but since it was so tightly tied to the Cloud infrastructure it was simplified for Omni.

inscrutable 4y ago

This looks great! For the 2x general improvement, how much of that is due to setting/modifying postgres settings and adding good indexes, versus improvements to the code itself?

GabeWeiss_ 4y ago
Code improvements. Indices aren't taken into account at all since that's so workload dependent. Having said that, the index advisor can make finding the right indices to improve performance there more easily as well. Vacuum management handles figuring out the right time to do garbage collection for you, etc. None of that is part of that 2x improvement.
- inscrutable 4y ago
  That's impressive, surprised there's that much juice that could be squeezed without changing architecture. I ask because AWS Aurora is flattered in comparison to RDS due to e.g. some dumb oob pg settings that a lot of people don't change. From seeing the cloud alloydb I had imagined most of the improvements were due to the wal-shipping and cloud native aspects.

antman 4y ago

What does it mean in terms of license?

GabeWeiss_ 4y ago
We don't have specific details yet about licensing or pricing. Stay tuned as we get closer to GA. Having said that, we're pushing hard for a free usage tier and the product team isn't pushing back. :)

ethicalsmacker 4y ago

Maybe I'm old.. but everytime I read Alloy I think of the old modeling language out of MIT to prove a program's correctness. Gives me PTSD thinking of grad school. https://alloytools.org/ https://alloytools.org/

chadbr 4y ago

A quick demo: AlloyDB Omni Columnar Engine Fast Analytics Demo - YouTube https://www.youtube.com/watch?v=f_dvdKMq6og https://www.youtube.com/watch?v=f_dvdKMq6og

vira28 4y ago

Since we run Vanilla PG on on-prem, this is interesting. Will try once we get clarity on licensing. (Probably we can't use if the code is not open, so let's see)

GabeWeiss_ 4y ago
As a heads up, we're unlikely to make the codebase for it open. I might be able to convince the product team to open source some components of it (our GM has talked about this before in a couple articles), the whole thing won't be open sourced.
- yandie 4y ago
  What's the plan for the license for this? Especially path to production?
  - GabeWeiss_ 4y ago
    I'm assuming you mean "I start using this under a free dev license, and want to shift up to the paid production version"? And what that looks like? If that's the case, I don't know yet. That's literally being discussed/hammered as we speak, and we likely won't have a good answer until we're ready to go public preview (not sure on timeline right now, depends heavily on how the tech preview goes). This stage of the game is literally a "We built an awesome thing, please poke at it and tell us where it does/doesn't work!".

jefflynn 4y ago

Redrock Postgres - Runing Cloud Native Databse Anywhere Network Attached Tablespace https://doc.rockdata.net/admin/network-tablespace/ https://doc.rockdata.net/admin/network-tablespace/ PostgreSQL Multitenant https://doc.rockdata.net/features/multitenant/ https://doc.rockdata.net/features/multitenant/