3 ms·

I'm kind of interested in why we can't make a better database with all of our modern technology. Postgres is a fantastic workhorse, but it was also released in

I'm kind of interested in why we can't make a better database with all of our modern technology. Postgres is a fantastic workhorse, but it was also released in the late 80s. Who, who among you will create the database of the future... And not lock it behind bizarro licenses which force me to use telemetry.

trescenzi 2y ago
I guess I’d ask why is something having been first released in the late 80s, or any decade, as positive or negative? Some things are still used because they solve the problems people have. Some things are still used just because of industry capture. I’m not honestly sure where I’d put Postgres. Are there specific things you’d want from a modern database?
- 999900000999 2y ago
  Relating to the article, better scaling. Saying run it on a bigger box is a very brute force way to optimize an application. While they come up with some other tricks here, that's ultimately what's scaling postgres means. If I imagine a better database, it would have native support for scaling, a postgres compatible data layer as well as first party support for NoSQL( JSONB columns don't cut it since if you have simultaneous writes unpredictable behavior tends to occur). It needs to also have a permissible license
  - throwaway7783 2y ago
    Can you please expand on the JSONB unpredictable behavior? We are about to embark on a journey to move some of our data from MongoDB to postgres (with some JSONB). While we don't have significant concurrent writes to a table, would be very helpful to understand the issues
    - 999900000999 2y ago
      https://github.com/shrinerb/shrine/discussions/665 https://github.com/shrinerb/shrine/discussions/665 I've never personally encountered this, but I've seen other HN contributors mention it. https://news.ycombinator.com/item?id=43189535 https://news.ycombinator.com/item?id=43189535 From what I can tell, unlike mongo, some postgres queries will try to update the entire JSONB data object vs a single field. This can lead to race conditions.
      - throwaway7783 2y ago
        Ah, thanks. The first link seems something specific to Shrine. The bottomline is concurrent updates to different parts of JSONB need row level locking for correct behavior in Postgresql. This is not an important issue for us. Thank you for the pointers
        ahoka 2y ago
        You can trivially do it without locking using a version column.
    - leonidasv 2y ago
      You may be interested in FerretDB[1]. [1] https://www.ferretdb.com/ https://www.ferretdb.com/
  - aprdm 2y ago
    Why is it brute force and why is it bad ?
  - Reefersleep 2y ago
    What's an example of non-brute force scaling?
    - 999900000999 2y ago
      Something like a queuing system, re-architecting the database to handle a higher load without just throwing more machines on it. A cache.
    - HDThoreaun 2y ago
      The problem with postgres scaling is that you have to have a single master which means horizontal scaling really only gives you more reads and a failover. Eventually you wont be able to find a server big enough to handle all the writes, and if you get enough reads with even a small number of writes single master setups fall over. Distributed computing gets complicated very quickly but the gist here is basically that you need to be able to have multiple instances that can accept writes. Lots of literature on this but good starting points imo would be the paxos paper https://lamport.azurewebsites.net/pubs/time-clocks.pdf https://lamport.azurewebsites.net/pubs/time-clocks.pdf and dynamo db paper https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf https://www.allthingsdistributed.com/files/amazon-dynamo-sos...
  - mike_hearn 2y ago
    What does permissible license mean? If you mean open source, no such database exists AFAIK. If you mean you can run it locally for free for dev purposes, on prem without telemetry etc, then Oracle is clearly the best option. Compared to Postgres, Oracle DB: • Scales horizontally with full SQL and transactional consistency. That means both write and read masters, not replicas - you can use database nodes with storage smaller than your database, or with no storage, and they are fully ACID. • Has full transactional MQ support, along with many other features. • Can scale elastically. • Doesn't require vacuuming or have problems with XID wraparound. These are all Postgresisms that don't affect Oracle due to its better MVCC engine design. • Has first party support for NoSQL that resolves your concern (see SODA and JSON duality views). I should note that I have a COI because I work part time at Oracle Labs (and this post is my own yadda yadda), but you're asking why does no such database exist and whether anyone can make one. The database you're asking for not only exists but is one of the world's most popular databases. It's also actually quite cheap thanks to the elastic scaling. Spec out a managed Oracle DB in Oracle's cloud using the default elastic scaling option and you'll find it's cheaper than an Amazon Postgres RDS for similar max specs!
    - 999900000999 2y ago
      Does Oracle offer a Firebase like application/layer with authentication and stuff. Can you get me some Oracle cloud credits ? First positive thing I’ve ever heard about an oracle project !
      - mike_hearn 2y ago
        Oracle Cloud (called OCI) has an always-free offering, you don't need credits. You can just sign up and use some small quantity of resources for nothing indefinitely. That includes a managed Oracle database: https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/autonomous-always-free.html https://docs.oracle.com/en/cloud/paas/autonomous-database/se... There's also some sort of startup credits program with a brochure here, apparently you can just fill out a form and get some credits with an option to apply for more. But I don't know much about that. I've used the always-free programme for some personal stuff and it worked fine so I never needed to think about credits. https://www.oracle.com/a/ocom/docs/free-cloud.pdf https://www.oracle.com/a/ocom/docs/free-cloud.pdf I have to admit I'm not really familiar with Firebase, I thought that was some managed service for mobile apps, but Oracle DB comes with some stuff that sounds similar. And Oracle Cloud is an AWS-style cloud, it has a ton of high level services for things. ORDS is a REST binding layer that lets you export tables, views, stored procedures and NoSQL JSON document stores over HTTP without writing a middleman server yourself. You can drive it directly from the browser. ORDS supports OAuth2 or can be integrated with custom auth schemes from what I understand. I've not used ORDS myself yet but probably will in the near future. https://www.oracle.com/database/technologies/appdev/rest.html https://www.oracle.com/database/technologies/appdev/rest.htm... Firebase IIRC when it first launched was known for push streaming of changes. Oracle DB lets you subscribe to the results of SQL queries and get push notifications when they change, either directly via driver callbacks or into a message queue for async processing later. It's pretty easy to hook such notifications up to web sockets or SSE or similar, in fact I've done that in my current project. There's also a thing called APEX which is a bundled visual low-code app builder. I've never used it but I've used apps built with it, and it must be quite flexible as they all had a lot of features and looked very different. You can tell you're using an APEX app because they have a lot of colons in the URLs for some reason. Here's a random example of one from outside of Oracle that exports a database of dubious scientific research papers: https://dbrech.irit.fr/pls/apex/f?p=9999:1 https://dbrech.irit.fr/pls/apex/f?p=9999:1:::::: I'm not holding it up as a great example, there are probably better examples out there, it's just a one that came to mind that's public and I used before.
DoctorOW 2y ago
Have you looked at CockroachDB? PostgreSQL compatibility with modern comforts (e.g. easy cloud deployments, horizontal scaling, memory safe language)
- rednafi 2y ago
  Came here to say this. Cockroach solves the sharding issue by adopting consistent hashing-based data distribution, as described in the Dynamo paper. However, their cloud solution is a bit expensive to get started with.
  - I_am_tiberius 2y ago
    Does CockroachdB already support ltree?
    - rednafi 2y ago
      It doesn’t. It doesn’t support most of the Postgres extensions. We got away with it because we weren’t doing anything beyond vanilla Postgres.
HighlandSpring 2y ago
There are "better" databases but they're better given some particular definition that may not be relevant to your needs. If SQL/the relational model and ACID semantics is what you need then postgres is simply the best in class. The fact it dates back to the 80s is probably an advantage (requirement?) when it comes to solving a problem really well
wmf 2y ago
Anyone who creates a better database is going to want to get paid for it which either means DBaaS or those weird licenses.
thinkingtoilet 2y ago
Postgres 17.4 was released last month. Show some respect to the devs.
HDThoreaun 2y ago
Postgres is the most worked on database in the world right now. Its original release date doesnt mean work stopped. No-sql was a new thing a decade-ish ago but most companies probably dont need it. New data platforms focused on scaling like snowflake and cockroach have come too but again for most use cases postgres is better.
moltar 2y ago
Look at AWS presentation/talk about Aurora DSQL [1] It’s a Postgres facade. But everything beyond that is a complete reimagining and a rewrite to scale independently. I personally think it’s going to eat a lot of market share when it solves some remaining limitations. [1] https://youtu.be/huGmR_mi5dQ?si=ALw4XjdDJBxkZWRv https://youtu.be/huGmR_mi5dQ?si=ALw4XjdDJBxkZWRv

trescenzi 2y ago

I guess I’d ask why is something having been first released in the late 80s, or any decade, as positive or negative? Some things are still used because they solve the problems people have. Some things are still used just because of industry capture. I’m not honestly sure where I’d put Postgres. Are there specific things you’d want from a modern database?

999900000999 2y ago
Relating to the article, better scaling. Saying run it on a bigger box is a very brute force way to optimize an application. While they come up with some other tricks here, that's ultimately what's scaling postgres means. If I imagine a better database, it would have native support for scaling, a postgres compatible data layer as well as first party support for NoSQL( JSONB columns don't cut it since if you have simultaneous writes unpredictable behavior tends to occur). It needs to also have a permissible license
- throwaway7783 2y ago
  Can you please expand on the JSONB unpredictable behavior? We are about to embark on a journey to move some of our data from MongoDB to postgres (with some JSONB). While we don't have significant concurrent writes to a table, would be very helpful to understand the issues
  - 999900000999 2y ago
    https://github.com/shrinerb/shrine/discussions/665 https://github.com/shrinerb/shrine/discussions/665 I've never personally encountered this, but I've seen other HN contributors mention it. https://news.ycombinator.com/item?id=43189535 https://news.ycombinator.com/item?id=43189535 From what I can tell, unlike mongo, some postgres queries will try to update the entire JSONB data object vs a single field. This can lead to race conditions.
    - throwaway7783 2y ago
      Ah, thanks. The first link seems something specific to Shrine. The bottomline is concurrent updates to different parts of JSONB need row level locking for correct behavior in Postgresql. This is not an important issue for us. Thank you for the pointers
      - ahoka 2y ago
        You can trivially do it without locking using a version column.
  - leonidasv 2y ago
    You may be interested in FerretDB[1]. [1] https://www.ferretdb.com/ https://www.ferretdb.com/
- aprdm 2y ago
  Why is it brute force and why is it bad ?
- Reefersleep 2y ago
  What's an example of non-brute force scaling?
  - 999900000999 2y ago
    Something like a queuing system, re-architecting the database to handle a higher load without just throwing more machines on it. A cache.
  - HDThoreaun 2y ago
    The problem with postgres scaling is that you have to have a single master which means horizontal scaling really only gives you more reads and a failover. Eventually you wont be able to find a server big enough to handle all the writes, and if you get enough reads with even a small number of writes single master setups fall over. Distributed computing gets complicated very quickly but the gist here is basically that you need to be able to have multiple instances that can accept writes. Lots of literature on this but good starting points imo would be the paxos paper https://lamport.azurewebsites.net/pubs/time-clocks.pdf https://lamport.azurewebsites.net/pubs/time-clocks.pdf and dynamo db paper https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf https://www.allthingsdistributed.com/files/amazon-dynamo-sos...
- mike_hearn 2y ago
  What does permissible license mean? If you mean open source, no such database exists AFAIK. If you mean you can run it locally for free for dev purposes, on prem without telemetry etc, then Oracle is clearly the best option. Compared to Postgres, Oracle DB: • Scales horizontally with full SQL and transactional consistency. That means both write and read masters, not replicas - you can use database nodes with storage smaller than your database, or with no storage, and they are fully ACID. • Has full transactional MQ support, along with many other features. • Can scale elastically. • Doesn't require vacuuming or have problems with XID wraparound. These are all Postgresisms that don't affect Oracle due to its better MVCC engine design. • Has first party support for NoSQL that resolves your concern (see SODA and JSON duality views). I should note that I have a COI because I work part time at Oracle Labs (and this post is my own yadda yadda), but you're asking why does no such database exist and whether anyone can make one. The database you're asking for not only exists but is one of the world's most popular databases. It's also actually quite cheap thanks to the elastic scaling. Spec out a managed Oracle DB in Oracle's cloud using the default elastic scaling option and you'll find it's cheaper than an Amazon Postgres RDS for similar max specs!
  - 999900000999 2y ago
    Does Oracle offer a Firebase like application/layer with authentication and stuff. Can you get me some Oracle cloud credits ? First positive thing I’ve ever heard about an oracle project !
    - mike_hearn 2y ago
      Oracle Cloud (called OCI) has an always-free offering, you don't need credits. You can just sign up and use some small quantity of resources for nothing indefinitely. That includes a managed Oracle database: https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/autonomous-always-free.html https://docs.oracle.com/en/cloud/paas/autonomous-database/se... There's also some sort of startup credits program with a brochure here, apparently you can just fill out a form and get some credits with an option to apply for more. But I don't know much about that. I've used the always-free programme for some personal stuff and it worked fine so I never needed to think about credits. https://www.oracle.com/a/ocom/docs/free-cloud.pdf https://www.oracle.com/a/ocom/docs/free-cloud.pdf I have to admit I'm not really familiar with Firebase, I thought that was some managed service for mobile apps, but Oracle DB comes with some stuff that sounds similar. And Oracle Cloud is an AWS-style cloud, it has a ton of high level services for things. ORDS is a REST binding layer that lets you export tables, views, stored procedures and NoSQL JSON document stores over HTTP without writing a middleman server yourself. You can drive it directly from the browser. ORDS supports OAuth2 or can be integrated with custom auth schemes from what I understand. I've not used ORDS myself yet but probably will in the near future. https://www.oracle.com/database/technologies/appdev/rest.html https://www.oracle.com/database/technologies/appdev/rest.htm... Firebase IIRC when it first launched was known for push streaming of changes. Oracle DB lets you subscribe to the results of SQL queries and get push notifications when they change, either directly via driver callbacks or into a message queue for async processing later. It's pretty easy to hook such notifications up to web sockets or SSE or similar, in fact I've done that in my current project. There's also a thing called APEX which is a bundled visual low-code app builder. I've never used it but I've used apps built with it, and it must be quite flexible as they all had a lot of features and looked very different. You can tell you're using an APEX app because they have a lot of colons in the URLs for some reason. Here's a random example of one from outside of Oracle that exports a database of dubious scientific research papers: https://dbrech.irit.fr/pls/apex/f?p=9999:1 https://dbrech.irit.fr/pls/apex/f?p=9999:1:::::: I'm not holding it up as a great example, there are probably better examples out there, it's just a one that came to mind that's public and I used before.

DoctorOW 2y ago

Have you looked at CockroachDB? PostgreSQL compatibility with modern comforts (e.g. easy cloud deployments, horizontal scaling, memory safe language)

rednafi 2y ago
Came here to say this. Cockroach solves the sharding issue by adopting consistent hashing-based data distribution, as described in the Dynamo paper. However, their cloud solution is a bit expensive to get started with.
- I_am_tiberius 2y ago
  Does CockroachdB already support ltree?
  - rednafi 2y ago
    It doesn’t. It doesn’t support most of the Postgres extensions. We got away with it because we weren’t doing anything beyond vanilla Postgres.

HighlandSpring 2y ago

There are "better" databases but they're better given some particular definition that may not be relevant to your needs. If SQL/the relational model and ACID semantics is what you need then postgres is simply the best in class. The fact it dates back to the 80s is probably an advantage (requirement?) when it comes to solving a problem really well

wmf 2y ago

Anyone who creates a better database is going to want to get paid for it which either means DBaaS or those weird licenses.

thinkingtoilet 2y ago

Postgres 17.4 was released last month. Show some respect to the devs.

HDThoreaun 2y ago

Postgres is the most worked on database in the world right now. Its original release date doesnt mean work stopped. No-sql was a new thing a decade-ish ago but most companies probably dont need it. New data platforms focused on scaling like snowflake and cockroach have come too but again for most use cases postgres is better.

moltar 2y ago

Look at AWS presentation/talk about Aurora DSQL [1] It’s a Postgres facade. But everything beyond that is a complete reimagining and a rewrite to scale independently. I personally think it’s going to eat a lot of market share when it solves some remaining limitations. [1] https://youtu.be/huGmR_mi5dQ?si=ALw4XjdDJBxkZWRv https://youtu.be/huGmR_mi5dQ?si=ALw4XjdDJBxkZWRv