4 ms·

Architecture decisions in Neon, a serverless Postgres service

55 points by shenli3514 3y ago

imslavko 3y ago
Thoroughly enjoyed the article since I have heard about Neon but never understood what it offers on the technical level over other PG-compatible projects. The article mentions that a consequence of separating storage from compute is that compute nodes cache hot pages in memory and load cold pages from object storage (like S3?) when needed. Does anyone know what are the consequences of this decision. In case of a query that touches multiple rarely used pages, would that incur high latency and ingress? How does that penalty compare to a vanilla postgres running on AWS and storing pages on EBS?
- nikita 3y ago
  (neon ceo) Yes you will have an S3 fetch latency for downloading a layer file for such data. However we don't page out aggressively so it's a very unlikely event. Most of the time you will be fetching from a locally attached SSD which is superior and more predictable than EBS. Where it works super well is the long tail of small and rarely used database - it pushes our costs way down. Another advantage is that if the whole region goes down the S3 copy survives. Finally if you separate storage and compute you can be serverless and scale to 0.
  - imslavko 3y ago
    Thanks Nikita!
Morelesshell 3y ago
I'm more curious how someone is allowed to have the time and money to build this? I would love to spend 1-3 years with other good people to just build good tech :(
- dsign 3y ago
  Business as usual: somebody funds it--not necessarily the developers--in exchange for a slice of the pie if the project is successful. If it will be succesfull, time will tell. But serverless postgres would be really sweet to have. Hell, AWS itself should be funding this project. I run my Postgres for hobby projects at home because AWS Aurora serverless postgres is very expensive and doesn't scale down to zero. I wouldn't mind paying a dollar or two for storage the months I don't use that project, and a little more when I actually need to query my databases. At work, my $$$$$$ employer pays $$-employees to poke at $$$-devs who forget to shut down an Aurora serverless Postgres.
- nikita 3y ago
  This is a very deliberate effort which is venture funded. It started with a deck three years ago and a team of myself, Heikki and Stas. Since it’s my second company (prev was singlestore) the early funding was relatively easy. Later funding was done based on our traction.
  - Morelesshell 3y ago
    Uj thanks for that insight. GZ :)
jmull 3y ago
I'm pretty bullish on this tech. A clean, low-friction separation of storage and compute for a top-tier database like Postgres is huge. I think it will take some time for the implications to sink in though, because it changes some assumptions about how to integrate a databases with a system that are so fundamental that we rarely think about it... (not to mention, it probably doesn't make sense to change existing systems, where the fundamental system decisions have already been made.) We tend to centralize the database in system design because they are complex and expensive to run, and you can amortize those costs the more you put into a single database. But of course, that also makes the central database bigger, more complex and more expensive. And once everything is in a central store, we tend to bind everything together (with joins, references, procs, etc. Each step in that direction tends to make sense on its own, but the end result is the "big ball of mud" architecture where everything is interconnected and there's no separation of concerns, which makes the system fragile and increasingly difficult to change. With this tech the pressure to centralize the database is relieved. E.g., if you have a situation where you are adding some new, logically distinct data to your system -- maybe just a few tables and maybe with a variable load, especially at first -- traditionally you probably would lean hard toward adding this to an existing central database since the overhead of spinning up and running a new one would be prohibitive. But now you can just make that a separate database -- spinning up a managed one is easy and you only need to pay for as much storage and compute as you actually need to use (which is really about the same as if you had put the tables in an existing central database anyway). Of course, systems would still tend to have significant central data, but that could be based on actual app concerns, not operational ones. Anyway, I'm loving this and can't wait to see the long term impact. You can have as many databases as logically makes senseseparate data logically without exploding the operational cost of your system. E.g. you can spin out a separate a database that has light (or even no load) most of the time yet can seamlessly scale to heavy load. And it can do this independent of how much data is in it, and you don't need to worry about it's affect on day-to-day data access in your system since you
- atombender 3y ago
  We already have it with YugabyteDB, which is open source (unlike Neon), but doesn't seem to be the success story it should have been. At least I've heard barely a peep about it. Maybe they're not so great at marketing. Yugabyte is architected similarly to other similar databases, with an SQL layer on top of a distributed, replicated key/value store based on RocksDB. The different being that the SQL layer is Postgres. Yugabyte tightly couples the two, though, though it seems like this is mostly for convenience, in that the "TServer" (tablet server) is managed by Yugabyte itself. I wonder if it could support an architecture more similar to TiDB, where the storage is independently scalable from compute. CockroachDB and TiDB are based on a similar design, but Yugabyte stands out as being a "real" Postgres. It even supports normal Postgres extensions via "CREATE EXTENSION".

55 points by shenli3514 3y ago

Thoroughly enjoyed the article since I have heard about Neon but never understood what it offers on the technical level over other PG-compatible projects. The article mentions that a consequence of separating storage from compute is that compute nodes cache hot pages in memory and load cold pages from object storage (like S3?) when needed. Does anyone know what are the consequences of this decision. In case of a query that touches multiple rarely used pages, would that incur high latency and ingress? How does that penalty compare to a vanilla postgres running on AWS and storing pages on EBS?

nikita 3y ago
(neon ceo) Yes you will have an S3 fetch latency for downloading a layer file for such data. However we don't page out aggressively so it's a very unlikely event. Most of the time you will be fetching from a locally attached SSD which is superior and more predictable than EBS. Where it works super well is the long tail of small and rarely used database - it pushes our costs way down. Another advantage is that if the whole region goes down the S3 copy survives. Finally if you separate storage and compute you can be serverless and scale to 0.
- imslavko 3y ago
  Thanks Nikita!

Morelesshell 3y ago

I'm more curious how someone is allowed to have the time and money to build this? I would love to spend 1-3 years with other good people to just build good tech :(

dsign 3y ago
Business as usual: somebody funds it--not necessarily the developers--in exchange for a slice of the pie if the project is successful. If it will be succesfull, time will tell. But serverless postgres would be really sweet to have. Hell, AWS itself should be funding this project. I run my Postgres for hobby projects at home because AWS Aurora serverless postgres is very expensive and doesn't scale down to zero. I wouldn't mind paying a dollar or two for storage the months I don't use that project, and a little more when I actually need to query my databases. At work, my $$$$$$ employer pays $$-employees to poke at $$$-devs who forget to shut down an Aurora serverless Postgres.
nikita 3y ago
This is a very deliberate effort which is venture funded. It started with a deck three years ago and a team of myself, Heikki and Stas. Since it’s my second company (prev was singlestore) the early funding was relatively easy. Later funding was done based on our traction.
- Morelesshell 3y ago
  Uj thanks for that insight. GZ :)

jmull 3y ago

I'm pretty bullish on this tech. A clean, low-friction separation of storage and compute for a top-tier database like Postgres is huge. I think it will take some time for the implications to sink in though, because it changes some assumptions about how to integrate a databases with a system that are so fundamental that we rarely think about it... (not to mention, it probably doesn't make sense to change existing systems, where the fundamental system decisions have already been made.) We tend to centralize the database in system design because they are complex and expensive to run, and you can amortize those costs the more you put into a single database. But of course, that also makes the central database bigger, more complex and more expensive. And once everything is in a central store, we tend to bind everything together (with joins, references, procs, etc. Each step in that direction tends to make sense on its own, but the end result is the "big ball of mud" architecture where everything is interconnected and there's no separation of concerns, which makes the system fragile and increasingly difficult to change. With this tech the pressure to centralize the database is relieved. E.g., if you have a situation where you are adding some new, logically distinct data to your system -- maybe just a few tables and maybe with a variable load, especially at first -- traditionally you probably would lean hard toward adding this to an existing central database since the overhead of spinning up and running a new one would be prohibitive. But now you can just make that a separate database -- spinning up a managed one is easy and you only need to pay for as much storage and compute as you actually need to use (which is really about the same as if you had put the tables in an existing central database anyway). Of course, systems would still tend to have significant central data, but that could be based on actual app concerns, not operational ones. Anyway, I'm loving this and can't wait to see the long term impact. You can have as many databases as logically makes senseseparate data logically without exploding the operational cost of your system. E.g. you can spin out a separate a database that has light (or even no load) most of the time yet can seamlessly scale to heavy load. And it can do this independent of how much data is in it, and you don't need to worry about it's affect on day-to-day data access in your system since you

atombender 3y ago
We already have it with YugabyteDB, which is open source (unlike Neon), but doesn't seem to be the success story it should have been. At least I've heard barely a peep about it. Maybe they're not so great at marketing. Yugabyte is architected similarly to other similar databases, with an SQL layer on top of a distributed, replicated key/value store based on RocksDB. The different being that the SQL layer is Postgres. Yugabyte tightly couples the two, though, though it seems like this is mostly for convenience, in that the "TServer" (tablet server) is managed by Yugabyte itself. I wonder if it could support an architecture more similar to TiDB, where the storage is independently scalable from compute. CockroachDB and TiDB are based on a similar design, but Yugabyte stands out as being a "real" Postgres. It even supports normal Postgres extensions via "CREATE EXTENSION".