4 ms·

Not too familiar with Neon other than the basics - its premise is that you use S3 as bottomless storage for Postgres and it’s otherwise the same as standard Pos

Not too familiar with Neon other than the basics - its premise is that you use S3 as bottomless storage for Postgres and it’s otherwise the same as standard Postgres right? And this is all open source? Why are people paying? Can’t you use a cloud provider and have them host this for you?

refset 1y ago
> you use S3 as bottomless storage for Postgres [...] Why are people paying? It's vastly more complicated to do this efficiently than you might imagine. Postgres' internal architecture is built around a very different set of assumptions (pages, WAL, local disk etc.) than what the S3 API offers.
- amazingamazing 1y ago
  I get that, but my understanding is that they opened sourced this itself, no?
  - refset 1y ago
    It's not clear to me that the _entire_ Neon stack is OSS and available to self-host (though they do share a lot of OSS code, which is great), and in any case, it's not currently supported/documented beyond some "local development" instructions, e.g. "We do not officially support use of autoscaling externally" [0] > Can’t you use a cloud provider and have them host this for you? If it really is all OSS, then I guess the moat is the impressive execution of this team. [0] https://github.com/neondatabase/autoscaling https://github.com/neondatabase/autoscaling
    - tristan957 1y ago
      The only component which is not currently open source is the control plane.
    - felixrydberg 1y ago
      There are some sections of the code that is close-source. Like the Console, but the devs are very active in their Discord. There is a small community trying to self-host, and from what I have seen gotten most of it working. There isn't an easy way to start it up for a more junior developer, but if you want to self-host it you are able to since all "components" are available
- aitchnyu 1y ago
  Would "essential internal abstraction, with the introduction of read stream APIs" evolve into full storage abstraction? https://pganalyze.com/blog/5mins-postgres-17-streaming-io https://pganalyze.com/blog/5mins-postgres-17-streaming-io
Icathian 1y ago
Hosting and operating the autoscaling of the various services (compute, pageserver, safekeeper, storage broker) that it takes to make all that work is complex enough that most folks would rather not. Same as any other "managed X" service.
yencabulator 1y ago
Neon is only scale-to-zero from the point of view of the tenants; whoever runs the software, must keep nodes running, including triple-replicated WAL storage.
- deleted 1y ago
  [deleted]

refset 1y ago

> you use S3 as bottomless storage for Postgres [...] Why are people paying? It's vastly more complicated to do this efficiently than you might imagine. Postgres' internal architecture is built around a very different set of assumptions (pages, WAL, local disk etc.) than what the S3 API offers.

amazingamazing 1y ago
I get that, but my understanding is that they opened sourced this itself, no?
- refset 1y ago
  It's not clear to me that the _entire_ Neon stack is OSS and available to self-host (though they do share a lot of OSS code, which is great), and in any case, it's not currently supported/documented beyond some "local development" instructions, e.g. "We do not officially support use of autoscaling externally" [0] > Can’t you use a cloud provider and have them host this for you? If it really is all OSS, then I guess the moat is the impressive execution of this team. [0] https://github.com/neondatabase/autoscaling https://github.com/neondatabase/autoscaling
  - tristan957 1y ago
    The only component which is not currently open source is the control plane.
  - felixrydberg 1y ago
    There are some sections of the code that is close-source. Like the Console, but the devs are very active in their Discord. There is a small community trying to self-host, and from what I have seen gotten most of it working. There isn't an easy way to start it up for a more junior developer, but if you want to self-host it you are able to since all "components" are available
aitchnyu 1y ago
Would "essential internal abstraction, with the introduction of read stream APIs" evolve into full storage abstraction? https://pganalyze.com/blog/5mins-postgres-17-streaming-io https://pganalyze.com/blog/5mins-postgres-17-streaming-io

Icathian 1y ago

Hosting and operating the autoscaling of the various services (compute, pageserver, safekeeper, storage broker) that it takes to make all that work is complex enough that most folks would rather not. Same as any other "managed X" service.

yencabulator 1y ago

Neon is only scale-to-zero from the point of view of the tenants; whoever runs the software, must keep nodes running, including triple-replicated WAL storage.

deleted 1y ago
[deleted]