4 ms·

Neon: The serverless open source alternative to AWS Aurora Postgres

49 points by kiyanwang 4y ago

hdjjhhvvhga 4y ago
Thank you for the description of the software in the submission title, I wish everyone was doing that.
wizwit999 4y ago
No offense to the author, and looks like a cool initiative. But isn't the whole point of things like Aurora serverless that it's a managed service that you don't have to worry about. Who would self host it? I see this a lot w/ open source projects... the first question is where's the cloud version.
- kwerk 4y ago
  The repo mentions a “free tier” and their website describes managed service.
- nikita 4y ago
  The cloud version is at neon.tech. The foundation of serverless is separation of storage and compute. This cloud native storage is proprietary at AWS Aurora (and architecture is old). Neons storage is open source
- orev 4y ago
  The problem with using managed services is you dump a ton of time making a solution that uses their custom APIs, and then you can be held for ransom to them because they can keep raising the price and you have nowhere to go. Having alternatives that are API compatible is a great way to ensure competition and protect against this.
  - nikita 4y ago
    This is precisely right. Another point is that separation of storage and compute is decisively the right architecture for the cloud. Large cloud providers build their own implementation of this architecture which fragments the ecosystem. Having a canonical open source implementation just makes sense.
  - smitty1e 4y ago
    One specific issue is data exfiltration. Cloud providers can turn into a Hotel California when the tera- and petabyte costs for liberating one's assets are calculated.
- scosman 4y ago
  This has a few big improvements over Aurora. Aurora has scaleable disk, but is still 1 big master Postgres instance (and optional read replicas), with all the drawbacks that comes with in scaling speed, cost and failure modes. This appears to be a distributed many-peer-node Postgres, with an all new abstracted distributed storage layer. This allows a lot of neat things with scaling/cost, and innovation on the storage layer (like their writeout to S3). Basically this is true serverless postgres if the claims are correct, and that's amazing.
endisneigh 4y ago
Is there something like this where you can self host, and also combine with managed hosts for bursts? I guess the issue is most Postgres databases don’t scale down to zero.
- nikita 4y ago
  We are exploring those things. You can do it both ways. Self host storage and let us manage compute. This way all data is in your infrastructure. We can also have you manage compute this way you can have more control over you customer. Right now we manage both for simplicity of the user experience.
flakiness 4y ago
FYI: This serverless in term of "if someone runs a large cluster for the customers", not the serverless you can run on existing serverless infrastructure like AWS Lambda. It seems this is backed by a company https://neon.tech/ https://neon.tech/ so we in theory have that someone who runs the cluster. So this is probably more aligned with the services like planetscale or cockroach lab.
- nikita 4y ago
  Yes. Also if you run your app on AWS Lambda it's a lot easier to integrate with other serverless infra.
jvolkman 4y ago
Discussion from 7 days ago (330+ comments): https://news.ycombinator.com/item?id=31536827 https://news.ycombinator.com/item?id=31536827

49 points by kiyanwang 4y ago

Thank you for the description of the software in the submission title, I wish everyone was doing that.

No offense to the author, and looks like a cool initiative. But isn't the whole point of things like Aurora serverless that it's a managed service that you don't have to worry about. Who would self host it? I see this a lot w/ open source projects... the first question is where's the cloud version.

kwerk 4y ago
The repo mentions a “free tier” and their website describes managed service.
nikita 4y ago
The cloud version is at neon.tech. The foundation of serverless is separation of storage and compute. This cloud native storage is proprietary at AWS Aurora (and architecture is old). Neons storage is open source
orev 4y ago
The problem with using managed services is you dump a ton of time making a solution that uses their custom APIs, and then you can be held for ransom to them because they can keep raising the price and you have nowhere to go. Having alternatives that are API compatible is a great way to ensure competition and protect against this.
- nikita 4y ago
  This is precisely right. Another point is that separation of storage and compute is decisively the right architecture for the cloud. Large cloud providers build their own implementation of this architecture which fragments the ecosystem. Having a canonical open source implementation just makes sense.
- smitty1e 4y ago
  One specific issue is data exfiltration. Cloud providers can turn into a Hotel California when the tera- and petabyte costs for liberating one's assets are calculated.
scosman 4y ago
This has a few big improvements over Aurora. Aurora has scaleable disk, but is still 1 big master Postgres instance (and optional read replicas), with all the drawbacks that comes with in scaling speed, cost and failure modes. This appears to be a distributed many-peer-node Postgres, with an all new abstracted distributed storage layer. This allows a lot of neat things with scaling/cost, and innovation on the storage layer (like their writeout to S3). Basically this is true serverless postgres if the claims are correct, and that's amazing.

endisneigh 4y ago

Is there something like this where you can self host, and also combine with managed hosts for bursts? I guess the issue is most Postgres databases don’t scale down to zero.

nikita 4y ago
We are exploring those things. You can do it both ways. Self host storage and let us manage compute. This way all data is in your infrastructure. We can also have you manage compute this way you can have more control over you customer. Right now we manage both for simplicity of the user experience.

flakiness 4y ago

FYI: This serverless in term of "if someone runs a large cluster for the customers", not the serverless you can run on existing serverless infrastructure like AWS Lambda. It seems this is backed by a company https://neon.tech/ https://neon.tech/ so we in theory have that someone who runs the cluster. So this is probably more aligned with the services like planetscale or cockroach lab.

nikita 4y ago
Yes. Also if you run your app on AWS Lambda it's a lot easier to integrate with other serverless infra.

jvolkman 4y ago

Discussion from 7 days ago (330+ comments): https://news.ycombinator.com/item?id=31536827 https://news.ycombinator.com/item?id=31536827