5 ms·

Storage is the most surprisingly expensive part now. $1.50/GB is kind of a tough pill to swallow. We've been exploring the idea of shifting from Aurora to Neon.

by bilalq 3y ago

Storage is the most surprisingly expensive part now. $1.50/GB is kind of a tough pill to swallow. We've been exploring the idea of shifting from Aurora to Neon. With the recent pricing changes, our bill has exploded even with almost zero usage of compute time.

nikita 3y ago
We are currently lumping storage retention history (we store all the history for a period of time so you can time travel) and backups together. We are rethinking this, please stay tuned
- bilalq 3y ago
  Even if backups/history were taking up zero space, $1.50/GB seems really high for raw storage. The rest of the pricing seems reasonable to me. We're only around 100GB right now, but can see that ballooning up in the future and raising some concerns. By the way, I do want to say that branching is a game changer. The recent usability improvements like graphs/metrics and being able to reset a branch to the state of another one without affecting the endpoint is so nice. We previously had a messy script that took care of creating a new branch, moving endpoints, renaming, deleting, etc. More fine-grained permissions for users on projects would be my number one ask at this point, but overall, I really appreciate the improvements the Neon team has made in recent months.
  - throwaway5959 3y ago
    I’m confused, are we talking about object storage here? Like $0.023/GB in S3 storage cost?
    - est31 3y ago
      Neon keeps an NVME cache in the Pageservers, and it also keeps copies of the data in S3, one for the main storage, and one for the backup case. The data also gets stored on special replicated storage (Safekeepers). So it might be in 6 different places at the same time (3 safekeepers, 1 pageserver, 2 S3 buckets), details depending on the data's lifecycle through the system. This architecture delivers really good safety: Once your transaction commits, the data is already replicated across different AZs, and this is done without there being an S3 request each time. It also means that Neon can deliver features like branching. (Neon engineer)
      - leftcenterright 3y ago
        > So it might be in 6 different places at the same time (3 safekeepers, 1 pageserver, 2 S3 buckets), details depending on the data's lifecycle through the system. I hope it is encrypted on S3 at least? :)
        est31 3y ago
        Yes, it's encrypted at rest both on the NVME drives as well as on S3. https://neon.tech/docs/introduction/architecture-overview https://neon.tech/docs/introduction/architecture-overview There is no system of per-project encryption keys though.
  - tristan957 3y ago
    Permissions improvements have been duly noted by others, so expect that work to begin sometime in the near future. Reducing our COGS is very important, and also something that we will be working on. I would definitely reach out to our sales team for a more custom quote. (Neon engineer)
mj4e 3y ago
(Neon PM). If you're willing to discuss your project's specifics, contact me directly mike@neon.tech as I'd love to chat.
csomar 3y ago
Yeah, I am not sure what "scales to zero" really means. The storage costs + pre-paid compute could essentially buy you an AWS instance. Sure you do get things configured from the get-go but if you can get your way through Terraform that can work too. Either way, if you get big enough (will use more storage + compute), it makes sense to move. I'd be against such platforms as they make changes to Postgres that might make you depend on them.
- tristan957 3y ago
  One case where scale to zero can help is if your main branch scales to zero, that means more compute hours for your non-main branches. FWIW, Neon is really "just" Postgres. I can't think of anything Neon specific that would lock you in, other than that we don't support certain kinds of extensions yet. But if you were going to use those extensions, then you probably wouldn't have picked Neon in the first place. I also wouldn't consider that a lock-in. Do you have any examples that you might be aware of in Neon? Postgres compatibility is something that we take very seriously. (Neon employee)
GordonS 3y ago
This almost feels like how cloud providers charge excessive fees for excess bandwidth. Storage is cheap, even with replication, especially at Neon scale.

by bilalq 3y ago

nikita 3y ago

We are currently lumping storage retention history (we store all the history for a period of time so you can time travel) and backups together. We are rethinking this, please stay tuned

bilalq 3y ago
Even if backups/history were taking up zero space, $1.50/GB seems really high for raw storage. The rest of the pricing seems reasonable to me. We're only around 100GB right now, but can see that ballooning up in the future and raising some concerns. By the way, I do want to say that branching is a game changer. The recent usability improvements like graphs/metrics and being able to reset a branch to the state of another one without affecting the endpoint is so nice. We previously had a messy script that took care of creating a new branch, moving endpoints, renaming, deleting, etc. More fine-grained permissions for users on projects would be my number one ask at this point, but overall, I really appreciate the improvements the Neon team has made in recent months.
- throwaway5959 3y ago
  I’m confused, are we talking about object storage here? Like $0.023/GB in S3 storage cost?
  - est31 3y ago
    Neon keeps an NVME cache in the Pageservers, and it also keeps copies of the data in S3, one for the main storage, and one for the backup case. The data also gets stored on special replicated storage (Safekeepers). So it might be in 6 different places at the same time (3 safekeepers, 1 pageserver, 2 S3 buckets), details depending on the data's lifecycle through the system. This architecture delivers really good safety: Once your transaction commits, the data is already replicated across different AZs, and this is done without there being an S3 request each time. It also means that Neon can deliver features like branching. (Neon engineer)
    - leftcenterright 3y ago
      > So it might be in 6 different places at the same time (3 safekeepers, 1 pageserver, 2 S3 buckets), details depending on the data's lifecycle through the system. I hope it is encrypted on S3 at least? :)
      - est31 3y ago
        Yes, it's encrypted at rest both on the NVME drives as well as on S3. https://neon.tech/docs/introduction/architecture-overview https://neon.tech/docs/introduction/architecture-overview There is no system of per-project encryption keys though.
- tristan957 3y ago
  Permissions improvements have been duly noted by others, so expect that work to begin sometime in the near future. Reducing our COGS is very important, and also something that we will be working on. I would definitely reach out to our sales team for a more custom quote. (Neon engineer)

mj4e 3y ago

(Neon PM). If you're willing to discuss your project's specifics, contact me directly mike@neon.tech as I'd love to chat.

csomar 3y ago

Yeah, I am not sure what "scales to zero" really means. The storage costs + pre-paid compute could essentially buy you an AWS instance. Sure you do get things configured from the get-go but if you can get your way through Terraform that can work too. Either way, if you get big enough (will use more storage + compute), it makes sense to move. I'd be against such platforms as they make changes to Postgres that might make you depend on them.

tristan957 3y ago
One case where scale to zero can help is if your main branch scales to zero, that means more compute hours for your non-main branches. FWIW, Neon is really "just" Postgres. I can't think of anything Neon specific that would lock you in, other than that we don't support certain kinds of extensions yet. But if you were going to use those extensions, then you probably wouldn't have picked Neon in the first place. I also wouldn't consider that a lock-in. Do you have any examples that you might be aware of in Neon? Postgres compatibility is something that we take very seriously. (Neon employee)

GordonS 3y ago

This almost feels like how cloud providers charge excessive fees for excess bandwidth. Storage is cheap, even with replication, especially at Neon scale.