5 ms·

Database is a thing I never want to deal with at such low level. Such approach will be brittle, very brittle. Managed data storage with SLA guarantees is not ea

Database is a thing I never want to deal with at such low level. Such approach will be brittle, very brittle. Managed data storage with SLA guarantees is not easy and there are quite a few companies specializing on that for a reason. As a proof, check out fly.io forums. They are full with posts about suddenly broken Postgres instances.

craigkerstiens 4y ago
FWIW, managed providers can fully plug into fly just fine. Here's actually a profile of performance times [1] of Fly with various providers and configurations, along with a repo [2] to reproduce/create the same setup yourself. 1. https://webstack.dancroak.com/ https://webstack.dancroak.com/ 2. https://github.com/croaky/webstack https://github.com/croaky/webstack *Disclaimer I work at one of those fully managed database providers.
- Scarbutt 4y ago
  Why are the response times (api checks) so high?
  - lukeasrodgers 4y ago
    Yeah those response times are in the realm of “why would I even consider that”. If I am trying to tune my db queries to have p95 latency of 1ms (for example) there’s no way I would choose an architecture that then threw that all out the window with ~100ms network latency. Hopefully I am misunderstanding those numbers somehow.
  - sb8244 4y ago
    I saw this earlier and thoroughly didn't understand what is going on with that. I can't make any sense of why that would be so high. Feels like a better test would be for the healthcheck to time the query RTT and report that back. Completely remove the web request from the equation.
- samwillis 4y ago
  I would love it if Crunchy Bridge was available on Fly. Have you considered offering it alongside the other clouds you run on? I could see a Crunchy Data managed version of Fly Postgres's doing super well. Combining the best aspects of both your experience managing HA Postgres and backup, with the distributed read replicas from Fly.
- ngrilly 4y ago
  Would be interesting to see the request-response round trip time between the Go service and the SQL database, for each combination.
mrkurt 4y ago
You should definitely use a managed Postgres. So should many of the people in our forum. This is part of why we wrote this! That said, the vast majority of PG users on Fly have no issues. Most issues are the result of people trying to spend as little money as possible on their underlying infrastructure. When you have a 1GB disk, you'll run it out of space pretty quickly. You may not pay us for it, though, which is what they value most. We tell people to use CrunchyBridge _all_ the time. It's better than our Postgres for many people.
- aidenn0 4y ago
  Right; managed Postgres makes sense for businesses. For my hobby projects, the fact that I can't even get pricing without first creating an account is a signal to look somewhere else.
  - voltagex_ 4y ago
    I don't even see pricing for this when logged in, FWIW.
  - craigkerstiens 4y ago
    Pricing is available at https://www.crunchydata.com/pricing/calculator https://www.crunchydata.com/pricing/calculator. To add-on to that, we really liked the Fly pricing model where if you consumed less than X we wouldn't charge you. Recently we introduced a $10 a month plan and if you suspend the database while developing and your bill is under $5 you won't get charged.
    - aidenn0 4y ago
      Thanks I did find it after I commented; I had to follow two buttons from the front-page that were the same color as the background though ("Learn More" -> "Check out pricing" [which was a contrasting color] -> "View Pricing Breakdown") [edit] Also other managed DB hosts have similar anti-funnels for pricing, so I'm not trying to pick on Crunchy Data.
      - 4y ago
keehun 4y ago
I was recently just one of those affected by a random issue! I was made aware in the forums that maybe my VM had run out of space. That would've helped if an error message immediately alerted to my exhausted disk space. I realized that what I actually wanted was a managed provider and moved (back) to Heroku just for postgres. I recently moved out of there for my backend hosting & postgres to fly.io. My backend hosting stays on fly.io but will be putting my postgres in a managed platform.
- worthless-trash 4y ago
  Just to be clear, your code makes postgresql connections from fly's servers to heroku's servers, I'm a little surprised. I'm not a web programmer, more of a traditional systems programmer, I have some questions. I assume you use SSL, and these are not made on demand (connection pooling ?) On 'cold start' do you get massive latency ? What kind of latency is there between client and server ? What made you choose Heroku and not a specific managed postgresql service ? Did you try flys postgres service?
  - keehun 4y ago
    > Did you try flys postgres service? Yes. I wrote: "I was recently just one of those affected by a random issue!" I don't notice any measurable latency. Heroku exposes their postgres instances directly on the public web (secured with a strong user/pass combo of course), so there's no difference, networking-wise. There's no additional reverse proxy or some tunnel to route through. Heroku was the cheapest for the very small instance I needed. The connections are made on-demand and are insanely fast.
    - worthless-trash 4y ago
      Thanks for the answers I'm going to look into this method.

craigkerstiens 4y ago

FWIW, managed providers can fully plug into fly just fine. Here's actually a profile of performance times [1] of Fly with various providers and configurations, along with a repo [2] to reproduce/create the same setup yourself. 1. https://webstack.dancroak.com/ https://webstack.dancroak.com/ 2. https://github.com/croaky/webstack https://github.com/croaky/webstack *Disclaimer I work at one of those fully managed database providers.

Scarbutt 4y ago
Why are the response times (api checks) so high?
- lukeasrodgers 4y ago
  Yeah those response times are in the realm of “why would I even consider that”. If I am trying to tune my db queries to have p95 latency of 1ms (for example) there’s no way I would choose an architecture that then threw that all out the window with ~100ms network latency. Hopefully I am misunderstanding those numbers somehow.
- sb8244 4y ago
  I saw this earlier and thoroughly didn't understand what is going on with that. I can't make any sense of why that would be so high. Feels like a better test would be for the healthcheck to time the query RTT and report that back. Completely remove the web request from the equation.
samwillis 4y ago
I would love it if Crunchy Bridge was available on Fly. Have you considered offering it alongside the other clouds you run on? I could see a Crunchy Data managed version of Fly Postgres's doing super well. Combining the best aspects of both your experience managing HA Postgres and backup, with the distributed read replicas from Fly.
ngrilly 4y ago
Would be interesting to see the request-response round trip time between the Go service and the SQL database, for each combination.

mrkurt 4y ago

You should definitely use a managed Postgres. So should many of the people in our forum. This is part of why we wrote this! That said, the vast majority of PG users on Fly have no issues. Most issues are the result of people trying to spend as little money as possible on their underlying infrastructure. When you have a 1GB disk, you'll run it out of space pretty quickly. You may not pay us for it, though, which is what they value most. We tell people to use CrunchyBridge _all_ the time. It's better than our Postgres for many people.

aidenn0 4y ago
Right; managed Postgres makes sense for businesses. For my hobby projects, the fact that I can't even get pricing without first creating an account is a signal to look somewhere else.
- voltagex_ 4y ago
  I don't even see pricing for this when logged in, FWIW.
- craigkerstiens 4y ago
  Pricing is available at https://www.crunchydata.com/pricing/calculator https://www.crunchydata.com/pricing/calculator. To add-on to that, we really liked the Fly pricing model where if you consumed less than X we wouldn't charge you. Recently we introduced a $10 a month plan and if you suspend the database while developing and your bill is under $5 you won't get charged.
  - aidenn0 4y ago
    Thanks I did find it after I commented; I had to follow two buttons from the front-page that were the same color as the background though ("Learn More" -> "Check out pricing" [which was a contrasting color] -> "View Pricing Breakdown") [edit] Also other managed DB hosts have similar anti-funnels for pricing, so I'm not trying to pick on Crunchy Data.
    - 4y ago

keehun 4y ago

I was recently just one of those affected by a random issue! I was made aware in the forums that maybe my VM had run out of space. That would've helped if an error message immediately alerted to my exhausted disk space. I realized that what I actually wanted was a managed provider and moved (back) to Heroku just for postgres. I recently moved out of there for my backend hosting & postgres to fly.io. My backend hosting stays on fly.io but will be putting my postgres in a managed platform.

worthless-trash 4y ago
Just to be clear, your code makes postgresql connections from fly's servers to heroku's servers, I'm a little surprised. I'm not a web programmer, more of a traditional systems programmer, I have some questions. I assume you use SSL, and these are not made on demand (connection pooling ?) On 'cold start' do you get massive latency ? What kind of latency is there between client and server ? What made you choose Heroku and not a specific managed postgresql service ? Did you try flys postgres service?
- keehun 4y ago
  > Did you try flys postgres service? Yes. I wrote: "I was recently just one of those affected by a random issue!" I don't notice any measurable latency. Heroku exposes their postgres instances directly on the public web (secured with a strong user/pass combo of course), so there's no difference, networking-wise. There's no additional reverse proxy or some tunnel to route through. Heroku was the cheapest for the very small instance I needed. The connections are made on-demand and are insanely fast.
  - worthless-trash 4y ago
    Thanks for the answers I'm going to look into this method.