12 ms·

Does it have basic functioning other stuff? I am shocked at how our production usage of Fly has gone. Even basic stuff as support not being able to just... look

by k8svet 3y ago

Does it have basic functioning other stuff? I am shocked at how our production usage of Fly has gone. Even basic stuff as support not being able to just... look up internal platform issues. Cryptic/non-existent error messages. I'm not impressed. It feels like it's compelling to those scared of or ignorant of Kubernetes. I thought I was over Kubernetes, but Fly makes me miss it.

deleted 3y ago
[deleted]
chachra 3y ago
Been on it 7 months, 0 issues. Feel like you're alone on this potentially.
- deleted 3y ago
  [deleted]
- weird-eye-issue 3y ago
  Alone? Every thread about Fly has complaints about reliability and people complain about it on Twitter too
  - nixgeek 3y ago
    That hasn’t been my experience with Fly but I’m sorry to hear it seems to be others :(
  - chachra 3y ago
    ok possibly not alone, maybe the issues happened before I started using them extensively. I've had ~no downtime that affects me in 7 months. I do wish they had some features I need, but their support and responses are top notch. And I've lost much less hair and time than I would going full-blown AWS or another cloud provider.
  - jokethrowaway 3y ago
    To be fair most hosting providers come with plenty of public complaints about downtime. The big ones do way better, the best one is AWS, then GC and last Azure. They cost stupid money though. Digital ocean has been terrible for me, some regions just go down every month and I lose thousands of requests, increasing my churn rate. Fly.io had tons of weird issues but it got better in the last months. It's still very incomplete in terms of functionality and figuring out how to deploy the first time is a massive pain. My plan is to add Hetzner and load balance with bunnycdn across DO and H
  - loloquwowndueo 3y ago
    Every thread on the Internet about any product or service has complaints.
    - weird-eye-issue 3y ago
      Not to this extent, it has always stood out to me in particular
    - throwaway220033 3y ago
      [flagged]
      - loloquwowndueo 3y ago
        Wow that devolved into aggression pretty quickly.
        throwaway220033 3y ago
        I'd correct that as a corrupt definition of "aggression".
      - matthewmacleod 3y ago
        This isn't the place for silly, unjustified personal attacks. Stop it.
    - weird-eye-issue 3y ago
      Actually here is a good example: Cloudflare. Sure people complain a ton about privacy but I haven't seen a single complaint about the reliability of Cloudflare Workers or similar product in the dozens of threads I've seen on HN
  - jrockway 3y ago
    It's hard to tell how meaningful the reviews are. I have used AWS, GCP, DigialOcean, and Linode throughout my career. Every single one of these, through no fault of myself or my team, messed up and caused downtime. Like, you can get most SRE types in a room to laugh if you blurt out "us-east-1", because it's known to be so unreliable. And yet, it's where every Fortune 500 puts every service; we laugh about the reliability and it's literally powering the economy just fine. So yes, a lot of people on HN complain about fly's reliability. fly posts to HN a lot and gives them the opportunity. Is it actually meaningful compared to the alternatives? It's very hard to tell.
    - tptacek 3y ago
      Hoo boy. First: this is 100% a "live by the sword, die by the sword" situation for us. We're as aware as anybody about our weird HN darling status (this is a post from two months ago, about an announcement from many months ago, that spent like 12 hours plastered to the front page; we have no idea why it hit today, and it actually stepped on another thing we wanted to post today so don't think we secretly orchestrated any of this!). We've allowed ourselves to be ultra-visible here, and threads like this are natural consequence. Moreover: a lot of this criticism is well warranted! I can cough up a litany of mitigating factors (the guy who stored his database in ephemeral instance storage instead of a volume, for instance), but I mean, come on. The single most highly upvoted and trafficked thing we've ever written was a post a year ago owning up to reliability issues on the platform. People have definitely had issues! A fun cop-out answer here is to note all the times people compare us to AWS or Cloudflare, as if we were a hyperscaler public cloud. More fun still is to search HN for stories about us-east-1. We certainly do that to self-sooth internally! And: also? If your only consideration for picking a place to host an application is platform reliability? You're hosting on AWS anyways. But it's still a cop-out. So I guess I'd sum all this up as: we've picked a hard problem to work on. Things are mathematically guaranteed to go wrong even if we're perfect, and we are not that. People should take criticisms of us on these threads seriously. We do. This is a tough crowd (the threads, if not the vote scores on our blog post) and there's value in that. Over the last year, and through this upcoming year, staffing for infra reliability has been the single biggest driver of hiring at Fly.io, I think that's the right call, and I think the fact that we occasionally get mauled on threads is part of what enabled us to make that call. (Ordinarily I'd shut up about this stuff and let the thread die out itself, but some dearly loved user of ours took a stand and said they'd never had any problems on us, which: you can imagine the "ohhhhh nooooooo" montage that took place in my brain when I read that someone had essentially dared the thread to come up with times when we'd sucked for some user, so I guess all bets are off. Go easy on Xe, though: they really are just an ultra-helpful uncynical person, and kind of walked into a buzzsaw here).
- uo21tp5hoyg 3y ago
  https://community.fly.io/t/reliability-its-not-great/11253 https://community.fly.io/t/reliability-its-not-great/11253
- heeton 3y ago
  Not alone, I’ve been part of two teams who have evaluated fly and hit weird reliability or stability issues, deemed it not ready yet.
- yawnxyz 3y ago
  this is what I thought, until once I spent two days to publish a new, trivial code change to my Fly.io hosted API — it just wouldn't update! And every time I tried to re-publish it'd give me a slightly different error. When it works, it's brilliant. The problem is that it hasn't worked too well in the last few months.
xena 3y ago
Can you email the first two letters of my username at fly.io with more details? I'd love to find out what you've been having trouble with so I can help make the situation better any way I can. Thanks!
- bongobingo1 3y ago
  Another support.flycombinator.com classic.
  - azinman2 3y ago
    Would you rather them be unresponsive?
    - lostemptations5 3y ago
      It's HN -- if the company proved responsive it might invalidate his OP and everyone who band wagons on it.
  - zmgsabst 3y ago
    Why would you care about customer problems if they don’t embarrass you in public? /s
    - keeganpoppen 3y ago
      the only thing easier than them responding in this thread is someone making this comment in this thread…
  - rob 3y ago
    [flagged]
    - joshi4 3y ago
      It seems to me that your comment is personally targeting OP and I think that is quite out of line.
pech0rin 3y ago
Yep they have terrible reliability and support. Couldn’t deploy for 2 days once and they actually told me to use another company. Unmanaged dbs masquerading as managed. Random downtime. I could go on but it’s not a production ready service and I moved off of it months ago.
- biorach 3y ago
  > Unmanaged dbs masquerading as managed Are you talking about fly postgres? Because I use it and feel they've been pretty clear that it's unmanaged.
  - andy_ppp 3y ago
    Seriously! That's crazy. I need to setup terraform and move to AWS before launching I guess.
    - biorach 3y ago
      > Seriously! That's crazy huh? it does what it says on the tin. nothing crazy about it. They spell out for you in detail what they offer: https://fly.io/docs/postgres/getting-started/what-you-should-know/ https://fly.io/docs/postgres/getting-started/what-you-should... And suggest external providers if you need managed postgres: https://fly.io/docs/postgres/getting-started/what-you-should-know/#recommended-external-providers https://fly.io/docs/postgres/getting-started/what-you-should...
      - andy_ppp 3y ago
        I was shocked because I didn't realise it wasn't managed. Even Digital Ocean offer managed Postgres. If you are offering a service like Fly I think the database should be managed personally, the whole point of Fly.io is to provide abstractions to make production simpler. Do you think the type of user who is using fly.io is interested in or capable of managing their own Postgres database? I'd rather just trust RDS or another provider.
awestroke 3y ago
I have run several services on Fly for almost a year now, have not had any issues.
parhamn 3y ago
I was hoping to migrate to Fly.io and during my testing I found that simple deploys would drop connections for a few seconds during a deploy switch over. Try a `watch -n 2 curl <serviceipv4>` during a deploy to see for yourself (try any one of the the strategies documented including blue-green). I wonder how many people know this? When I tested it I was hoping for at worst early termination of old connections with no dropped new connections and at best I expected them to gracefully wait for old connections to finish. But nope, just a full downtime switch over every time. But then when you think about the network topology described in their blog posts, you realize theres no way it could've been done correctly to begin with. It's very rare for me to comment negatively on a service but that fact that this was the case paired with the way support acted like we were crazy when we sent video evidence of it definitely irked me for infrastructure company standards. Wouldn't recommend it outside of toy applications now. > It feels like it's compelling to those scared of or ignorant of Kubernetes I've written pretty large deployment systems for kubernetes. This isn't it. Theres a real space for heroku-like deploys done properly and no one is really doing it well (or at least without ridiculously thin or expensive compute resources)
- asaddhamani 3y ago
  Yeah I had a similar experience where I got builds frozen for a couple days, such that I was not able to release any updates. When I emailed their support, I got an auto-response asking me to post in the forum. Pretty much all hosts are expected to offer a ticket system even for their unmanaged services if its a problem on their side. I just moved over all my stuff to Render.com, it's more expensive, but its been reliable so far.
  - loloquwowndueo 3y ago
    The first (pinned) post in the fly.io forum explains it: https://community.fly.io/t/fly-io-support-community-vs-email-read-this-first/9962 https://community.fly.io/t/fly-io-support-community-vs-email...
    - malfist 3y ago
morgante 3y ago
Unfortunately this is a pretty common story. Half the people I know who adopted Fly migrated off it. I was very excited about Fly originally, and built an entire orchestrator on top of Fly machines—until they had a multi-day outage where it took days to even get a response. Kubernetes can be complex, but at least that complexity is (a) controllable and (b) fairly well-trodden.
- loloquwowndueo 3y ago
  Fly.io is not comparable to Kubernetes. It’s a bit like comparing AWS to Terraform. Or to clarify your comment, Kubernetes on which cloud? Amazon? google? Linode?
  - jrockway 3y ago
    Kubernetes on AWS, GCP, and Linode are all controllable and well-trodden. I definitely understand the comparison between Kubernetes and fly. You have couple apps that are totally unrelated, managed by separate teams, and you want to figure out how you can avoid the two teams duplicating effort. One option is to use something like fly.io, where you get a command line you run to build your project and push the binary to a server. Another option is to self-host infrastructure like Kubernetes, and eventually get that down to one command to build and push (or have your CI system do it). The end result that organizations are aiming for are similar; developers code the code and then the code runs in production. Frankly, a lot of toil and human effort is spent on this task, and everyone is aiming to get it to take less effort. fly.io is an approach. Kubernetes is an approach. Terraform on AWS is an approach.
    - loloquwowndueo 3y ago
      Maybe you’re comparing flyctl with Kubernetes? That’d be a slightly more valid comparison albeit flyctl is much less ambitious by choice and design. That said, using flyctl to orchestrate your deployments is not the only way to Fly. Example: https://fly.io/blog/fks/ https://fly.io/blog/fks/
  - morgante 3y ago
throwaway220033 3y ago
I switched to Kamal and Hetzner. It's the sweet spot.
rmbyrro 3y ago
I find it amazing how much bad vibes fly.io gets here. It looks worse than AWS or Azure to me. Never used the service, but based on what I hear, I'll never try...
M_Joverflow 3y ago
I don't see any reason use Fly. There are more mature, feature-richer and cheaper solutions out there. We have the big complex ones like AWS, Azure, GCP, the easier more affordable all rounder like DO, Render, the hosting plattforms like Vercel, Heroku and finally the biggest bang for your money barebones like Hetzner. Why should I choose Fly? How come they are so prominent on hackernews? Are they backed by VC and get their default 400 upvotes by backers? I get the impression that Fly posts here are kind of sponsored.

by k8svet 3y ago

deleted 3y ago

[deleted]

chachra 3y ago

Been on it 7 months, 0 issues. Feel like you're alone on this potentially.

deleted 3y ago
[deleted]
weird-eye-issue 3y ago
Alone? Every thread about Fly has complaints about reliability and people complain about it on Twitter too
- nixgeek 3y ago
  That hasn’t been my experience with Fly but I’m sorry to hear it seems to be others :(
- chachra 3y ago
  ok possibly not alone, maybe the issues happened before I started using them extensively. I've had ~no downtime that affects me in 7 months. I do wish they had some features I need, but their support and responses are top notch. And I've lost much less hair and time than I would going full-blown AWS or another cloud provider.
- jokethrowaway 3y ago
  To be fair most hosting providers come with plenty of public complaints about downtime. The big ones do way better, the best one is AWS, then GC and last Azure. They cost stupid money though. Digital ocean has been terrible for me, some regions just go down every month and I lose thousands of requests, increasing my churn rate. Fly.io had tons of weird issues but it got better in the last months. It's still very incomplete in terms of functionality and figuring out how to deploy the first time is a massive pain. My plan is to add Hetzner and load balance with bunnycdn across DO and H
- loloquwowndueo 3y ago
  Every thread on the Internet about any product or service has complaints.
  - weird-eye-issue 3y ago
    Not to this extent, it has always stood out to me in particular
  - throwaway220033 3y ago
    [flagged]
    - loloquwowndueo 3y ago
      Wow that devolved into aggression pretty quickly.
      - throwaway220033 3y ago
        I'd correct that as a corrupt definition of "aggression".
    - matthewmacleod 3y ago
      This isn't the place for silly, unjustified personal attacks. Stop it.
  - weird-eye-issue 3y ago
    Actually here is a good example: Cloudflare. Sure people complain a ton about privacy but I haven't seen a single complaint about the reliability of Cloudflare Workers or similar product in the dozens of threads I've seen on HN
- jrockway 3y ago
  It's hard to tell how meaningful the reviews are. I have used AWS, GCP, DigialOcean, and Linode throughout my career. Every single one of these, through no fault of myself or my team, messed up and caused downtime. Like, you can get most SRE types in a room to laugh if you blurt out "us-east-1", because it's known to be so unreliable. And yet, it's where every Fortune 500 puts every service; we laugh about the reliability and it's literally powering the economy just fine. So yes, a lot of people on HN complain about fly's reliability. fly posts to HN a lot and gives them the opportunity. Is it actually meaningful compared to the alternatives? It's very hard to tell.
  - tptacek 3y ago
    Hoo boy. First: this is 100% a "live by the sword, die by the sword" situation for us. We're as aware as anybody about our weird HN darling status (this is a post from two months ago, about an announcement from many months ago, that spent like 12 hours plastered to the front page; we have no idea why it hit today, and it actually stepped on another thing we wanted to post today so don't think we secretly orchestrated any of this!). We've allowed ourselves to be ultra-visible here, and threads like this are natural consequence. Moreover: a lot of this criticism is well warranted! I can cough up a litany of mitigating factors (the guy who stored his database in ephemeral instance storage instead of a volume, for instance), but I mean, come on. The single most highly upvoted and trafficked thing we've ever written was a post a year ago owning up to reliability issues on the platform. People have definitely had issues! A fun cop-out answer here is to note all the times people compare us to AWS or Cloudflare, as if we were a hyperscaler public cloud. More fun still is to search HN for stories about us-east-1. We certainly do that to self-sooth internally! And: also? If your only consideration for picking a place to host an application is platform reliability? You're hosting on AWS anyways. But it's still a cop-out. So I guess I'd sum all this up as: we've picked a hard problem to work on. Things are mathematically guaranteed to go wrong even if we're perfect, and we are not that. People should take criticisms of us on these threads seriously. We do. This is a tough crowd (the threads, if not the vote scores on our blog post) and there's value in that. Over the last year, and through this upcoming year, staffing for infra reliability has been the single biggest driver of hiring at Fly.io, I think that's the right call, and I think the fact that we occasionally get mauled on threads is part of what enabled us to make that call. (Ordinarily I'd shut up about this stuff and let the thread die out itself, but some dearly loved user of ours took a stand and said they'd never had any problems on us, which: you can imagine the "ohhhhh nooooooo" montage that took place in my brain when I read that someone had essentially dared the thread to come up with times when we'd sucked for some user, so I guess all bets are off. Go easy on Xe, though: they really are just an ultra-helpful uncynical person, and kind of walked into a buzzsaw here).
uo21tp5hoyg 3y ago
https://community.fly.io/t/reliability-its-not-great/11253 https://community.fly.io/t/reliability-its-not-great/11253
heeton 3y ago
Not alone, I’ve been part of two teams who have evaluated fly and hit weird reliability or stability issues, deemed it not ready yet.
yawnxyz 3y ago
this is what I thought, until once I spent two days to publish a new, trivial code change to my Fly.io hosted API — it just wouldn't update! And every time I tried to re-publish it'd give me a slightly different error. When it works, it's brilliant. The problem is that it hasn't worked too well in the last few months.

xena 3y ago

Can you email the first two letters of my username at fly.io with more details? I'd love to find out what you've been having trouble with so I can help make the situation better any way I can. Thanks!

bongobingo1 3y ago
Another support.flycombinator.com classic.
- azinman2 3y ago
  Would you rather them be unresponsive?
  - lostemptations5 3y ago
    It's HN -- if the company proved responsive it might invalidate his OP and everyone who band wagons on it.
- zmgsabst 3y ago
  Why would you care about customer problems if they don’t embarrass you in public? /s
  - keeganpoppen 3y ago
    the only thing easier than them responding in this thread is someone making this comment in this thread…
- rob 3y ago
  [flagged]
  - joshi4 3y ago
    It seems to me that your comment is personally targeting OP and I think that is quite out of line.

pech0rin 3y ago

Yep they have terrible reliability and support. Couldn’t deploy for 2 days once and they actually told me to use another company. Unmanaged dbs masquerading as managed. Random downtime. I could go on but it’s not a production ready service and I moved off of it months ago.

biorach 3y ago
> Unmanaged dbs masquerading as managed Are you talking about fly postgres? Because I use it and feel they've been pretty clear that it's unmanaged.
- andy_ppp 3y ago
  Seriously! That's crazy. I need to setup terraform and move to AWS before launching I guess.
  - biorach 3y ago
    > Seriously! That's crazy huh? it does what it says on the tin. nothing crazy about it. They spell out for you in detail what they offer: https://fly.io/docs/postgres/getting-started/what-you-should-know/ https://fly.io/docs/postgres/getting-started/what-you-should... And suggest external providers if you need managed postgres: https://fly.io/docs/postgres/getting-started/what-you-should-know/#recommended-external-providers https://fly.io/docs/postgres/getting-started/what-you-should...
    - andy_ppp 3y ago
      I was shocked because I didn't realise it wasn't managed. Even Digital Ocean offer managed Postgres. If you are offering a service like Fly I think the database should be managed personally, the whole point of Fly.io is to provide abstractions to make production simpler. Do you think the type of user who is using fly.io is interested in or capable of managing their own Postgres database? I'd rather just trust RDS or another provider.

awestroke 3y ago

I have run several services on Fly for almost a year now, have not had any issues.

parhamn 3y ago

I was hoping to migrate to Fly.io and during my testing I found that simple deploys would drop connections for a few seconds during a deploy switch over. Try a `watch -n 2 curl <serviceipv4>` during a deploy to see for yourself (try any one of the the strategies documented including blue-green). I wonder how many people know this? When I tested it I was hoping for at worst early termination of old connections with no dropped new connections and at best I expected them to gracefully wait for old connections to finish. But nope, just a full downtime switch over every time. But then when you think about the network topology described in their blog posts, you realize theres no way it could've been done correctly to begin with. It's very rare for me to comment negatively on a service but that fact that this was the case paired with the way support acted like we were crazy when we sent video evidence of it definitely irked me for infrastructure company standards. Wouldn't recommend it outside of toy applications now. > It feels like it's compelling to those scared of or ignorant of Kubernetes I've written pretty large deployment systems for kubernetes. This isn't it. Theres a real space for heroku-like deploys done properly and no one is really doing it well (or at least without ridiculously thin or expensive compute resources)

asaddhamani 3y ago
Yeah I had a similar experience where I got builds frozen for a couple days, such that I was not able to release any updates. When I emailed their support, I got an auto-response asking me to post in the forum. Pretty much all hosts are expected to offer a ticket system even for their unmanaged services if its a problem on their side. I just moved over all my stuff to Render.com, it's more expensive, but its been reliable so far.
- loloquwowndueo 3y ago
  The first (pinned) post in the fly.io forum explains it: https://community.fly.io/t/fly-io-support-community-vs-email-read-this-first/9962 https://community.fly.io/t/fly-io-support-community-vs-email...
  - malfist 3y ago

morgante 3y ago

Unfortunately this is a pretty common story. Half the people I know who adopted Fly migrated off it. I was very excited about Fly originally, and built an entire orchestrator on top of Fly machines—until they had a multi-day outage where it took days to even get a response. Kubernetes can be complex, but at least that complexity is (a) controllable and (b) fairly well-trodden.

loloquwowndueo 3y ago
Fly.io is not comparable to Kubernetes. It’s a bit like comparing AWS to Terraform. Or to clarify your comment, Kubernetes on which cloud? Amazon? google? Linode?
- jrockway 3y ago
  Kubernetes on AWS, GCP, and Linode are all controllable and well-trodden. I definitely understand the comparison between Kubernetes and fly. You have couple apps that are totally unrelated, managed by separate teams, and you want to figure out how you can avoid the two teams duplicating effort. One option is to use something like fly.io, where you get a command line you run to build your project and push the binary to a server. Another option is to self-host infrastructure like Kubernetes, and eventually get that down to one command to build and push (or have your CI system do it). The end result that organizations are aiming for are similar; developers code the code and then the code runs in production. Frankly, a lot of toil and human effort is spent on this task, and everyone is aiming to get it to take less effort. fly.io is an approach. Kubernetes is an approach. Terraform on AWS is an approach.
  - loloquwowndueo 3y ago
    Maybe you’re comparing flyctl with Kubernetes? That’d be a slightly more valid comparison albeit flyctl is much less ambitious by choice and design. That said, using flyctl to orchestrate your deployments is not the only way to Fly. Example: https://fly.io/blog/fks/ https://fly.io/blog/fks/
- morgante 3y ago

throwaway220033 3y ago

I switched to Kamal and Hetzner. It's the sweet spot.

rmbyrro 3y ago

I find it amazing how much bad vibes fly.io gets here. It looks worse than AWS or Azure to me. Never used the service, but based on what I hear, I'll never try...

M_Joverflow 3y ago

I don't see any reason use Fly. There are more mature, feature-richer and cheaper solutions out there. We have the big complex ones like AWS, Azure, GCP, the easier more affordable all rounder like DO, Render, the hosting plattforms like Vercel, Heroku and finally the biggest bang for your money barebones like Hetzner. Why should I choose Fly? How come they are so prominent on hackernews? Are they backed by VC and get their default 400 upvotes by backers? I get the impression that Fly posts here are kind of sponsored.