12 ms·

My thoughts about Fly.io (so far) and other newish technology I'm getting into

130 points by hartleybrody 4y ago

tlarkworthy 4y ago
Cloud Run is an alternative to fly.io that scales to zero so it can be much cheaper but with added cold starts.
- bobnamob 4y ago
  Isn’t the whole appeal of fly that they’re geolocated near your users, reducing latency so you can use fancy serverside rendering stacks like Phoenix? Cold start latency kinda ruins that no?
  - aenis 4y ago
    Latency on cloud run is not an issue, in my experience (been using it around 4 years now, on a fairly large scale). Its generally fast as is, and if you want to pre-allocate a minimum # of instances, cold starts are less of a problem (basically only if you are suddenly stampeded by a spike in traffic). But the whole smart routing and localized storage concepts are left for you to implement. You can have a bunch of cloud run services behind an ELB that does geo-proximity based routing but firestore is region-bound, and cloud spanner can be very expensive. Not saying there are no workarounds but it seems to me fly.io offers a much lower cost of entry here.
  - tlarkworthy 4y ago
    No cloud Run instance supports 1000 concurrent users and stays active for 15 min after no traffic, so only a small subset of users should ever hit a cold start.
- aenis 4y ago
  But, to get a cloud run based setup: - you need an elb - and this is not cheap (if you have an efficient backend, the elb will dwarf compute costs) - no persistent volumes, and you are encouraged to use gcs or firestore - each region requires a new deployment. No big deal but certainly not super easy to automate, esp. given the need to run behind an ELB (which you need on gcp to have a WAF) - google sdks for some languages suck big time. Most of python sdk is not async friendly, unbelievable as it seems. I do use cloud run for projects big and small, and rather like it, but its hardly a competitor to fly.io imho.
  - tlarkworthy 4y ago
    Yeah the load balancer bit sucks, I forgot about that. I use terraform for multi region deploys but yeah, the load balancer is a major cost.
csmpltn 4y ago
An unrelated, yet honest question. There have been many posts hitting the HN frontpage regarding fly.io recently. Is it healthy to have so much content about a single PAAS platform showing up here so often now?
- telotortium 4y ago
  For better or worse, fly.io has as a principal tptacek, who's at the top of the HN leaderboard and so has built up a lot of goodwill here.
  - tut-urut-utut 4y ago
    This actually makes me wonder if people here generally pay attention, who posted what? Does that influence actual upvote status? I never look at the person name when replying or voting, only the content. For example, I remembered this tptacek not because I remembered his posts, but because they get frequently mentioned in other people posts.
    - Multicomp 4y ago
      > Does that influence actual upvote status? It didn't for me. I also don't look at who posted links. But fly.io is leaning into a concept I want to use and want to become more of a thing, and that's using SQLite for more web scale applications. Because there's web scale of Google and there's web scale of the rest of us, and I'm sick of paying the operational overhead of having a full database cluster when the app doesn't utilize its features any more than it would a SQLite db. But in the past everyone turned their nose up at SQLite because they were cool and you were dirty and gross if you wanted to simplify things. Don't you know we just need horizontal scaling because any second now we're going to get more than 10 write requests per second? But fly.io leaning into litestream for replicating those databases is a thumbs up to using simple boring technology (SQLite), while still getting 80% of the benefits of doing containers hosted in a cloud platform.
    - snowwrestler 4y ago
      A lot of people on HN do pay attention to who is posting. This is why Cloudflare is generally beloved here as well: the principles show up in the comments whenever it’s mentioned.
Dave3of5 4y ago
A little note about read replicas and problems I've discovered. It's often the case in code that you write a value to the DB then immediately read it back often in the form of a different query somewhere else. If you are setup to do some kind of a round robin read from the read replicas you can often get a different read from what you wrote as the value hasn't replicated to your read replicas yet. The solution is to use the write endpoint when reading after a write. He says that here but just wanted to point out that it can happen inside an api and cause real issues with data.
- danpalmer 4y ago
  This depends on the database and consistency level it’s enforcing. You can often configure databases to require an ack from the replicas before it returns, so that you’ll be able to read your writes. This obviously has a trade off with speed. Some databases are cleverer about this. Things like Spanner and FoundationDB work differently so as to be fast to both read and write, but they’re much more complex to operate and use. There is another quick trick though… if a client performs a write, set a bit in their session that causes all of their reads to come from the primary database for a short period, maybe a few seconds, just enough to cover the replication latency. This is a hack. It’s got a lot of downsides, but it’s a quick way to patch the problem if truly necessary.
  - manmal 4y ago
    I think that's an interesting idea. Here's a post that benchmarked multi-region Postgres (Elixir/Phoenix on fly.io): https://nathanwillson.com/blog/posts/2021-09-25-fly-multi-db/#3-configure-phoenix-elixir--but-really-ecto- https://nathanwillson.com/blog/posts/2021-09-25-fly-multi-db... According to the post, for some users (residing in Japan, and the primary instance being located in Amsterdam), a query could take ~200ms (median). If multiple queries are performed for each request, that could mean 1 second or more per API call - not so great if that's the case for multiple seconds after each write. I think this would eventually lead to putting more code in stored procedures, begging the question: why not use a distributed DB like Fauna in the first place? Alternatively, the replication problem could be accounted for in the app itself. E.g. the SPA or the edge instance could retry reads following a write until the change from the primary instance has propagated, and up until then pretend that everything went fine. In case a write isn't replicated within 10 seconds or so, show an error to the user and let them retry the write action. This could lead to duplicate entries, but I'd estimate the chance for that to be quite low.
julianbuse 4y ago
I think some things need to be built out some more, like their postgres and adding more storage after creation, but in general it's really enjoyable to use. The pricing seems fair, and their blog is intersting and fun to read.
- tpetry 4y ago
  Completely agree. Their offering for running containers is great, but nobody wants to maintain a database. They should add DBaaS on their architecture with automatic backups etc.
  - jamesmcintyre 4y ago
    I totally agree with this! This would really add so much value to fly.io. And if they don't want to allocate resources to this right now I wonder if they could work with DBaaS providers like Planet Scale or FaunaDB where they'd wireguard their nearest instances into the fly.io network, add cli integrations/automations that'd link to their respective dashboards, etc. I plan to use fly.io + planetscale and I'm hoping to still get low latency between those two services but it's no where near the low latency cloudflare can achieve with their new edge redis/db offerings (or the fly.io db at edge strategies) but after looking into fly.io's db strategies I really feel hesitant to take on that level of devops/additional-engineering when something like planetscale provides so much value out of the box. Hope the fly.io team has something in the works either way! (And I'd love if they chime in with any input here in terms of performance between fly.io and existing DBaaS providers that are regionally replicated by default.)
marban 4y ago
I found render.com more convincing than fly.io (which looked more like a beta product with a prime-time landing page), with both of them not being anywhere close to making me jump from GCP.
pw 4y ago
The end of this article raises the issue of whether Fly.io’s USP, deploying app servers close to your users, is useful for run of the mill web apps. And as much as I like Fly.io and the people associated with it, I’ve wondered this myself. It just seems like serving to US customers from any major US data center is generally fast enough. And I think this might even be true for the world of HTML-over-the-wire web stuff, which Fly.io seems to investing heavily in. No doubts there are plenty of more niche uses (if I were serving users internationally, I’d probably use Fly.io), but the use case just doesn’t seem as broad as the Heroku/PaaS comparisons make it out to be.
- dx034 4y ago
  Not only that, having one location for a world-wide user base is usually enough. You can optimize much more through rendering speed, blocking requests etc than by being closer to your user. And even if your page becomes really popular, 3 locations (Europe, US, East Asia) are enough to be <200ms to any user in the world. And it keeps your setup and cost much lower.
  - mrkurt 4y ago
    This pretty much fits our definition of "deploy app servers close to your users". One region works just fine for some apps. Some are worth going to three.
davidkuennen 4y ago
Do many companies actually need databases geolocated near users? I'm working on big and small projects/companies and that has never been any concern of ours. I always imagined it to be something only the very very big players care about. And as a big player I would usually bet on a big partner like AWS, GCP, Azure. Or am I missing something?
- sph 4y ago
  Web Scale was first a meme, than an ideal everyone pushes towards even when deploying their small scale blog. Who knows what'll happen if suddenly you get a million concurrent users tomorrow? Better scale it geographically and put it behind a CDN today. Look at those generous free tiers. Like you said, it's mostly snake oil except for very big players.
  - pid-1 4y ago
    I'd say using stuff like Netlify or GH Pages for static sites is worth it even if you have zero traffic. They legit are much easier to use than setting up your own VPS.
  - fauigerzigerk 4y ago
    This isn't quite the same thing as trying to scale like Google though. Low latency is very important for usability regardless of how many users you serve. How easy it is to achieve depends more on the geographical distribution of your users than on their number. If my app has a handful of users that are split between the US and Europe or Asia, and the app is 90% reads, then the distributed DB approach of fly.io or Cloudflare makes a lot of sense. It also adds considerable complexity though, so it's obviously a tradeoff.
- kitbrennan 4y ago
  Our business has an API that can be used for displaying dynamic information at point of sale (i.e. dynamic in that it cannot be cached and will need a DB call). While we encourage our customers to try and use us asynchronously, we have a number of enterprises that don't and therefore demand incredibly fast response times with low latency. They pay us accordingly, so as a result we have geolocated databases (in our case though, we are using AWS Aurora replication).
bijay6779 4y ago
nicoburns 4y ago
> But despite how much I want to learn the fly.io platform – it has been a bit tricky for me wrap my head around a good use-case for this type of distributed hosting service. Worth noting that you don't have to use the distributed aspect. I have my site hosted on a single one of a fly.io's smallest instances (which one can get 3 of for free), and even like this the performance is excellent (50ms response times), and it doesn't have the problem of spinning down when not in use like Heroku's free tier. It's nice to at least get a choice of regions. For example, the company I work for (not hosted on fly.io currently) only has customers in the UK and Ireland. So it's would be to be able to pop our servers there with a simple config setting.
- petercooper 4y ago
  Same. I'm really impressed with the experience on there now that I finally spent a day trying it out. The geodistribution stuff had no interest to me so I'd avoided them till now, but it's really the underlying tooling and experience that has won me over.
- hartleybrody 4y ago
  This is an excellent point. While their main value prop seems to be "servers closer to your users" you could also just use them as a drop-in replacement for something like heroku and just use one region to simplify the mental model, pricing and orchestration.
kif 4y ago
> But despite how much I want to learn the fly.io platform – it has been a bit tricky for me wrap my head around a good use-case for this type of distributed hosting service. The distributed features are there for when you need them – I don't think you have to use them. Or am I missing something?
dfee 4y ago
I’ve got a deploy running on Fly.io, but I didn’t go with the buildpack option; instead I’m pushing a locally built docker image (buildpacks don’t support pnpm). One big miss, though, is you’ll still need a database and s3, so I’m not sure if I totally understand the value.
dom__inic 4y ago
Unrelated to this topic, but I think you should apply this one line of CSS to your stylesheet - it improves your text aesthetics and readability :) html { -webkit-font-smoothing: "antialiased" }
- alx__ 4y ago
  First off, avoid changing that property. It doesn't improve things 10 years later, still relevant - https://usabilitypost.com/2012/11/05/stop-fixing-font-smoothing/ https://usabilitypost.com/2012/11/05/stop-fixing-font-smooth... Secondly, the issue may be the font being used. I don't recognize it, but probably not optimized for modern web screens. Or a "converted" typeface originally designed for print
rkangel 4y ago
I think his stack is a little confused. He's got HTMX and Phoenix in there. If you are using Phoenix then LiveView is the obvious approach to dynamically updating a page based on server stuff. It's a similar-ish architecture to HTMX, but integrated into the framework. The page is rendered on the server as normal, then when it loads on the client a web-socket is opened to a task on the server (page includes the LiveView JS). Then when something changes on the server, some new HTML generated and then the parts that have changed are sent down the websocket to the client to insert into the page. LiveView is part of Phoenix, leverages Elixir's concurrency, is very performant and a joy to use. HTMX is a way of getting similar functionality but for a conventional server rendered framework like Django which doesn't have any of this stuff built in. It would be challenging to build it in anyway because the concurrency isn't as powerful. Simplistically, Phoenix exists because Chris McCord was trying to do a LiveView equivalent in Ruby, had issues, went on a search discovered Elixir. So either use: Elixir + Phoenix + Phoenix LiveView Or: Python + Django + HTMX (Python and Django can be substituted for other frameworks like Rails) In both cases, Alpine can then be useful to sprinkle in some clientside only UI features.
- jjdeveloper 4y ago
  Not even sure Alpine is needed anymore as they have Phoenix.LiveView.JS now https://fly.io/phoenix-files/sdeb-toggling-element/ https://fly.io/phoenix-files/sdeb-toggling-element/
- ellen364 4y ago
  From the article, I’m not sure if the author is using all of Phoenix + HTMX + alpine.js or just exploring the combos to see what works. I recently started playing with Phoenix and the intro to channels and LiveView has been a bit confusing. E.g. a few days ago I wondered if it was worth using something like Svelte for the frontend and then realised I could just use LiveView. As a newbie to the ecosystem, it’s taking a while to get the lay of the land and start understanding the options.
- BeFlatXIII 4y ago
  Thanks for this explainer. This is the missing “here is when it’s redundant” guideline when investigating whether to add to your stack.
ewalk153 4y ago
Flyio does promote a pattern for avoiding the distributed write database complexity: request replay, a single main write database, and replicated read dbs.\1 When a request comes in to write on a read server that attempt a db write, the request is aborted and replayed on the main write server. With some clever assumptions such as “get requests rarely write to the db” and “post request usually do”, much of the write traffic can skip the read vms. They created a ruby rack middleware\2 to standardize this pattern for Ruby on Rails. \1 https://fly.io/blog/run-ordinary-rails-apps-globally/ https://fly.io/blog/run-ordinary-rails-apps-globally/ 2\ https://github.com/superfly/fly-ruby https://github.com/superfly/fly-ruby
- fauigerzigerk 4y ago
  >Flyio does promote a pattern for avoiding the distributed write database complexity I think the author is talking about the complexity of dealing with read after write situations.
jjdeveloper 4y ago
I’m unsure why you would need HTMX and alpine. Phoenix I believe is capable of handling what both of those would provide, or perhaps I’m missing something?
maliker 4y ago
I’ve used fly.io for a couple new projects. The main thing I like about it is that it supports affordable and easy to use persistent volumes, something the container services (GCP, AWS) don’t. This lets me test locally with volumes in a way that is identical to how things will work when deployed. With the other container hosts, I’ve had to refactor to use cloud storage services like S3. Fly.io also has a clean, highly usable CLI and minimal set of services unlike the hundreds of options on other providers. But that’s just icing on top—the volume support is the big advantage for me.
satyrnein 4y ago
Tired of SPA complexity? Try server side rendering, now with websockets, globally distributed nodes, read replicas, and eventual consistency! All of this tech sounds cool, but like the author, I'm unsure when it's called for.
- chrismccord 4y ago
  To tame the snark from the quoted comment, I think it's worth breaking down. Current SPA trends are about deploying your app separate from the backend, often CDN-style close to the user (because speed of light matters). Most apps at scale use caching for reads on hot-code paths, so now we have "eventual consistency" in the mix. Elixir is distributed out of the box, so while "global distribution" sounds fanciful, it's literally baked into the Virtual Machine, Fly simply gives us a private ipv6 network across the globe. All Elixir sees is a cluster of hosts that it can connect to, and it's off to the races. What I'm getting at is all this snark actually describes most application folks build today at any scale, and we build distributed apps with Elixir because it's a distributed platform. > All of this tech sounds cool, but like the author, I'm unsure when it's called for. Imagine if you could write your dynamic UI with realtime updates, and you didn't have to bootstrap JSON apis, or GraphQL schemas for it. Imagine doing `PubSub.broadcast(room, "new_message", ...)` and it gets sent globally to all your instances – with no external dependency. Want to show some activity on the page when something happens on the cluster? Broadcast the event, then write 3 lines of code to update your UI. Imagine writing "naive" template code that renders some markup, but what falls out is smaller payloads on the wire than your carefully typed and specified GraphQL schemas that require serialization rules for all your objects. Imagine doing away with all that and gaining all the benefits of payload size. If that sounds interesting, Phoenix + LiveView would be called for any time you wanted a dynamic UI or realtime updates and bonus points if you care about writing less code and killing layers of abstraction. Fly would be called for for the same reason folks use CDN's today, to serve resources of the app close to the user, except we just serve the app there instead.
  - satyrnein 4y ago
    It really is pretty cool stuff, and it may not be additional complexity for those who are already at the scale where they're dealing with CDNs and caching hot paths, as you say. For the long tail of Rails/Django/Laravel apps sitting on Heroku or a pair of EC2s in Virginia, who are looking at SPAs with trepidation, I think the case is less obvious. Sorry if the snark was excessive and thanks for your reply!
calltrak 4y ago
simonw 4y ago
> It seems like this would add a whole new class of bugs, like “I just submitted a form to change a setting and when the page reloaded, it still showed my previous value in the form” – since the write hadn’t propagated to the local read replica yet. There's a very solid solution to this that isn't as widely known as it should be. Read after write consistency is extremely important. If a user makes an edit to their content and then can't see that edit in the next page they load they will assume things are broken, and that the site has lost their content. This is really bad! The best fix for this is to make sure that all reads from that user are directed to the lead database for a short period of time after they make an edit. The Fly replay header is perfect for this. Here's what to do: Any time a user performs a write (which should involve a POST request), set a cookie with a very short time expiry - 5s perhaps, though monitor your worst case replica lag to pick the right value. I have trust issues with clocks in user's browsers, so I like to do this by including a value of the cookie that's the server-time when it should expire. In your application's top-level middleware, look for that cookie. If a user has it and the court time has not been reached yet, send a Fly replay header that internally redirects the request to the lead region. This guarantees that users who have just performed a write won't see stale data from a lagging replica. And the implementation is a dozen or so lines of code. Obviously this won't work for every product - if you're building a chat app where every active user writes to the database every few seconds implementing this will send almost every piece of traffic to your leaders leaving your replicas with not much to do. But if your application fits the common pattern where 95% of traffic are reads and only a small portion of your users are causing writes at any one time I would expect this to be extremely effective. Fly replay headers are explained in detail here: https://fly.io/blog/globally-distributed-postgres/ https://fly.io/blog/globally-distributed-postgres/
- simonw 4y ago
  There's another, more sophisticated trick that works for some databases: tracking a global transaction counter of some sort, persisting that in a cookie when a user makes a write and redirecting the user to the lead database if the replica they are talking to hasn't made it to that point yet. Chris McCord describes how Elixir does that with PostgreSQL here: https://news.ycombinator.com/item?id=31434094 https://news.ycombinator.com/item?id=31434094 Wikipedia implements this trick on top of PHP and MySQL global transaction IDs (GTIDs) so it definitely scales!
chrismccord 4y ago
One of the points about read replicas and read-your-own-writes is correct to call out, but on the Elixir side we have an answer to that: > It seems like this would add a whole new class of bugs, like “I just submitted a form to change a setting and when the page reloaded, it still showed my previous value in the form” – since the write hadn’t propagated to the local read replica yet. Elixir is distributed out of the box, so nodes can message each other. This allowed us to easily ship a `fly_postgres_elixir` library that guarantees read-your-own-writes: https://github.com/superfly/fly_postgres_elixir https://github.com/superfly/fly_postgres_elixir It does this by sending writes to the primary region over RPC (via distributed elixir). The write is performed on a primary instance adjacent to the DB, then the result, and the postgres log-sequence-number, is sent back to the remote node. When the library gets a result of the RPC write, it blocks locally until its local read replica matches an LSN >= write LSN, then the result is returned to the caller This gives us read-your-own-writes for the end-user, and the calling code remains unchanged for standard code paths. This doesn't solve all classes of race conditions – for example you may broadcast a message over Phoenix.PubSub that causes a read on the remote node for data that isn't yet replicated, but typically you'd avoid an N query problem from pubsub in general by populating the data in the message on the publisher beforehand. There's no completely avoiding the fact you have a distributed system where the speed of light matters, but it's Fly's (and Phoenix's) goal to push those concerns back as far as possible. For read heavy apps, or apps that use caching layers for reads, developers already face these kinds of problems. If you think of your read-replicas as cache with a convenient SQL interface, you can avoid most foot guns. I'm happy to answer other questions as it relates to Phoenix, Fly or what Phoenix + Fly enables from my perspective.
- benwilson-512 4y ago
  That's very cool. Presumably as well this could be opt out if you had specific operations (perhaps from a write-only API) that don't need to wait to do further writes?
  - chrismccord 4y ago
    Yeah we expose interfaces to ignore blocking on the LSN, but the way this works is by proxying the Ecto Repo interface with our own Repo. So you could call your underlying Repo directly if you wanted to perform a write without blocking on the LSN as well.
Mo3 4y ago
Fly.io, as sorry as I am to say, does not come close to the functionality Heroku offers yet. Redis instances are single-region single-replica, for example. On another note, as soon as they offer serverless functions and solid redundant Redis + SQL I'll be thinking about moving some of our production services over there for a test run.
- tptacek 4y ago
  Just so we're clear: our take right now is that if you want Redis, you should run Redis as an app. The Redis we provide "built in" to the platform is an artifact of an earlier iteration of Fly.io.
  - Mo3 4y ago
    Yes - which unfortunately results in a single-instance, single-region Redis instance, as far as I understood.
    - tptacek 4y ago
      Not sure I follow, sorry. If you want multi-region Redis, you can deploy multi-region Redis.
      - Mo3 4y ago
        Now I am not quite sure if I follow haha. Sorry. The documentation states, > "to get a Redis instance with persistent storage running in a single region." Would I be correct to assume that I would have to take care of configuring the clustering? I'm obviously aware I could deploy multiple Redis instances in different regions, but then I would end up with different Redis instances, no?
bilater 4y ago
Noob question but don't Netlify and Vercel do this already at least for Next.js apps (cache to the edge, run serverless functions on edge nodes etc)?
tptacek 4y ago
Just a quick note that the list of applications for Fly.io at the end of this post was taken from our Launch HN --- https://news.ycombinator.com/item?id=22616857 https://news.ycombinator.com/item?id=22616857 --- and we've changed (expanded) since then. When we launched, we didn't do persistent storage for instances, so it didn't make as much sense to run ordinary apps here; rather, the idea was that you'd run your full-stack app somewhere like us-east-1, and carve off performance-sensitive bits and run them on Fly.io. That's "edge computing". But a bit over a year ago, we added persistent volumes, and then we built Fly Postgres on top of it. You can store files on Fly.io or use a bunch of different databases, some of which we support directly. So it makes a lot more sense to run arbitrary applications, like a Rails or Elixir app, which is not something we would have said back in March 2020.
Stampo00 4y ago
I'm enjoying fly.io so far. I just dropped DigitalOcean because of their price hike. No hard feelings. I was barely using it, and the product is growing more towards full-featured apps and teams, which is not as good a fit for me, an individual just screwing around. I don't fault them. I'm not their target customer. Fly.io is very much designed for use primarily via their CLI tool. Their web interface needs some polish. But it does everything it says on the tin, for a price that's more than reasonable. I only used Heroku briefly so I can't comment on similarities or differences with any authority. As someone who is already very comfortable with container-based development, I'm happy with fly.io.
zoomzoom 4y ago
There's been a lot of talk about fly.io lately - it's clearly an awesome and exciting platform. But I'd have to agree with the author here that it doesn't solve the core problems faced by most web devs and web dev teams. There are 3 relevant (for this comment) "performance layers" in building software: - Cycle time of a team or of the project - this is affected the most by language/framework choice, DevOps infrastructure, and team working style - this should be measured in days/weeks - Feedback loop for an individual dev working on a new ticket - this is based on the team's cycle time but in addition is really about the dev environment, team collaboration, how the team maintains quality, and how well-defined work is before being started - this should be measured in muinutes/hours - Performance of the software deployed in terms of response time to end users - milliseconds Fly.io helps the most with category #3. But how often is that really the most important issue in choosing where to deploy your app? If an alternative made small sacrifices there (for example went form 99.99% performance to 99%) but gained velocity for individual devs and the team to be able to ship better product more quickly, would the company/project be better off? At Coherence (www.withcoherence.com) - disclosure that I'm a cofounder - we're laser-focused on a post-Heroku development platform that goers further than Heroku on categories 1 & 2 above (where I'd argue Heroku is still the gold standard) rather than focusing on category 3. We're super early but in closed beta - if it sounds exciting please check us out and request a demo on the site!

130 points by hartleybrody 4y ago

tlarkworthy 4y ago

Cloud Run is an alternative to fly.io that scales to zero so it can be much cheaper but with added cold starts.

bobnamob 4y ago
Isn’t the whole appeal of fly that they’re geolocated near your users, reducing latency so you can use fancy serverside rendering stacks like Phoenix? Cold start latency kinda ruins that no?
- aenis 4y ago
  Latency on cloud run is not an issue, in my experience (been using it around 4 years now, on a fairly large scale). Its generally fast as is, and if you want to pre-allocate a minimum # of instances, cold starts are less of a problem (basically only if you are suddenly stampeded by a spike in traffic). But the whole smart routing and localized storage concepts are left for you to implement. You can have a bunch of cloud run services behind an ELB that does geo-proximity based routing but firestore is region-bound, and cloud spanner can be very expensive. Not saying there are no workarounds but it seems to me fly.io offers a much lower cost of entry here.
- tlarkworthy 4y ago
  No cloud Run instance supports 1000 concurrent users and stays active for 15 min after no traffic, so only a small subset of users should ever hit a cold start.
aenis 4y ago
But, to get a cloud run based setup: - you need an elb - and this is not cheap (if you have an efficient backend, the elb will dwarf compute costs) - no persistent volumes, and you are encouraged to use gcs or firestore - each region requires a new deployment. No big deal but certainly not super easy to automate, esp. given the need to run behind an ELB (which you need on gcp to have a WAF) - google sdks for some languages suck big time. Most of python sdk is not async friendly, unbelievable as it seems. I do use cloud run for projects big and small, and rather like it, but its hardly a competitor to fly.io imho.
- tlarkworthy 4y ago
  Yeah the load balancer bit sucks, I forgot about that. I use terraform for multi region deploys but yeah, the load balancer is a major cost.

csmpltn 4y ago

An unrelated, yet honest question. There have been many posts hitting the HN frontpage regarding fly.io recently. Is it healthy to have so much content about a single PAAS platform showing up here so often now?

telotortium 4y ago
For better or worse, fly.io has as a principal tptacek, who's at the top of the HN leaderboard and so has built up a lot of goodwill here.
- tut-urut-utut 4y ago
  This actually makes me wonder if people here generally pay attention, who posted what? Does that influence actual upvote status? I never look at the person name when replying or voting, only the content. For example, I remembered this tptacek not because I remembered his posts, but because they get frequently mentioned in other people posts.
  - Multicomp 4y ago
    > Does that influence actual upvote status? It didn't for me. I also don't look at who posted links. But fly.io is leaning into a concept I want to use and want to become more of a thing, and that's using SQLite for more web scale applications. Because there's web scale of Google and there's web scale of the rest of us, and I'm sick of paying the operational overhead of having a full database cluster when the app doesn't utilize its features any more than it would a SQLite db. But in the past everyone turned their nose up at SQLite because they were cool and you were dirty and gross if you wanted to simplify things. Don't you know we just need horizontal scaling because any second now we're going to get more than 10 write requests per second? But fly.io leaning into litestream for replicating those databases is a thumbs up to using simple boring technology (SQLite), while still getting 80% of the benefits of doing containers hosted in a cloud platform.
  - snowwrestler 4y ago
    A lot of people on HN do pay attention to who is posting. This is why Cloudflare is generally beloved here as well: the principles show up in the comments whenever it’s mentioned.

Dave3of5 4y ago

A little note about read replicas and problems I've discovered. It's often the case in code that you write a value to the DB then immediately read it back often in the form of a different query somewhere else. If you are setup to do some kind of a round robin read from the read replicas you can often get a different read from what you wrote as the value hasn't replicated to your read replicas yet. The solution is to use the write endpoint when reading after a write. He says that here but just wanted to point out that it can happen inside an api and cause real issues with data.

danpalmer 4y ago
This depends on the database and consistency level it’s enforcing. You can often configure databases to require an ack from the replicas before it returns, so that you’ll be able to read your writes. This obviously has a trade off with speed. Some databases are cleverer about this. Things like Spanner and FoundationDB work differently so as to be fast to both read and write, but they’re much more complex to operate and use. There is another quick trick though… if a client performs a write, set a bit in their session that causes all of their reads to come from the primary database for a short period, maybe a few seconds, just enough to cover the replication latency. This is a hack. It’s got a lot of downsides, but it’s a quick way to patch the problem if truly necessary.
- manmal 4y ago
  I think that's an interesting idea. Here's a post that benchmarked multi-region Postgres (Elixir/Phoenix on fly.io): https://nathanwillson.com/blog/posts/2021-09-25-fly-multi-db/#3-configure-phoenix-elixir--but-really-ecto- https://nathanwillson.com/blog/posts/2021-09-25-fly-multi-db... According to the post, for some users (residing in Japan, and the primary instance being located in Amsterdam), a query could take ~200ms (median). If multiple queries are performed for each request, that could mean 1 second or more per API call - not so great if that's the case for multiple seconds after each write. I think this would eventually lead to putting more code in stored procedures, begging the question: why not use a distributed DB like Fauna in the first place? Alternatively, the replication problem could be accounted for in the app itself. E.g. the SPA or the edge instance could retry reads following a write until the change from the primary instance has propagated, and up until then pretend that everything went fine. In case a write isn't replicated within 10 seconds or so, show an error to the user and let them retry the write action. This could lead to duplicate entries, but I'd estimate the chance for that to be quite low.

julianbuse 4y ago

I think some things need to be built out some more, like their postgres and adding more storage after creation, but in general it's really enjoyable to use. The pricing seems fair, and their blog is intersting and fun to read.

tpetry 4y ago
Completely agree. Their offering for running containers is great, but nobody wants to maintain a database. They should add DBaaS on their architecture with automatic backups etc.
- jamesmcintyre 4y ago
  I totally agree with this! This would really add so much value to fly.io. And if they don't want to allocate resources to this right now I wonder if they could work with DBaaS providers like Planet Scale or FaunaDB where they'd wireguard their nearest instances into the fly.io network, add cli integrations/automations that'd link to their respective dashboards, etc. I plan to use fly.io + planetscale and I'm hoping to still get low latency between those two services but it's no where near the low latency cloudflare can achieve with their new edge redis/db offerings (or the fly.io db at edge strategies) but after looking into fly.io's db strategies I really feel hesitant to take on that level of devops/additional-engineering when something like planetscale provides so much value out of the box. Hope the fly.io team has something in the works either way! (And I'd love if they chime in with any input here in terms of performance between fly.io and existing DBaaS providers that are regionally replicated by default.)

marban 4y ago

I found render.com more convincing than fly.io (which looked more like a beta product with a prime-time landing page), with both of them not being anywhere close to making me jump from GCP.

pw 4y ago

The end of this article raises the issue of whether Fly.io’s USP, deploying app servers close to your users, is useful for run of the mill web apps. And as much as I like Fly.io and the people associated with it, I’ve wondered this myself. It just seems like serving to US customers from any major US data center is generally fast enough. And I think this might even be true for the world of HTML-over-the-wire web stuff, which Fly.io seems to investing heavily in. No doubts there are plenty of more niche uses (if I were serving users internationally, I’d probably use Fly.io), but the use case just doesn’t seem as broad as the Heroku/PaaS comparisons make it out to be.

dx034 4y ago
Not only that, having one location for a world-wide user base is usually enough. You can optimize much more through rendering speed, blocking requests etc than by being closer to your user. And even if your page becomes really popular, 3 locations (Europe, US, East Asia) are enough to be <200ms to any user in the world. And it keeps your setup and cost much lower.
- mrkurt 4y ago
  This pretty much fits our definition of "deploy app servers close to your users". One region works just fine for some apps. Some are worth going to three.

davidkuennen 4y ago

Do many companies actually need databases geolocated near users? I'm working on big and small projects/companies and that has never been any concern of ours. I always imagined it to be something only the very very big players care about. And as a big player I would usually bet on a big partner like AWS, GCP, Azure. Or am I missing something?

sph 4y ago
Web Scale was first a meme, than an ideal everyone pushes towards even when deploying their small scale blog. Who knows what'll happen if suddenly you get a million concurrent users tomorrow? Better scale it geographically and put it behind a CDN today. Look at those generous free tiers. Like you said, it's mostly snake oil except for very big players.
- pid-1 4y ago
  I'd say using stuff like Netlify or GH Pages for static sites is worth it even if you have zero traffic. They legit are much easier to use than setting up your own VPS.
- fauigerzigerk 4y ago
  This isn't quite the same thing as trying to scale like Google though. Low latency is very important for usability regardless of how many users you serve. How easy it is to achieve depends more on the geographical distribution of your users than on their number. If my app has a handful of users that are split between the US and Europe or Asia, and the app is 90% reads, then the distributed DB approach of fly.io or Cloudflare makes a lot of sense. It also adds considerable complexity though, so it's obviously a tradeoff.
kitbrennan 4y ago
Our business has an API that can be used for displaying dynamic information at point of sale (i.e. dynamic in that it cannot be cached and will need a DB call). While we encourage our customers to try and use us asynchronously, we have a number of enterprises that don't and therefore demand incredibly fast response times with low latency. They pay us accordingly, so as a result we have geolocated databases (in our case though, we are using AWS Aurora replication).

bijay6779 4y ago

nicoburns 4y ago

> But despite how much I want to learn the fly.io platform – it has been a bit tricky for me wrap my head around a good use-case for this type of distributed hosting service. Worth noting that you don't have to use the distributed aspect. I have my site hosted on a single one of a fly.io's smallest instances (which one can get 3 of for free), and even like this the performance is excellent (50ms response times), and it doesn't have the problem of spinning down when not in use like Heroku's free tier. It's nice to at least get a choice of regions. For example, the company I work for (not hosted on fly.io currently) only has customers in the UK and Ireland. So it's would be to be able to pop our servers there with a simple config setting.

petercooper 4y ago
Same. I'm really impressed with the experience on there now that I finally spent a day trying it out. The geodistribution stuff had no interest to me so I'd avoided them till now, but it's really the underlying tooling and experience that has won me over.
hartleybrody 4y ago
This is an excellent point. While their main value prop seems to be "servers closer to your users" you could also just use them as a drop-in replacement for something like heroku and just use one region to simplify the mental model, pricing and orchestration.

kif 4y ago

> But despite how much I want to learn the fly.io platform – it has been a bit tricky for me wrap my head around a good use-case for this type of distributed hosting service. The distributed features are there for when you need them – I don't think you have to use them. Or am I missing something?

dfee 4y ago

I’ve got a deploy running on Fly.io, but I didn’t go with the buildpack option; instead I’m pushing a locally built docker image (buildpacks don’t support pnpm). One big miss, though, is you’ll still need a database and s3, so I’m not sure if I totally understand the value.

dom__inic 4y ago

Unrelated to this topic, but I think you should apply this one line of CSS to your stylesheet - it improves your text aesthetics and readability :) html { -webkit-font-smoothing: "antialiased" }

alx__ 4y ago
First off, avoid changing that property. It doesn't improve things 10 years later, still relevant - https://usabilitypost.com/2012/11/05/stop-fixing-font-smoothing/ https://usabilitypost.com/2012/11/05/stop-fixing-font-smooth... Secondly, the issue may be the font being used. I don't recognize it, but probably not optimized for modern web screens. Or a "converted" typeface originally designed for print

rkangel 4y ago

I think his stack is a little confused. He's got HTMX and Phoenix in there. If you are using Phoenix then LiveView is the obvious approach to dynamically updating a page based on server stuff. It's a similar-ish architecture to HTMX, but integrated into the framework. The page is rendered on the server as normal, then when it loads on the client a web-socket is opened to a task on the server (page includes the LiveView JS). Then when something changes on the server, some new HTML generated and then the parts that have changed are sent down the websocket to the client to insert into the page. LiveView is part of Phoenix, leverages Elixir's concurrency, is very performant and a joy to use. HTMX is a way of getting similar functionality but for a conventional server rendered framework like Django which doesn't have any of this stuff built in. It would be challenging to build it in anyway because the concurrency isn't as powerful. Simplistically, Phoenix exists because Chris McCord was trying to do a LiveView equivalent in Ruby, had issues, went on a search discovered Elixir. So either use: Elixir + Phoenix + Phoenix LiveView Or: Python + Django + HTMX (Python and Django can be substituted for other frameworks like Rails) In both cases, Alpine can then be useful to sprinkle in some clientside only UI features.

jjdeveloper 4y ago
Not even sure Alpine is needed anymore as they have Phoenix.LiveView.JS now https://fly.io/phoenix-files/sdeb-toggling-element/ https://fly.io/phoenix-files/sdeb-toggling-element/
ellen364 4y ago
From the article, I’m not sure if the author is using all of Phoenix + HTMX + alpine.js or just exploring the combos to see what works. I recently started playing with Phoenix and the intro to channels and LiveView has been a bit confusing. E.g. a few days ago I wondered if it was worth using something like Svelte for the frontend and then realised I could just use LiveView. As a newbie to the ecosystem, it’s taking a while to get the lay of the land and start understanding the options.
BeFlatXIII 4y ago
Thanks for this explainer. This is the missing “here is when it’s redundant” guideline when investigating whether to add to your stack.

ewalk153 4y ago

Flyio does promote a pattern for avoiding the distributed write database complexity: request replay, a single main write database, and replicated read dbs.\1 When a request comes in to write on a read server that attempt a db write, the request is aborted and replayed on the main write server. With some clever assumptions such as “get requests rarely write to the db” and “post request usually do”, much of the write traffic can skip the read vms. They created a ruby rack middleware\2 to standardize this pattern for Ruby on Rails. \1 https://fly.io/blog/run-ordinary-rails-apps-globally/ https://fly.io/blog/run-ordinary-rails-apps-globally/ 2\ https://github.com/superfly/fly-ruby https://github.com/superfly/fly-ruby

fauigerzigerk 4y ago
>Flyio does promote a pattern for avoiding the distributed write database complexity I think the author is talking about the complexity of dealing with read after write situations.

jjdeveloper 4y ago

I’m unsure why you would need HTMX and alpine. Phoenix I believe is capable of handling what both of those would provide, or perhaps I’m missing something?

maliker 4y ago

I’ve used fly.io for a couple new projects. The main thing I like about it is that it supports affordable and easy to use persistent volumes, something the container services (GCP, AWS) don’t. This lets me test locally with volumes in a way that is identical to how things will work when deployed. With the other container hosts, I’ve had to refactor to use cloud storage services like S3. Fly.io also has a clean, highly usable CLI and minimal set of services unlike the hundreds of options on other providers. But that’s just icing on top—the volume support is the big advantage for me.

satyrnein 4y ago

Tired of SPA complexity? Try server side rendering, now with websockets, globally distributed nodes, read replicas, and eventual consistency! All of this tech sounds cool, but like the author, I'm unsure when it's called for.

chrismccord 4y ago
To tame the snark from the quoted comment, I think it's worth breaking down. Current SPA trends are about deploying your app separate from the backend, often CDN-style close to the user (because speed of light matters). Most apps at scale use caching for reads on hot-code paths, so now we have "eventual consistency" in the mix. Elixir is distributed out of the box, so while "global distribution" sounds fanciful, it's literally baked into the Virtual Machine, Fly simply gives us a private ipv6 network across the globe. All Elixir sees is a cluster of hosts that it can connect to, and it's off to the races. What I'm getting at is all this snark actually describes most application folks build today at any scale, and we build distributed apps with Elixir because it's a distributed platform. > All of this tech sounds cool, but like the author, I'm unsure when it's called for. Imagine if you could write your dynamic UI with realtime updates, and you didn't have to bootstrap JSON apis, or GraphQL schemas for it. Imagine doing `PubSub.broadcast(room, "new_message", ...)` and it gets sent globally to all your instances – with no external dependency. Want to show some activity on the page when something happens on the cluster? Broadcast the event, then write 3 lines of code to update your UI. Imagine writing "naive" template code that renders some markup, but what falls out is smaller payloads on the wire than your carefully typed and specified GraphQL schemas that require serialization rules for all your objects. Imagine doing away with all that and gaining all the benefits of payload size. If that sounds interesting, Phoenix + LiveView would be called for any time you wanted a dynamic UI or realtime updates and bonus points if you care about writing less code and killing layers of abstraction. Fly would be called for for the same reason folks use CDN's today, to serve resources of the app close to the user, except we just serve the app there instead.
- satyrnein 4y ago
  It really is pretty cool stuff, and it may not be additional complexity for those who are already at the scale where they're dealing with CDNs and caching hot paths, as you say. For the long tail of Rails/Django/Laravel apps sitting on Heroku or a pair of EC2s in Virginia, who are looking at SPAs with trepidation, I think the case is less obvious. Sorry if the snark was excessive and thanks for your reply!

calltrak 4y ago

simonw 4y ago

> It seems like this would add a whole new class of bugs, like “I just submitted a form to change a setting and when the page reloaded, it still showed my previous value in the form” – since the write hadn’t propagated to the local read replica yet. There's a very solid solution to this that isn't as widely known as it should be. Read after write consistency is extremely important. If a user makes an edit to their content and then can't see that edit in the next page they load they will assume things are broken, and that the site has lost their content. This is really bad! The best fix for this is to make sure that all reads from that user are directed to the lead database for a short period of time after they make an edit. The Fly replay header is perfect for this. Here's what to do: Any time a user performs a write (which should involve a POST request), set a cookie with a very short time expiry - 5s perhaps, though monitor your worst case replica lag to pick the right value. I have trust issues with clocks in user's browsers, so I like to do this by including a value of the cookie that's the server-time when it should expire. In your application's top-level middleware, look for that cookie. If a user has it and the court time has not been reached yet, send a Fly replay header that internally redirects the request to the lead region. This guarantees that users who have just performed a write won't see stale data from a lagging replica. And the implementation is a dozen or so lines of code. Obviously this won't work for every product - if you're building a chat app where every active user writes to the database every few seconds implementing this will send almost every piece of traffic to your leaders leaving your replicas with not much to do. But if your application fits the common pattern where 95% of traffic are reads and only a small portion of your users are causing writes at any one time I would expect this to be extremely effective. Fly replay headers are explained in detail here: https://fly.io/blog/globally-distributed-postgres/ https://fly.io/blog/globally-distributed-postgres/

simonw 4y ago
There's another, more sophisticated trick that works for some databases: tracking a global transaction counter of some sort, persisting that in a cookie when a user makes a write and redirecting the user to the lead database if the replica they are talking to hasn't made it to that point yet. Chris McCord describes how Elixir does that with PostgreSQL here: https://news.ycombinator.com/item?id=31434094 https://news.ycombinator.com/item?id=31434094 Wikipedia implements this trick on top of PHP and MySQL global transaction IDs (GTIDs) so it definitely scales!

chrismccord 4y ago

One of the points about read replicas and read-your-own-writes is correct to call out, but on the Elixir side we have an answer to that: > It seems like this would add a whole new class of bugs, like “I just submitted a form to change a setting and when the page reloaded, it still showed my previous value in the form” – since the write hadn’t propagated to the local read replica yet. Elixir is distributed out of the box, so nodes can message each other. This allowed us to easily ship a `fly_postgres_elixir` library that guarantees read-your-own-writes: https://github.com/superfly/fly_postgres_elixir https://github.com/superfly/fly_postgres_elixir It does this by sending writes to the primary region over RPC (via distributed elixir). The write is performed on a primary instance adjacent to the DB, then the result, and the postgres log-sequence-number, is sent back to the remote node. When the library gets a result of the RPC write, it blocks locally until its local read replica matches an LSN >= write LSN, then the result is returned to the caller This gives us read-your-own-writes for the end-user, and the calling code remains unchanged for standard code paths. This doesn't solve all classes of race conditions – for example you may broadcast a message over Phoenix.PubSub that causes a read on the remote node for data that isn't yet replicated, but typically you'd avoid an N query problem from pubsub in general by populating the data in the message on the publisher beforehand. There's no completely avoiding the fact you have a distributed system where the speed of light matters, but it's Fly's (and Phoenix's) goal to push those concerns back as far as possible. For read heavy apps, or apps that use caching layers for reads, developers already face these kinds of problems. If you think of your read-replicas as cache with a convenient SQL interface, you can avoid most foot guns. I'm happy to answer other questions as it relates to Phoenix, Fly or what Phoenix + Fly enables from my perspective.

benwilson-512 4y ago
That's very cool. Presumably as well this could be opt out if you had specific operations (perhaps from a write-only API) that don't need to wait to do further writes?
- chrismccord 4y ago
  Yeah we expose interfaces to ignore blocking on the LSN, but the way this works is by proxying the Ecto Repo interface with our own Repo. So you could call your underlying Repo directly if you wanted to perform a write without blocking on the LSN as well.

Mo3 4y ago

Fly.io, as sorry as I am to say, does not come close to the functionality Heroku offers yet. Redis instances are single-region single-replica, for example. On another note, as soon as they offer serverless functions and solid redundant Redis + SQL I'll be thinking about moving some of our production services over there for a test run.

tptacek 4y ago
Just so we're clear: our take right now is that if you want Redis, you should run Redis as an app. The Redis we provide "built in" to the platform is an artifact of an earlier iteration of Fly.io.
- Mo3 4y ago
  Yes - which unfortunately results in a single-instance, single-region Redis instance, as far as I understood.
  - tptacek 4y ago
    Not sure I follow, sorry. If you want multi-region Redis, you can deploy multi-region Redis.
    - Mo3 4y ago
      Now I am not quite sure if I follow haha. Sorry. The documentation states, > "to get a Redis instance with persistent storage running in a single region." Would I be correct to assume that I would have to take care of configuring the clustering? I'm obviously aware I could deploy multiple Redis instances in different regions, but then I would end up with different Redis instances, no?

bilater 4y ago

Noob question but don't Netlify and Vercel do this already at least for Next.js apps (cache to the edge, run serverless functions on edge nodes etc)?

tptacek 4y ago

Just a quick note that the list of applications for Fly.io at the end of this post was taken from our Launch HN --- https://news.ycombinator.com/item?id=22616857 https://news.ycombinator.com/item?id=22616857 --- and we've changed (expanded) since then. When we launched, we didn't do persistent storage for instances, so it didn't make as much sense to run ordinary apps here; rather, the idea was that you'd run your full-stack app somewhere like us-east-1, and carve off performance-sensitive bits and run them on Fly.io. That's "edge computing". But a bit over a year ago, we added persistent volumes, and then we built Fly Postgres on top of it. You can store files on Fly.io or use a bunch of different databases, some of which we support directly. So it makes a lot more sense to run arbitrary applications, like a Rails or Elixir app, which is not something we would have said back in March 2020.

Stampo00 4y ago

I'm enjoying fly.io so far. I just dropped DigitalOcean because of their price hike. No hard feelings. I was barely using it, and the product is growing more towards full-featured apps and teams, which is not as good a fit for me, an individual just screwing around. I don't fault them. I'm not their target customer. Fly.io is very much designed for use primarily via their CLI tool. Their web interface needs some polish. But it does everything it says on the tin, for a price that's more than reasonable. I only used Heroku briefly so I can't comment on similarities or differences with any authority. As someone who is already very comfortable with container-based development, I'm happy with fly.io.

zoomzoom 4y ago

There's been a lot of talk about fly.io lately - it's clearly an awesome and exciting platform. But I'd have to agree with the author here that it doesn't solve the core problems faced by most web devs and web dev teams. There are 3 relevant (for this comment) "performance layers" in building software: - Cycle time of a team or of the project - this is affected the most by language/framework choice, DevOps infrastructure, and team working style - this should be measured in days/weeks - Feedback loop for an individual dev working on a new ticket - this is based on the team's cycle time but in addition is really about the dev environment, team collaboration, how the team maintains quality, and how well-defined work is before being started - this should be measured in muinutes/hours - Performance of the software deployed in terms of response time to end users - milliseconds Fly.io helps the most with category #3. But how often is that really the most important issue in choosing where to deploy your app? If an alternative made small sacrifices there (for example went form 99.99% performance to 99%) but gained velocity for individual devs and the team to be able to ship better product more quickly, would the company/project be better off? At Coherence (www.withcoherence.com) - disclosure that I'm a cofounder - we're laser-focused on a post-Heroku development platform that goers further than Heroku on categories 1 & 2 above (where I'd argue Heroku is still the gold standard) rather than focusing on category 3. We're super early but in closed beta - if it sounds exciting please check us out and request a demo on the site!