34 ms·

Last Mile Redis

260 points by pw 5y ago

spondyl 5y ago
These posts always strike a nice balance between doing a deep dive on a topic while still staying accessible and building the reader up to the "Aha!" step by step although I have things deployed on Fly so maybe I'm a little biased at this point
jedberg 5y ago
> The most boring way to exploit geographic cache locality is "just a bunch of Redii". I'm not a latin expert, but 'is' is not a latin ending. The closest would be 'es', but then the plural would be Redia or Reda.
- nerdponx 5y ago
  "-is" is typical 3rd declension. If it's not neuter (how do you pick a gender for loanwords in Romance languages?), it'd be Redes. If it's neuter it'd be Reda as you said. Or maybe it'd follow the -eris pattern and be Rederes or Redera.
  - jedberg 5y ago
    I thought 3rd declension is just 's' for the nominative singular. So we'd need to know what the stem is.
    - nerdponx 5y ago
      The 3rd declension nominative singular can be anything. "Red-" and "Reder-" were my two guesses at reasonable stems. I wouldn't be surprised if Neo-Latin hobbyists had rules for things like this.
- mrkurt 5y ago
  This might surprise you, but I am also not a latin expert. Maybe the plural of Redis is just Redis.
  - jedberg 5y ago
    :) Sorry I was just being pedantic on a Friday afternoon. The rest of the blog post was great and I agree with all your points. I've been advocating for immutable cache keys for years, for the exact reason you mention.
    - mrkurt 5y ago
      Oh I got a kick out of it!
  - twic 5y ago
    Surely Redare.
- Koshkin 5y ago
  Clavis “key”, plural with -es. “Redii” would be plural of “Redius”; “Redes” is the correct one. (In English, “Redises” would be just as good.)
- devoutsalsa 5y ago
  If we're playing Warhammer 40K, the chapter keeping all these Redises around the galaxy alive might be called the Cachess Redisari. Now I wanna see what happens when get a little too twisted by the chaos.
vlmutolo 5y ago
Fly is the only company that consistently makes me want to reconsider my negative views on "the cloud", and it's absolutely 100% due to their incredible blog posts. As for this particular post, it's great as always, but I would have liked to see more specifics on how applications might write to the main replica vs their local Redis instance.
- mrkurt 5y ago
  Well that's nice of you to say. Would you believe we also have negative views of the cloud? You're right, we kind of glossed over how to do that. People usually just keep two Redis connections in their code, something like `regionalRedis` and `globalRedis`. It's cheap to keep Redis connections around. I can't really think of a better way to handle it, it's kind of a weird problem because not _all_ writes need to go a certain place, just writes you deem global.
  - alberth 5y ago
    Since I personally believe fly.io has the most potential in redefining how apps are deployed (though, with no disrespect I don’t believe it’s quite there yet) … is there any plans to providing a PaaS like offering where I can deploy my favorite web framework to your edge network and you handle the OS, Database, App server, etc. Said another way, I just write my web app code and fly.io handles literally everything else. (I don’t even mess with docker etc. Just my app code and be done)
    - mrkurt 5y ago
      Yes, this is the dream. We're definitely not there yet. This is especially important for DBs. What I'd really love is for us to work with DB "owners" to jointly provide managed DBs. One thing I hate about AWS is that they have a lopsided, parasitic relationship with the people who build interesting databases.
      - alberth 5y ago
        Even though people love to hate PHP, people have been able to achieve that dream of only writing code and a hosting provider takes care of the rest for almost 20 years now. I recall in thr early 2000s having a personal VPS account with Dreamhost and doing just that since they managed the OS, Database and Apache/nginx. It’s amazing how in many ways - deploying code over the years has only become radically harder than simpler.
deleted 5y ago
[deleted]
ruuda 5y ago
I'm still not entirely convinced that serving apps closer to users solves more problems than it creates, but I am always impressed by the fly.io blog posts. The writing style makes them a joy to read.
- void_mint 5y ago
  Most pieces of technology create as many problems as they're meant to solve. The cost-benefit is always around whether or not the solved problems are worth the cost of the new problems. If latency is such a problem that you'd want to deploy servers and caches as close as possible to a user, you'd probably gladly pay for the reduction of that latency at the cost of the new class(es) of problem. It was an interesting decision to make a SaaS out of this solution, though, as I think 99% of problems are not solved by having machines closer to their users.
  - nine_k 5y ago
    Not only latency but possibly also the cost / availability of massive egress. This is why e.g. YouTube has local caches around the world. Other heavily-hit caches with heavy content may make sense, too; I can't easily imagine a case beside serving video, though.
    - xmprt 5y ago
      That's an interesting point. However, I wonder if there are extra costs to serve a request from Australia to the US (there are definitely extra costs but I wonder if either the user or server pay them). If these costs are exposed to either of the ends, then it might be cost effective to create something that serve requests from local caches.
      - tomnipotent 5y ago
        > if either the user or server pay them Most of the big operators have peering agreements in place, but that doesn't mean every participant has infinite bandwidth. Google Global Cache and Netflix Open Appliance go a long way to reducing costs by avoiding interconnect where possible.
taf2 5y ago
We do this to locally distribute our javascript… from our main rails app we publish out to each region in aws where we have a cluster of redis and nginx and a little bit of lua to grab and render data out… it’s super fast and very easy to maintain … you get your nice normalized database with all its complex queries and then on the client side you get your nice big blob of context all precomputed and perfect for rendering exactly what the client needs… bonus you cache the context data for a shorter time in each local nginx worker for even faster response times then serving from disk…
manigandham 5y ago
Fly has become a serious contender. I remember the early days where they were more of a reverse proxy with javascript workers but have since evolved into a fantastic platform, solving for both compute and data. I might have to launch a startup just to get a chance to use them.
- gingerlime 5y ago
  yes it looks really interesting. I wonder about the reverse proxy (with caching) aspect since we’re looking at caching some of our Rails pages, and to purge them would mean upgrade to Cloudflare enterprise which is expensive for us. Is fly a more flexible CDN contender then? or would we be better off with bunny or keycdn for this use-case? where does the reverse proxy route fit in this picture?
  - manigandham 5y ago
    Fly is no longer a CDN or reverse proxy. It lets you run anything packaged into container in several regions around the world and serve traffic to them all through a single domain. You can build a CDN on top but if you just need basic CDN features then you should probably look at something else. Cloudflare allows purging all of the content, or individual URLs, for free through the UI or API. You only need enterprise for the more advanced tag-based purging. You can also look at Fastly which is another configurable CDN. BunnyCDN is also good. Start simple and then move as you need more.
    - gingerlime 5y ago
      Thank you. Yes, we need more advanced purging functionality. Enterprise seems like the logical next step, but the price increase is hard to swallow. Using something like Bunny/Fastly etc can save us a bundle, but then we're kinda ditching all the other built-in features. I guess that's exactly the Cloudflare play to get you started cheap... I wasn't aware fly evolved from a CDN. I just saw your comment and the docs mentioning speeding Heroku apps, running nginx proxy, openresty etc, so was curious if it's something worth looking into.
ipsum2 5y ago
I'm curious what the moat of fly.io is. Wouldn't Cloudflare use the same Firecracker VMs to replicate the same experience here if it becomes successful?
- mike_d 5y ago
  The moat is they have super smart folks who keep iterating on the right problems. Cloudflare is too busy trying to build a unified stack for boiling the entire ocean.
  - Siilwyn 5y ago
    What do you mean?
    - manigandham 5y ago
      They're referring to the Cloudflare product suite which has massively grown from CDN to registrar to application security to corporate networking and more. I agree that it can be hard to see and use all the different product lines given how fractured everything is.
  - ipsum2 5y ago
    Smart people aren't a moat, even if they're super smart.
deleted 5y ago
[deleted]
potamic 5y ago
> Fly runs apps (and databases) close to users, by taking Docker images and transmogrifying them into Firecracker micro-vms, running on our hardware around the world. Any reason you run the apps on micro-vms? Why not directly on a container runtime?
- href 5y ago
  A container runtime would not provide proper isolation. The VMs run on bare metal along with the VMs of other users. See https://fly.io/blog/sandboxing-and-workload-isolation/ https://fly.io/blog/sandboxing-and-workload-isolation/
  - potamic 5y ago
    Thank you. That was a great post.
- alexchamberlain 5y ago
  From https://fly.io/docs/reference/architecture/ https://fly.io/docs/reference/architecture/, > MicroVMs provide strong hardware-virtualization-based security and workload isolation, this allows us to safely run applications from different customers on shared hardware.
friseurtermin 5y ago
Great post! I'm not familiar with all of fly's inner workings but I'm guessing that each app gets its own Redis instance at each location (rather than all of your customers sharing instances) because "global" item replication is a bit cheaper if "global" == locations where the app actually exists. I wonder if there's a way for an application to write to a subset of locations (i.e. locations all over the US, I just want to update something in California). Obv I could have each of my CA locations do a local update but maybe I don't want to
- tptacek 5y ago
  The Redis instances we're talking about here are "just" Fly.io apps; they're not like a special platform feature or anything. Which is good news; it means you can do whatever you want with Redis. You could, I guess, do a replica-of-replicas tree configuration, with app nodes holding connections (Redis connections are really cheap) to their local node, a macro-regional node, and the global node and selectively synchronizing. Or you could go the other way and lose some Redis servers, keeping Redis servers only in certain regions. We make it sort of straightforward for apps to suss out the topology with DNS queries.
fulafel 5y ago
I'm sure others will be curious about their concrete locations: Locations: https://js.fly.io/docs/#datacenter-locations https://js.fly.io/docs/#datacenter-locations (TLDR: mostly local in the US so far, with sparse presence globally)
- alexchamberlain 5y ago
  I think that link is out of date? There is a different list here: https://fly.io/docs/reference/regions/ https://fly.io/docs/reference/regions/

260 points by pw 5y ago

These posts always strike a nice balance between doing a deep dive on a topic while still staying accessible and building the reader up to the "Aha!" step by step although I have things deployed on Fly so maybe I'm a little biased at this point

jedberg 5y ago

> The most boring way to exploit geographic cache locality is "just a bunch of Redii". I'm not a latin expert, but 'is' is not a latin ending. The closest would be 'es', but then the plural would be Redia or Reda.

nerdponx 5y ago
"-is" is typical 3rd declension. If it's not neuter (how do you pick a gender for loanwords in Romance languages?), it'd be Redes. If it's neuter it'd be Reda as you said. Or maybe it'd follow the -eris pattern and be Rederes or Redera.
- jedberg 5y ago
  I thought 3rd declension is just 's' for the nominative singular. So we'd need to know what the stem is.
  - nerdponx 5y ago
    The 3rd declension nominative singular can be anything. "Red-" and "Reder-" were my two guesses at reasonable stems. I wouldn't be surprised if Neo-Latin hobbyists had rules for things like this.
mrkurt 5y ago
This might surprise you, but I am also not a latin expert. Maybe the plural of Redis is just Redis.
- jedberg 5y ago
  :) Sorry I was just being pedantic on a Friday afternoon. The rest of the blog post was great and I agree with all your points. I've been advocating for immutable cache keys for years, for the exact reason you mention.
  - mrkurt 5y ago
    Oh I got a kick out of it!
- twic 5y ago
  Surely Redare.
Koshkin 5y ago
Clavis “key”, plural with -es. “Redii” would be plural of “Redius”; “Redes” is the correct one. (In English, “Redises” would be just as good.)
devoutsalsa 5y ago
If we're playing Warhammer 40K, the chapter keeping all these Redises around the galaxy alive might be called the Cachess Redisari. Now I wanna see what happens when get a little too twisted by the chaos.

vlmutolo 5y ago

Fly is the only company that consistently makes me want to reconsider my negative views on "the cloud", and it's absolutely 100% due to their incredible blog posts. As for this particular post, it's great as always, but I would have liked to see more specifics on how applications might write to the main replica vs their local Redis instance.

mrkurt 5y ago
Well that's nice of you to say. Would you believe we also have negative views of the cloud? You're right, we kind of glossed over how to do that. People usually just keep two Redis connections in their code, something like `regionalRedis` and `globalRedis`. It's cheap to keep Redis connections around. I can't really think of a better way to handle it, it's kind of a weird problem because not _all_ writes need to go a certain place, just writes you deem global.
- alberth 5y ago
  Since I personally believe fly.io has the most potential in redefining how apps are deployed (though, with no disrespect I don’t believe it’s quite there yet) … is there any plans to providing a PaaS like offering where I can deploy my favorite web framework to your edge network and you handle the OS, Database, App server, etc. Said another way, I just write my web app code and fly.io handles literally everything else. (I don’t even mess with docker etc. Just my app code and be done)
  - mrkurt 5y ago
    Yes, this is the dream. We're definitely not there yet. This is especially important for DBs. What I'd really love is for us to work with DB "owners" to jointly provide managed DBs. One thing I hate about AWS is that they have a lopsided, parasitic relationship with the people who build interesting databases.
    - alberth 5y ago
      Even though people love to hate PHP, people have been able to achieve that dream of only writing code and a hosting provider takes care of the rest for almost 20 years now. I recall in thr early 2000s having a personal VPS account with Dreamhost and doing just that since they managed the OS, Database and Apache/nginx. It’s amazing how in many ways - deploying code over the years has only become radically harder than simpler.

deleted 5y ago

[deleted]

ruuda 5y ago

I'm still not entirely convinced that serving apps closer to users solves more problems than it creates, but I am always impressed by the fly.io blog posts. The writing style makes them a joy to read.

void_mint 5y ago
Most pieces of technology create as many problems as they're meant to solve. The cost-benefit is always around whether or not the solved problems are worth the cost of the new problems. If latency is such a problem that you'd want to deploy servers and caches as close as possible to a user, you'd probably gladly pay for the reduction of that latency at the cost of the new class(es) of problem. It was an interesting decision to make a SaaS out of this solution, though, as I think 99% of problems are not solved by having machines closer to their users.
- nine_k 5y ago
  Not only latency but possibly also the cost / availability of massive egress. This is why e.g. YouTube has local caches around the world. Other heavily-hit caches with heavy content may make sense, too; I can't easily imagine a case beside serving video, though.
  - xmprt 5y ago
    That's an interesting point. However, I wonder if there are extra costs to serve a request from Australia to the US (there are definitely extra costs but I wonder if either the user or server pay them). If these costs are exposed to either of the ends, then it might be cost effective to create something that serve requests from local caches.
    - tomnipotent 5y ago
      > if either the user or server pay them Most of the big operators have peering agreements in place, but that doesn't mean every participant has infinite bandwidth. Google Global Cache and Netflix Open Appliance go a long way to reducing costs by avoiding interconnect where possible.

taf2 5y ago

We do this to locally distribute our javascript… from our main rails app we publish out to each region in aws where we have a cluster of redis and nginx and a little bit of lua to grab and render data out… it’s super fast and very easy to maintain … you get your nice normalized database with all its complex queries and then on the client side you get your nice big blob of context all precomputed and perfect for rendering exactly what the client needs… bonus you cache the context data for a shorter time in each local nginx worker for even faster response times then serving from disk…

manigandham 5y ago

Fly has become a serious contender. I remember the early days where they were more of a reverse proxy with javascript workers but have since evolved into a fantastic platform, solving for both compute and data. I might have to launch a startup just to get a chance to use them.

gingerlime 5y ago
yes it looks really interesting. I wonder about the reverse proxy (with caching) aspect since we’re looking at caching some of our Rails pages, and to purge them would mean upgrade to Cloudflare enterprise which is expensive for us. Is fly a more flexible CDN contender then? or would we be better off with bunny or keycdn for this use-case? where does the reverse proxy route fit in this picture?
- manigandham 5y ago
  Fly is no longer a CDN or reverse proxy. It lets you run anything packaged into container in several regions around the world and serve traffic to them all through a single domain. You can build a CDN on top but if you just need basic CDN features then you should probably look at something else. Cloudflare allows purging all of the content, or individual URLs, for free through the UI or API. You only need enterprise for the more advanced tag-based purging. You can also look at Fastly which is another configurable CDN. BunnyCDN is also good. Start simple and then move as you need more.
  - gingerlime 5y ago
    Thank you. Yes, we need more advanced purging functionality. Enterprise seems like the logical next step, but the price increase is hard to swallow. Using something like Bunny/Fastly etc can save us a bundle, but then we're kinda ditching all the other built-in features. I guess that's exactly the Cloudflare play to get you started cheap... I wasn't aware fly evolved from a CDN. I just saw your comment and the docs mentioning speeding Heroku apps, running nginx proxy, openresty etc, so was curious if it's something worth looking into.

ipsum2 5y ago

I'm curious what the moat of fly.io is. Wouldn't Cloudflare use the same Firecracker VMs to replicate the same experience here if it becomes successful?

mike_d 5y ago
The moat is they have super smart folks who keep iterating on the right problems. Cloudflare is too busy trying to build a unified stack for boiling the entire ocean.
- Siilwyn 5y ago
  What do you mean?
  - manigandham 5y ago
    They're referring to the Cloudflare product suite which has massively grown from CDN to registrar to application security to corporate networking and more. I agree that it can be hard to see and use all the different product lines given how fractured everything is.
- ipsum2 5y ago
  Smart people aren't a moat, even if they're super smart.

deleted 5y ago

[deleted]

potamic 5y ago

> Fly runs apps (and databases) close to users, by taking Docker images and transmogrifying them into Firecracker micro-vms, running on our hardware around the world. Any reason you run the apps on micro-vms? Why not directly on a container runtime?

href 5y ago
A container runtime would not provide proper isolation. The VMs run on bare metal along with the VMs of other users. See https://fly.io/blog/sandboxing-and-workload-isolation/ https://fly.io/blog/sandboxing-and-workload-isolation/
- potamic 5y ago
  Thank you. That was a great post.
alexchamberlain 5y ago
From https://fly.io/docs/reference/architecture/ https://fly.io/docs/reference/architecture/, > MicroVMs provide strong hardware-virtualization-based security and workload isolation, this allows us to safely run applications from different customers on shared hardware.

friseurtermin 5y ago

Great post! I'm not familiar with all of fly's inner workings but I'm guessing that each app gets its own Redis instance at each location (rather than all of your customers sharing instances) because "global" item replication is a bit cheaper if "global" == locations where the app actually exists. I wonder if there's a way for an application to write to a subset of locations (i.e. locations all over the US, I just want to update something in California). Obv I could have each of my CA locations do a local update but maybe I don't want to

tptacek 5y ago
The Redis instances we're talking about here are "just" Fly.io apps; they're not like a special platform feature or anything. Which is good news; it means you can do whatever you want with Redis. You could, I guess, do a replica-of-replicas tree configuration, with app nodes holding connections (Redis connections are really cheap) to their local node, a macro-regional node, and the global node and selectively synchronizing. Or you could go the other way and lose some Redis servers, keeping Redis servers only in certain regions. We make it sort of straightforward for apps to suss out the topology with DNS queries.

fulafel 5y ago

I'm sure others will be curious about their concrete locations: Locations: https://js.fly.io/docs/#datacenter-locations https://js.fly.io/docs/#datacenter-locations (TLDR: mostly local in the US so far, with sparse presence globally)

alexchamberlain 5y ago
I think that link is out of date? There is a different list here: https://fly.io/docs/reference/regions/ https://fly.io/docs/reference/regions/