6 ms·

I think their usecase makes a lot of sense as their workloads consume a predefined amount of ram. As a customer you rent a VM with a specified amount of memory

I think their usecase makes a lot of sense as their workloads consume a predefined amount of ram. As a customer you rent a VM with a specified amount of memory so fly.io does not care about reclaiming it from a running VM.

no_wizard 3y ago
Depends on if they're using smart memory allocation to keep costs lower, IE, if they can pattern that certain workloads only need N amount of memory at Y time, they can effectively borrow memory from one VM for usage in another that has an opposite statistical likelihood of needing that memory. This is why paying for dedicated memory is often more expensive than its counter part, because that dedicated memory is not considered as part of pooling.
- adql 3y ago
  I think it would be easier to just enable KSM (KVM can use it to deduplicate memory and save some RAM on duplicate blocks like people loading same libraries into memory)
- nerpderp82 3y ago
  Shit like this is how you burn customers and then they find out you aren't changing the sheets, they change hotels. Just because you can doesn't mean you should.
  - ushakov 3y ago
    It’s a common technique though. I believe it’s called oversubscription, where you rent the same hardware to more tenants hoping they won’t use it all at once. Fly.io themselves admitted they’re oversubscribed and AWS is doing the same for years now Source: https://fly.io/blog/the-serverless-server/ https://fly.io/blog/the-serverless-server/
    - tptacek 3y ago
      We're not using virtio memory ballooning right now (we've never had the time to get it working).
  - geodel 3y ago
    >.. they change hotels And then they learn all hotels are doing exactly same thing. One hotel doing is risk, all hotels doing is industry standard. Airlines, hotels, restaurants, doctors and so on oversubscribe all the time. Whoever complains are free to move and add to their further disappointments.
    - imtringued 3y ago
      Memory (DDR4) is like 1.50€ per GB nowadays. There is no need to skimp on it. The most expensive component is still the CPU if you actually want to do work and not just idle your webserver 99% of the time.
      - no_wizard 3y ago
        this ignores just how competitive these services have become and how much competition there is in this marketplace. Anything that allows a company to balance user experience with cost of hardware will be evaluated. The fact is this is super common because even though RAM is super cheap, at scale, these things add up. If you as a provider don't have to spend 120K on RAM and can use that on more CPUs or something else, it can mean having a competitive edge over the provider that does have to spend that 120K.
        nerpderp82 3y ago
        If they were competitive, they would be within an order of magnitude of on-prem, but they are not, they are 5x or more. "Super Common" is a weasel word.
  - no_wizard 3y ago
    Super common though. Everyone from Hetzner to AWS does this, unless you specifically pay for full allocation, which is typically ~2x the price. Chances are you are using systems that do this and you haven't even noticed.
    - yokaze 3y ago
      Werner Vogels seems to disagree: https://twitter.com/Werner/status/25137574680 https://twitter.com/Werner/status/25137574680
      - rob-olmos 3y ago
        Tweeted Sep 21, 2010, might still be true for EC2 VMs though.
        cthalupa 3y ago
        There's the 't' series of instances that offer burstable CPU. AFAIK still 1:1 on memory though, and there's models that allow you to pay to stay un-throttled when using t family instances vs. getting throttled when out of burst credits.
  - klabb3 3y ago
    Almost everything is oversubscribed in our world. Malloc will happily “return” the 15 TiB you asked for. If 10000 people called 911 at the same time, only a tiny fraction would get through (and even fewer would get help). Evacuating a large city by road would result in giant traffic jams. There are 5-8x as many parking spots as there are cars (and we still can’t find a goddamn spot). And of course… the great toilet paper shortage of 2020.
  - mulmen 3y ago
    I don't see the problem. Is this not the value proposition of the cloud? At scale things like block level de-duplication and over-provisioning of compute are more efficient to say nothing of power and cooling. This efficiency drives down cost. As long as your data isn't leaking and you get the hardware you pay for when you need it what's the problem?
    - nerpderp82 3y ago
      Value to whom? Sell a thing and not deliver it. This this our current millieu to scam people and make the difference. The entire vibe of this thread is 1) everyone is doing it 2) efficiency drives cost down (to the vendor) but those savings are not passed to the consumer 3) "hardware you pay for when you need it", all predicated on the at IF and it doesn't happen. Oversubscription should always be opt-in, otherwise it is an underhanded scam.
- flaminHotSpeedo 3y ago
  Generalized oversubscription like that is very challenging if not impossible to do securely, since you want to keep workloads isolated to single tenant numa nodes. E.g. using the firecracker jailer: https://github.com/firecracker-microvm/firecracker/blob/main/docs/jailer.md https://github.com/firecracker-microvm/firecracker/blob/main...

no_wizard 3y ago

Depends on if they're using smart memory allocation to keep costs lower, IE, if they can pattern that certain workloads only need N amount of memory at Y time, they can effectively borrow memory from one VM for usage in another that has an opposite statistical likelihood of needing that memory. This is why paying for dedicated memory is often more expensive than its counter part, because that dedicated memory is not considered as part of pooling.

adql 3y ago
I think it would be easier to just enable KSM (KVM can use it to deduplicate memory and save some RAM on duplicate blocks like people loading same libraries into memory)
nerpderp82 3y ago
Shit like this is how you burn customers and then they find out you aren't changing the sheets, they change hotels. Just because you can doesn't mean you should.
- ushakov 3y ago
  It’s a common technique though. I believe it’s called oversubscription, where you rent the same hardware to more tenants hoping they won’t use it all at once. Fly.io themselves admitted they’re oversubscribed and AWS is doing the same for years now Source: https://fly.io/blog/the-serverless-server/ https://fly.io/blog/the-serverless-server/
  - tptacek 3y ago
    We're not using virtio memory ballooning right now (we've never had the time to get it working).
- geodel 3y ago
  >.. they change hotels And then they learn all hotels are doing exactly same thing. One hotel doing is risk, all hotels doing is industry standard. Airlines, hotels, restaurants, doctors and so on oversubscribe all the time. Whoever complains are free to move and add to their further disappointments.
  - imtringued 3y ago
    Memory (DDR4) is like 1.50€ per GB nowadays. There is no need to skimp on it. The most expensive component is still the CPU if you actually want to do work and not just idle your webserver 99% of the time.
    - no_wizard 3y ago
      this ignores just how competitive these services have become and how much competition there is in this marketplace. Anything that allows a company to balance user experience with cost of hardware will be evaluated. The fact is this is super common because even though RAM is super cheap, at scale, these things add up. If you as a provider don't have to spend 120K on RAM and can use that on more CPUs or something else, it can mean having a competitive edge over the provider that does have to spend that 120K.
      - nerpderp82 3y ago
        If they were competitive, they would be within an order of magnitude of on-prem, but they are not, they are 5x or more. "Super Common" is a weasel word.
- no_wizard 3y ago
  Super common though. Everyone from Hetzner to AWS does this, unless you specifically pay for full allocation, which is typically ~2x the price. Chances are you are using systems that do this and you haven't even noticed.
  - yokaze 3y ago
    Werner Vogels seems to disagree: https://twitter.com/Werner/status/25137574680 https://twitter.com/Werner/status/25137574680
    - rob-olmos 3y ago
      Tweeted Sep 21, 2010, might still be true for EC2 VMs though.
      - cthalupa 3y ago
        There's the 't' series of instances that offer burstable CPU. AFAIK still 1:1 on memory though, and there's models that allow you to pay to stay un-throttled when using t family instances vs. getting throttled when out of burst credits.
- klabb3 3y ago
  Almost everything is oversubscribed in our world. Malloc will happily “return” the 15 TiB you asked for. If 10000 people called 911 at the same time, only a tiny fraction would get through (and even fewer would get help). Evacuating a large city by road would result in giant traffic jams. There are 5-8x as many parking spots as there are cars (and we still can’t find a goddamn spot). And of course… the great toilet paper shortage of 2020.
- mulmen 3y ago
  I don't see the problem. Is this not the value proposition of the cloud? At scale things like block level de-duplication and over-provisioning of compute are more efficient to say nothing of power and cooling. This efficiency drives down cost. As long as your data isn't leaking and you get the hardware you pay for when you need it what's the problem?
  - nerpderp82 3y ago
    Value to whom? Sell a thing and not deliver it. This this our current millieu to scam people and make the difference. The entire vibe of this thread is 1) everyone is doing it 2) efficiency drives cost down (to the vendor) but those savings are not passed to the consumer 3) "hardware you pay for when you need it", all predicated on the at IF and it doesn't happen. Oversubscription should always be opt-in, otherwise it is an underhanded scam.
flaminHotSpeedo 3y ago
Generalized oversubscription like that is very challenging if not impossible to do securely, since you want to keep workloads isolated to single tenant numa nodes. E.g. using the firecracker jailer: https://github.com/firecracker-microvm/firecracker/blob/main/docs/jailer.md https://github.com/firecracker-microvm/firecracker/blob/main...