9 ms·

Based on your architecture diagram it looks like you're spinning up an instance per-user? As you're probably finding now, you will hit AWS limits quickly. You

Based on your architecture diagram it looks like you're spinning up an instance per-user? As you're probably finding now, you will hit AWS limits quickly. You might instead want to have a smaller pool of (larger) servers that you run co-resident VMs on with https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/. That will avoid account limits and also keep your AWS costs more predictable.

yamtaddle 4y ago
Just run them in Linux VMs with WASM, on the users' browsers. Make them all pay for it with higher utility bills and greater wear & tear on their hardware. trollface.jpg
- freeone3000 4y ago
  This is actually a good idea for this -- the user wants the education, they can pay for it with their own hardware. Keep your costs low!
  - cogman10 4y ago
    Probably a better experience for everyone. You just have to distribute the image (rather than running vms) and the user gets instantaneous responses.
    - georgyo 4y ago
      If it is hosted on AWS the bandwidth of distributing the images is likely more than the cost of the compute.
      - jacooper 4y ago
        Cloudflare exists
        hunter2_ 4y ago
        I thought Cloudflare only ensures high usage of the free tier for "web"-ish responses, which doesn't even include .txt files. But I suppose this use case is several orders of magnitude away from that of EasyList, at least in request rate.
        jacooper 4y ago
        I mean you could just pay and use R2 directly. I think it would still be much cheaper.
fduran 4y ago
Yes thanks!
temp0826 4y ago
I haven't fully grokked this yet, but one trick I've used in the past to get around limits is AWS Organizations, creating a sub-account per property. A bit more setup but can keep things cleaner administratively.
- icedchai 4y ago
  AWS will raise limits if you ask. Increasing EC2 instance limits is usually a quick turn around.
  - ericbarrett 4y ago
    Yes, the default limits are there to prevent abuse and runaway misconfigurations. They won't turn down revenue if you confirm it's intentional.
  - andrewstuart2 4y ago
    At least for the tests I've done on a small startup recently, they've also implemented some automatic quota increases for EC2. I ran commands that would have (or did) eclipsed my quota, and got an email that my quotas were bumped a few minutes later.
ilyt 4y ago
That's kinda nice use case for the WASM machine/linux emulators, then you just need to provide image and user can run it in the browser > You might instead want to have a smaller pool of (larger) servers that you run co-resident VMs on with https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/. That will avoid account limits and also keep your AWS costs more predictable. I'd imagine (still waiting for it to load lmao) most of it could be containers too.
- twalla 4y ago
  Someone else linked https://github.com/copy/v86 https://github.com/copy/v86 which seems really neat. I like making jokes with coworkers about implementing this or that bit of infra with WASM-based tools mostly to get a rise out of them but each time I make the joke I look into some of the tools or projects and the balance of joke to "I'm actually serious" shifts a little bit to the right.
- encryptluks2 4y ago
  So then users experience will be poor due to the slowness and non-standard implementation. A better solution IMO would be to provide a container with SSH access.
BossingAround 4y ago
Why not spin up containers instead of VMs? Seems to me containers would fit much better than VMs.
- cogman10 4y ago
  Bypassing container security is easier than bypassing VM security.
  - tamrix 4y ago
    Then wouldn't that be the ultimate test ;)
- spiffytech 4y ago
  Containers have a history of escape vulnerabilities, for reasons like sharing a kernel with the host and other containers. VMs are designed from the ground up to isolate guests, rather than focusing on application deployment. Firecracker is the modern container alternative in untrusted compute scenarios, with Fly.io even converting container images into Firecracker VMs.
  - NovemberWhiskey 4y ago
    >Containers have a history of escape vulnerabilities Generally agreed, but for this use-case do we care?
    - nneonneo 4y ago
      I haven’t gotten any of the challenges to load, but if you’re going to simulate a sysadmin it would make sense to give you high privileges (or even root) on the box. The more privileged you are inside a container, the more attack surface you expose.
      - BossingAround 4y ago
        Which is why you create a "dummy" host VM that hosts containers. Nobody's saying "host containers on your prod webserver." On the other hand, spinning up a VM for every user seems insane to me.

yamtaddle 4y ago

Just run them in Linux VMs with WASM, on the users' browsers. Make them all pay for it with higher utility bills and greater wear & tear on their hardware. trollface.jpg

freeone3000 4y ago
This is actually a good idea for this -- the user wants the education, they can pay for it with their own hardware. Keep your costs low!
- cogman10 4y ago
  Probably a better experience for everyone. You just have to distribute the image (rather than running vms) and the user gets instantaneous responses.
  - georgyo 4y ago
    If it is hosted on AWS the bandwidth of distributing the images is likely more than the cost of the compute.
    - jacooper 4y ago
      Cloudflare exists
      - hunter2_ 4y ago
        I thought Cloudflare only ensures high usage of the free tier for "web"-ish responses, which doesn't even include .txt files. But I suppose this use case is several orders of magnitude away from that of EasyList, at least in request rate.
        jacooper 4y ago
        I mean you could just pay and use R2 directly. I think it would still be much cheaper.

fduran 4y ago

Yes thanks!

temp0826 4y ago

I haven't fully grokked this yet, but one trick I've used in the past to get around limits is AWS Organizations, creating a sub-account per property. A bit more setup but can keep things cleaner administratively.

icedchai 4y ago
AWS will raise limits if you ask. Increasing EC2 instance limits is usually a quick turn around.
- ericbarrett 4y ago
  Yes, the default limits are there to prevent abuse and runaway misconfigurations. They won't turn down revenue if you confirm it's intentional.
- andrewstuart2 4y ago
  At least for the tests I've done on a small startup recently, they've also implemented some automatic quota increases for EC2. I ran commands that would have (or did) eclipsed my quota, and got an email that my quotas were bumped a few minutes later.

ilyt 4y ago

That's kinda nice use case for the WASM machine/linux emulators, then you just need to provide image and user can run it in the browser > You might instead want to have a smaller pool of (larger) servers that you run co-resident VMs on with https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/. That will avoid account limits and also keep your AWS costs more predictable. I'd imagine (still waiting for it to load lmao) most of it could be containers too.

twalla 4y ago
Someone else linked https://github.com/copy/v86 https://github.com/copy/v86 which seems really neat. I like making jokes with coworkers about implementing this or that bit of infra with WASM-based tools mostly to get a rise out of them but each time I make the joke I look into some of the tools or projects and the balance of joke to "I'm actually serious" shifts a little bit to the right.
encryptluks2 4y ago
So then users experience will be poor due to the slowness and non-standard implementation. A better solution IMO would be to provide a container with SSH access.

BossingAround 4y ago

Why not spin up containers instead of VMs? Seems to me containers would fit much better than VMs.

cogman10 4y ago
Bypassing container security is easier than bypassing VM security.
- tamrix 4y ago
  Then wouldn't that be the ultimate test ;)
spiffytech 4y ago
Containers have a history of escape vulnerabilities, for reasons like sharing a kernel with the host and other containers. VMs are designed from the ground up to isolate guests, rather than focusing on application deployment. Firecracker is the modern container alternative in untrusted compute scenarios, with Fly.io even converting container images into Firecracker VMs.
- NovemberWhiskey 4y ago
  >Containers have a history of escape vulnerabilities Generally agreed, but for this use-case do we care?
  - nneonneo 4y ago
    I haven’t gotten any of the challenges to load, but if you’re going to simulate a sysadmin it would make sense to give you high privileges (or even root) on the box. The more privileged you are inside a container, the more attack surface you expose.
    - BossingAround 4y ago
      Which is why you create a "dummy" host VM that hosts containers. Nobody's saying "host containers on your prod webserver." On the other hand, spinning up a VM for every user seems insane to me.