2 ms·

For GPUs startups can now leverage serverless GPU cloud providers: https://ramsrigoutham.medium.com/the-landscape-of-serverless-gpu-providers-in-2023-a21b0ff189

For GPUs startups can now leverage serverless GPU cloud providers: https://ramsrigoutham.medium.com/the-landscape-of-serverless-gpu-providers-in-2023-a21b0ff18901 https://ramsrigoutham.medium.com/the-landscape-of-serverless.... Much simpler than setting up K8s with scale-to-zero autoscaling nodegroups.

mattnewton 4y ago
Will have to revisit this, when we originally evaluated banana.dev and similar platforms they lacked the ability to mount a network drive to quickly load a bunch of model weights after spinup, which is a weird requirement we had with a previous pipeline that we don’t need with other products.
- thundergolfer 4y ago
  Modal.com has a network filesystem feature called 'shared volumes'. Disclaimer: I work for modal.com, but honestly Modal is the most comprehensive of the serverless GPU platforms because it didn't start as just a 'serverlessly run the latest open-source model' platform, it's aiming to be an end-to-end cloud platform.