Hi all, I’m working on a project that offers a new way to deploy AI models and save money on expensive cloud GPU instances. Currently, there are a few options

Hi all, I’m working on a project that offers a new way to deploy AI models and save money on expensive cloud GPU instances. Currently, there are a few options: 1. Rent a GPU from AWS, GCP, and Azure. It is the most popular but also the most expensive option. Renting A100 will cost about $3-4.5 per hour, which can rack up to $3000 monthly. If your workload can not utilize hardware fully, you can leverage services that charge per second or request instead of per hour, like Replicate. The typical cost is around $5 per hour; however, you'll save a lot of money if your computations need to run sporadically. 2. Work with less-known providers like Lambda, Coreweave, or runpod.io. You can get the same GPU for about $1.1-2.1 per hour. However, it is tough to get one at times. 3. Rent a cheaper GPU like A6000 for about $0.6 or consumer-grade RTX 4090 for approximately $0.4 from vast.ai. 4. Use your PC if you have one. The cost varies greatly depending on the electricity price in your area; we can assume around $0.1 per hour at $0.2 kWH and 500W power consumption. At full load, it will cost you around $70 per month. The last option is considerably cheaper than the others. However, managing a cluster at home or in the office requires qualifications and effort. Also, existing tools don't target this use case to the best of my knowledge. For example, we need to ensure that multiple workloads are not using the resources simultaneously. This is the problem that I’m trying to address with FairCompute by providing tools and API for managing your home cluster and workloads. Here is a demo leveraging my gaming PC at home to serve an AI model - https://huggingface.co/spaces/faircompute/stable-diffusion https://huggingface.co/spaces/faircompute/stable-diffusion. The demo is hosted on HuggingFace, but a home PC is used for inference. I would appreciate your feedback on how to make Fair more useful or your thoughts on how to solve the problem of AI model training and deployment cost. If you want to try Fair - check out the tutorial at https://faircompute.com/docs/docs/getting_started https://faircompute.com/docs/docs/getting_started