3 ms·
Does anyone know which (if any) serverless providers support LoRA? It should be possible to serve community models like Llama3-70B and dynamically load an adap
by anon373839 3y ago
Does anyone know which (if any) serverless providers support LoRA? It should be possible to serve community models like Llama3-70B and dynamically load an adapter on request, to greatly reduce the “cold start” problem.
- hansonw 3y agohttps://predibase.com https://predibase.com
- anon373839 3y agoNice find! Better pricing than Replicate, too.
- movedx01 3y agofireworks does: https://readme.fireworks.ai/reference/requirements-and-limits https://readme.fireworks.ai/reference/requirements-and-limit... but its limited to rank 64, CloudFlare workers too, but there rank is limited to as low as 8