In most workplaces that deal with LLMs you’ve got a few classes of people: 1. People who understand LLMs and know how to run them and have access to run them o

In most workplaces that deal with LLMs you’ve got a few classes of people: 1. People who understand LLMs and know how to run them and have access to run them on the cloud. 2. People who understand LLMs well enough but don’t have access to cloud resources - but still have a decent MacBook Pro. Or maybe access to cloud resources is done via overly tight pipelines. 3. People who are interested in LLMs but don’t have enough technical chops/time to get things going with Llama CPP. 4. People who are fans of LLMs but can’t even install stuff in their computer. This is clearly for #3 and it works well for that group of people. It could also be for #2 when they don’t want to spin up their own front end.