check out https://github.com/mlc-ai/web-llm https://github.com/mlc-ai/web-llm IMO the main limitation is access to powerful GPUs for running models locally and

check out https://github.com/mlc-ai/web-llm https://github.com/mlc-ai/web-llm IMO the main limitation is access to powerful GPUs for running models locally and the size of some models causing UX problems with cold starts