The problem is accessibility. We have everything else. If loras could be easily trained in Vulkan/DirectX, if vram pools were big enough for the smart models,

The problem is accessibility. We have everything else. If loras could be easily trained in Vulkan/DirectX, if vram pools were big enough for the smart models, basically if stuff just worked, you'd already see much more widespread local llm use in apps. The constraint technology is largely there, take a look at Outlines. So is the ability to efficiently "switch" personalities with Lorax. So are long context base models (Yi/InternLM/Mixtral). Even the super-fast Vulkan inference runtime is there (MLC-LLM). And yes, these are all early, finicky unintegrated implementations which is part of the problem.