3 ms·
There won't, because there's no money in it
by nly 3y ago
There won't, because there's no money in it
- joegibbs 3y agoThere is for whichever company wants to make use of it - say for example LLM-powered NPCs in singleplayer video games (which I'm working on) - pretty much economically unviable with the current model if you want to run everything on the cloud. You're selling a game once rather than selling a subscription, so you're locked in to paying for cloud-based LLM services indefinitely, unless you want to shut the game down after X years, which really annoys people. Also, you're incentivised to be stingy with the LLM-based features because the more you use it the more it costs you, but by running it locally you can offload that cost to the consumer.
- moneywoes 3y agowould love to learn more and help out on NPCs llms. What sort of performance hit are you seeing?
- joegibbs 3y agoSure, here's my writeup about it: https://jgibbs.dev/blogs/local-llm-npcs-in-unreal-engine https://jgibbs.dev/blogs/local-llm-npcs-in-unreal-engine. There's barely any performance hit, only a slight hitch when it starts generating a response.
- larodi 3y agoThere’s indeed a lot of GPU time not utilised even by modern games. I can definitely imagine new games utilising llms much more even and with local inference option.
- andy99 3y agoAt some point soon, LLMs matching current SOTA will run locally in a browser. If applications for LLMs emerge, having one available will be table stakes.