4 ms·
Seems like a great feature (and honestly allows us to do smarter things for strictly structured generation). I'm curious, what's your main motivation for local
by edunteman 3y ago
Seems like a great feature (and honestly allows us to do smarter things for strictly structured generation). I'm curious, what's your main motivation for local llms vs hosted APIs?
- imtringued 3y agoI'm pretty sure the primary reason is that you don't want an update of the hosted LLM to suddenly break your application without warning.
- ramses0 3y agoNot wanting to get dinged for $20/mo? Ability to use offline for local home automation (eg: "given a verbal input request, determine the home devices in scope and their on/off state" => "given the current devices state and the verbal request, generate a list of home assistant actions to perform"), using a custom model for the above, etc.
- yunohn 3y ago> Not wanting to get dinged for $20/mo? OpenAI (& peer) API pricing is honestly quite cheap - I spend way less than 20$ a month for my needs. On the other hand, ChatGPT offers a different experience, so I pay for that too. https://help.openai.com/en/articles/7127956-how-much-does-gpt-4-cost https://help.openai.com/en/articles/7127956-how-much-does-gp...