3 ms·
Running an LLM locally and paying for access to OpenAI are two separate concerns. But to address both: is it very relevant what LLM you use right now? Local or
by bredren 3y ago
Running an LLM locally and paying for access to OpenAI are two separate concerns.
But to address both: is it very relevant what LLM you use right now? Local or hosted, openAI or other?
It seems like the interface has converged around chat-based prompts.
New ideas for tuning or improving the efficiency of foundational models are published almost every week.
If one wants to build a product on top of of generative AI, why not simply start with what’s free or works with one’s dev environment?
Presumably, the interaction with or API to text-based gen AI will be very similar no matter what engine is best for your use case at any given time.
This would imply these backends will be swappable, the way web services are that copy AWS S3 APIs.
So, to return to the point, can’t people just build their product with openAI or other and plan to move away based on the cost and fit for their circumstances?
Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior?
It seems far-fetched to believe this tech can be constrained by legislation.
OpenAI can lobby all they want, it won’t necessarily buy them anything. Look what happened with FTX.
Since LLMs can be run locally and the engines be black boxes to the user, how could a legislative act really prevent them from being everywhere—-especially given the public utility.
- ignoramous 3y ago> Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior? This, infact, might be a better way to do inference anyway: https://twitter.com/Francis_YAO_/status/1675967988925710338 https://twitter.com/Francis_YAO_/status/1675967988925710338 > So, to return to the point, can’t people just build their product with openAI or other and plan to move away based on the cost and fit for their circumstances? Depends. There are signs that folks are buying into GPT-specific APIs (like function calls) which may not be as easy to migrate away from.
- bredren 3y agoAsking because I have not implemented these yet: is there anything unique about the syntax that it can't just be copied?
- ignoramous 3y agoSome (not all) projects are indeed "copying" the OpenAI APIs; ex: https://github.com/go-skynet/LocalAI/issues/588 https://github.com/go-skynet/LocalAI/issues/588
- joaogante 3y ago> Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior? It can be done -- it is the basis for assisted generation and related work. It does require full access to the model, to be time and money-efficient. See https://huggingface.co/blog/assisted-generation https://huggingface.co/blog/assisted-generation Disclaimer: I'm the author of the blog post linked above.