3 ms·
The use-cases and tooling around language models is very premature. So, any framework you build now will either look like bloatware or will remain close to just
by krishadi 3y ago
The use-cases and tooling around language models is very premature. So, any framework you build now will either look like bloatware or will remain close to just calling an API.
The dust around language models needs to settle a bit, for a useful framework to emerge from it.
For our own use-cases, I built a framework from scratch, and it was the best decision we made.
- riter 3y agovery much agreed re: dust settling. it makes no sense deploying any of these libraries to prod. as-is. best to understand a configuration / workflow / tuning / etc. that fits your data best and write it from scratch in golang/rust/whatever.
- Dowwie 3y agoAre these computationally expensive operations? If not, Elixir could fit.
- krishadi 3y agoThey are not all computationally expensive. The rate limiting step here is the LLM call itself over the API. So, async is definitely needed. The other aspects would be loading the template from filesystem. I would assume this could be something that's needs to be optimized in the application.
- data_maan 3y ago> For our own use-cases, I built a framework from scratch, and it was the best decision we made. My thinking precisely. So you just used the "raw" OpenAI (I presume?) API, and no other tech on top?
- krishadi 3y agoExactly. The most important part was working with Jinja templating. So, openai + jinja2.