12 ms·
> Most LLM client code uses LangChain From what I saw on several LLM projects, they usually don't use LangChain at all - if they really care about performance,
by ElectricalUnion 3y ago
> Most LLM client code uses LangChain
From what I saw on several LLM projects, they usually don't use LangChain at all - if they really care about performance, they go straight to huggingface/transformers or even directly to pytorch.
If you want use PyTorch and you're dead-set into using Ruby then you can just FFI LibTorch and you should be fine; torch-rb mostly does that.
Does it work "OK"? Well, I would say for some cases potentially better that Python, but mostly because in python-land version and package management is somewhat of a unmitigated dumpster fire.
- itake 3y agoLangChain is an API layer wrapping huggingface and/or pytorch. Huggingfacetransformers and pytorch solve a different set of problems than what LangChain solves. I understand that, but most programmers want the simple `pip install` `gem install` flow to work, without needing to create their own bindings code. torch-rb offers the barebones. You'd still need to re-implement the model from whitepapers or translate the python version.