3 ms·
Some LLM use cases are async, e.g. agents, "deep research" clones.
by walterbell 2y ago
Some LLM use cases are async, e.g. agents, "deep research" clones.
- diggan 2y agoNot to mention even simpler things, like wanting to tag all of your local notes based on the content, basically a bash loop you can run indefinitely and speed doesn't matter much, as long as it eventually finishes
- vunderba 2y agoAdditionally if all you're doing is simple tagging and classification, you can probably get away with a significantly smaller model (sub 14b parameter model) like Mistral 7b or Qwen 14b.