4 ms·
Isn't search engines quicker than calling a LLM ? It might have a huge impact between a 20ms search engine call and a 2s LLM call for the end user.
by Colegno 2mo ago
Isn't search engines quicker than calling a LLM ? It might have a huge impact between a 20ms search engine call and a 2s LLM call for the end user.
- fastball 2mo agoA 2s LLM call is pretty slow.
- gadflyinyoureye 2mo agoTry using Digital Ocean. Minutes spent on inference.
- quixoticaxolotl 2mo agoThey are already solving the problem with search engines, they're just using an LLM as a first pass to create better embeddings to run a similarity match on first. The difference in latency is likely made up for in accuracy.
- nullsanity 2mo ago[dead]