3 ms·
Has anyone managed to cleanly integrate Web search into local models (run with llama.cpp)? The biggest limitation of the class of models that fit into one or tw
by drnick1 4mo ago
Has anyone managed to cleanly integrate Web search into local models (run with llama.cpp)? The biggest limitation of the class of models that fit into one or two consumer GPUs is that they lack world knowledge, but presumably this can be remedied by enabling access to use the Internet.
- kroaton 4mo agoYou're late to the party, mate; we've been doing this for years. Grab a SearXNG instance, stand up an MCP server for it, and expose the tool into your system prompt. Or use Brave Search. Or Exa if you want to pay. Any of them work. The model will pick it up straight away. Even llama.cpp's bundled web UI handles it fine. Dead simple.
- drnick1 4mo ago> Grab a SearXNG instance, stand up an MCP server for it Which MCP server do you use?
- mwowow 4mo agoWorking fine with LM Studio + Web search plugin
- Havoc 4mo agoSearxng is the ghetto solution. Commercial uruky is good. Basically Kagi except you can also run api calls over it Neither is going to return much knowledge. Basically just relevant url so you need a second tool to grab them and there bot walls get tricky