4 ms·
HuggingChat – ChatGPT alternative with open source models
- giuscri 3y agoWhere do I find info on how web search in LLMs work and how they’re trained to do that?
- PhilippGille 3y agoIt's called retrieval augmented generation (RAG) and there's no extra training. The data (e.g. web search result) is given as input to the LLM. If you search for "retrieval augmented generation" you'll find papers, tutorials, videos etc about it.
- d4rkp4ttern 3y agoWeb search is not a capability of a “bare” LLM, but in an LLM-based system it can be done by giving the LLM access to a “web search tool”, I.e essentially you instruct it to output a specific structured text (typically json but doesn’t have to be) indicating its “intent” to search, and your wrapper intercepts/detects this structured response, does the actual search and returns the results (e.g snippets from top k results) into the context of the LLM amd have it use these to respond to your question. A similar thing can be done with external documents - your wrapper retrieves docs/fragments relevant to the query, puts them in the context and lets the LLM use them to answer the query. This is called Retrieval Augmented Generation (RAG). The above is a highly simplified description. In the Langroid library (a multi-agent framework from ex-CMU/UW-Madison researchers) we have these and more. For example here’s a script that combines web search and RAG: https://github.com/langroid/langroid/blob/main/examples/docqa/chat-search.py https://github.com/langroid/langroid/blob/main/examples/docq...
- joshstrange 3y agoI can’t wait for mistralai/Mixtral-8x7B-Instruct support in llama.cpp. This models does much better in a few tests compared to other open models. GPT4 still reigns supreme but this gets a lot closer IMHO.
- ZoomZoomZoom 3y agoIt's been pushed yesterday. Works just fine.
- crakenzak 3y agoI've been using this almost exclusively for `mistralai/Mixtral-8x7B-Instruct-v0.1`. Great service and hope it grows in popularity!