4 ms·
An LLM can be trained to pick a URL from a database of known URLs, or from search results. It may choose the wrong URL, of course, but at least it wouldn't hall
by poikroequ 2y ago
An LLM can be trained to pick a URL from a database of known URLs, or from search results. It may choose the wrong URL, of course, but at least it wouldn't hallucinate fake URLs.
- darby_nine 2y agoLLMs aren't "trained to pick a url from a database of known urls". Or at least, we have not collectively seen this yet, perhaps I am wrong. We explicitly wire things so we can use LLMs to generate data useful to index with existing search databases.... acting like LLMs are responsible for the reliability of identifying the source document is disingenuous when a large part of the value of an LLM as an asset is its role laundering other peoples' intellectual property without attribution.
- poikroequ 2y agoTry out phind.com for an example. The LLM generates a query for the search engine to fetch web results. It can include clickable links to the sources in its response. They accomplish this by training/fine-tuning the LLM to output special tokens, or special syntax, which is interpreted in code to perform some action, such as calling out to an API. This is how chatgpt and Gemini are able to automatically search the web and generate images and such. Yes, LLMs just output tokens, but those tokens can be interpreted to perform actions in code.
- SirMaster 2y agoWhy can't it just check the URL to see if it's valid and actually contains the data it thinks it contains before recommending it to the user. Don't modern LLMs like GPT4 have access to load web pages? The downside is it will slow down the output I guess.