2 ms·

If you were to actually try that you'd know that approach doesn't really work either. Or rather, it's not the silver bullet you hope it is. If you still think t

If you were to actually try that you'd know that approach doesn't really work either. Or rather, it's not the silver bullet you hope it is. If you still think that, go ahead and implement it. That's literally the main "output quality" struggle all AI providers are in. If you're just building a chatbot (like a pure ChatGPT/Claude interface-like) you risk massively increasing your latency and degrading your overall result quality for an attempt to improve a small scenario here or there. Seriously, try it. Take any "Tell us about X" prompt you like. Try it as-is with an LLM, then try it with + "; here are your encyclopedias: extract and formulate" I guarantee you that 99 times out of 100, the LLM will always reach out to the encyclopedia. The existing encyclopedia doesn't have a great LLM-like search interface that's able to find the most relevant parts to the LLM's query about X. In fact, you're building that part if I'm not mistaken. If you expect the encyclopedia to have that great search functionality that the LLM could use to always find the most relevant information about X, then you just pushed the problem one layer down. Someone will actually eventually have to tackle it. You can also see this in both ChatGPT and Claude outputs. Every now and then they will push a change to make it "more reliable" which basically makes it more likely to search the internet before answering a question. Which also happens to be more likely to skew its output based on SEO, current popular news and other nonesense. While nonscientific, I experience this everytime ChatGPT or Claude decide to do a web search instead of just answering the question. Ask it "I like tv show X, suggest tv shows like that" or "I like product X, suggest a similar product". If it uses the internet to search, it's a summary of the top gamed SEO results. Just whatever is popular atm, or whatever has commission links. Ask it not to use the internet and the result is surprisingly less.... "viral, SEO optimized, trended recently" type content.

mdp2021 1y ago
You are misunderstanding the proposed frame: implementations may be faulty, but the approach remains necessary for "LLMs as informers". I.e., the answer provider should only work vis-a-vis documentation founding the output. This implies that if we do not have good enough ways to retrieve information from repositories, we will have to invent them. Because the "LLM as informer" can only be allowed to formulate what it will find through the memex. It is possible that to that aim, LLMs can not be directly implemented as they are in the current general state. Also the problem of information reliability has to be tackled, in order to build such system (e.g. some sources rank higher). It is not a solved problem, but it is a clear one. In mission critical applications, you would not even allow asking John at the nearby desk for information he may confuse.

mdp2021 1y ago

You are misunderstanding the proposed frame: implementations may be faulty, but the approach remains necessary for "LLMs as informers". I.e., the answer provider should only work vis-a-vis documentation founding the output. This implies that if we do not have good enough ways to retrieve information from repositories, we will have to invent them. Because the "LLM as informer" can only be allowed to formulate what it will find through the memex. It is possible that to that aim, LLMs can not be directly implemented as they are in the current general state. Also the problem of information reliability has to be tackled, in order to build such system (e.g. some sources rank higher). It is not a solved problem, but it is a clear one. In mission critical applications, you would not even allow asking John at the nearby desk for information he may confuse.