3 ms·

This is a perfect example of snakeoil nonesense being sold in the AI tech market these days. People offering to wrap your LLM calls with another prompt asking t

This is a perfect example of snakeoil nonesense being sold in the AI tech market these days. People offering to wrap your LLM calls with another prompt asking the LLM to "reconsider" or "does this make sense to you", and selling you that artisanally crafted prompt at a premium. This is simply an information retrieval problem. LLMs don't retrieve information perfectly from their context. They are very good at it, but every now and then they'll introduce a change here or there. Changing an "Hello" to "Hi" doesn't really make any difference, but changing an "PS/2 Model 286" to "PS/2 Model 280" makes a huge difference. The LLM "knows"* this at some level because it "knows" that names are important to be reproduced in exact format. But everynow and then even names can change and sill generally mean the same thing, so everynow and then it'll change a name or an identifier for no reason. some of my favorite descriptions of this I have heard from people: - We need to introduce a reflection layer - We need a supervisor tree-like checks on responses - We need an evaluation-feedback mechanism for AI operations - We need to agents that operates and judges/supervisors/evaluators all apparently mean: const response = await getChatResponse([...messages, newMsg]); const {didGood, why} = await getChatResponse([RolePlayLikeAMeanJudgePromot, ...messages, newMsg, response]) if (!didGoog) { response = await getChatResonse([ThreateningMafiaGuyPrompt, [ ...messages, newMsg, response, why[]) } // loop like 4 times, maybe use different models and egg them on each other, like use Grok and tell it you got the response from Claude and that Claude was shit talking Grok all the way through it. Like it was unnecessary tbh. // this makes Grook work extra hard. It's actually been measured by kill-a-watt. *: I say "knows" to indicate just that the model training biases it that way

mdp2021 1y ago
We consider LLMs an "intuition" machine that can talk and partially understand, and do not let it retrieve information from its faulty memory and will, but force it to use an implemented memex to produce any output. "Tell us about X (X='PS/2 Models'); here are your encyclopedias: extract and formulate".
- eddythompson80 1y ago
  If you were to actually try that you'd know that approach doesn't really work either. Or rather, it's not the silver bullet you hope it is. If you still think that, go ahead and implement it. That's literally the main "output quality" struggle all AI providers are in. If you're just building a chatbot (like a pure ChatGPT/Claude interface-like) you risk massively increasing your latency and degrading your overall result quality for an attempt to improve a small scenario here or there. Seriously, try it. Take any "Tell us about X" prompt you like. Try it as-is with an LLM, then try it with + "; here are your encyclopedias: extract and formulate" I guarantee you that 99 times out of 100, the LLM will always reach out to the encyclopedia. The existing encyclopedia doesn't have a great LLM-like search interface that's able to find the most relevant parts to the LLM's query about X. In fact, you're building that part if I'm not mistaken. If you expect the encyclopedia to have that great search functionality that the LLM could use to always find the most relevant information about X, then you just pushed the problem one layer down. Someone will actually eventually have to tackle it. You can also see this in both ChatGPT and Claude outputs. Every now and then they will push a change to make it "more reliable" which basically makes it more likely to search the internet before answering a question. Which also happens to be more likely to skew its output based on SEO, current popular news and other nonesense. While nonscientific, I experience this everytime ChatGPT or Claude decide to do a web search instead of just answering the question. Ask it "I like tv show X, suggest tv shows like that" or "I like product X, suggest a similar product". If it uses the internet to search, it's a summary of the top gamed SEO results. Just whatever is popular atm, or whatever has commission links. Ask it not to use the internet and the result is surprisingly less.... "viral, SEO optimized, trended recently" type content.
  - mdp2021 1y ago
    You are misunderstanding the proposed frame: implementations may be faulty, but the approach remains necessary for "LLMs as informers". I.e., the answer provider should only work vis-a-vis documentation founding the output. This implies that if we do not have good enough ways to retrieve information from repositories, we will have to invent them. Because the "LLM as informer" can only be allowed to formulate what it will find through the memex. It is possible that to that aim, LLMs can not be directly implemented as they are in the current general state. Also the problem of information reliability has to be tackled, in order to build such system (e.g. some sources rank higher). It is not a solved problem, but it is a clear one. In mission critical applications, you would not even allow asking John at the nearby desk for information he may confuse.

mdp2021 1y ago

We consider LLMs an "intuition" machine that can talk and partially understand, and do not let it retrieve information from its faulty memory and will, but force it to use an implemented memex to produce any output. "Tell us about X (X='PS/2 Models'); here are your encyclopedias: extract and formulate".

eddythompson80 1y ago
If you were to actually try that you'd know that approach doesn't really work either. Or rather, it's not the silver bullet you hope it is. If you still think that, go ahead and implement it. That's literally the main "output quality" struggle all AI providers are in. If you're just building a chatbot (like a pure ChatGPT/Claude interface-like) you risk massively increasing your latency and degrading your overall result quality for an attempt to improve a small scenario here or there. Seriously, try it. Take any "Tell us about X" prompt you like. Try it as-is with an LLM, then try it with + "; here are your encyclopedias: extract and formulate" I guarantee you that 99 times out of 100, the LLM will always reach out to the encyclopedia. The existing encyclopedia doesn't have a great LLM-like search interface that's able to find the most relevant parts to the LLM's query about X. In fact, you're building that part if I'm not mistaken. If you expect the encyclopedia to have that great search functionality that the LLM could use to always find the most relevant information about X, then you just pushed the problem one layer down. Someone will actually eventually have to tackle it. You can also see this in both ChatGPT and Claude outputs. Every now and then they will push a change to make it "more reliable" which basically makes it more likely to search the internet before answering a question. Which also happens to be more likely to skew its output based on SEO, current popular news and other nonesense. While nonscientific, I experience this everytime ChatGPT or Claude decide to do a web search instead of just answering the question. Ask it "I like tv show X, suggest tv shows like that" or "I like product X, suggest a similar product". If it uses the internet to search, it's a summary of the top gamed SEO results. Just whatever is popular atm, or whatever has commission links. Ask it not to use the internet and the result is surprisingly less.... "viral, SEO optimized, trended recently" type content.
- mdp2021 1y ago
  You are misunderstanding the proposed frame: implementations may be faulty, but the approach remains necessary for "LLMs as informers". I.e., the answer provider should only work vis-a-vis documentation founding the output. This implies that if we do not have good enough ways to retrieve information from repositories, we will have to invent them. Because the "LLM as informer" can only be allowed to formulate what it will find through the memex. It is possible that to that aim, LLMs can not be directly implemented as they are in the current general state. Also the problem of information reliability has to be tackled, in order to build such system (e.g. some sources rank higher). It is not a solved problem, but it is a clear one. In mission critical applications, you would not even allow asking John at the nearby desk for information he may confuse.