5 ms·
LLMs do not have the technology to iteratively solve a complex problem. This is a fact. No graph will change this. You want “reasoning,” then you need to inve
by ChicagoDave 2y ago
LLMs do not have the technology to iteratively solve a complex problem.
This is a fact. No graph will change this.
You want “reasoning,” then you need to invent a new technology to iterate, validate, experiment, validate, query external expertise, and validate again. When we get that technology, then AI will become resilient in solving complex problems.
- sigmoid10 2y agoThat's false. It has been shown that LLMs can perform e.g. gradient descent internally [1], which can explain why they are so good at few shot prompting. The universal approximation theorem already tells us that a single layer is sufficient to approximate any function, so it should come as no surprise that modern deep networks with many layers should be able to perform iterative optimisations. [1] https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559
- ChicagoDave 2y agoWhat’s the success rate once you hit 20 prompts? What’s the success rate if you hit 30 prompts 40 prompts? I’m pretty sure that as you increase the complexity of your questioning, the LLM is just gonna flat out fail and no change to the vector database is going to improve that.
- sigmoid10 2y agoWhat are you talking about? This has nothing to do with vector databases. This is about the basic capabilities of attention networks and dense layers.
- slashdave 2y agoSo you want to encapsulate reason into a simple function?
- sigmoid10 2y agoWhy not? At the most simple level, the human brain also just takes a set of inputs and produces a set of outputs. There is a huge complicated function behind that, but complexity is no longer an issue thanks to modern compute capabilities.
- ChicagoDave 2y agoYou clearly don't understand short-term and long-term memory and using multiple disciplinary information in problem solving. LLMs can only provide answers to what already exists. They cannot invent new answers. Therefore, they cannot handle complexity.
- sigmoid10 2y agoThat's so wrong, I don't even know where to begin. You should really look up the fundamentals of how these models work instead of listening to the "statistical parrot" nonsense that is constantly spewed around HN.
- slashdave 2y agoIt doesn't sound like you understand what a function is. There is a mathematical definition you need to look up.
- sigmoid10 2y agoTo be honest it doesn't sound like you understand it. Or maybe look up the universal approximation theorem that says this is very much possible. Many people just have this dangerous tendency to put their own mind on a pedestal. That's how they justified their superiority over slaves or minorities in the past and it's how they will justify it over machines in the future.
- Iwan-Zotow 2y agoIs it related to KAN?
- ChicagoDave 2y agoYou have to wonder whose fortunes are tied to the LLM hype. I have no problem pointing out the falsehoods being pushed by these efforts. It may take time for the bubble to burst, but it will. Mark this comment.
- ChicagoDave 2y agoGoldman Sachs seems to agree with my assessment: https://www.goldmansachs.com/intelligence/pages/gs-research/gen-ai-too-much-spend-too-little-benefit/report.pdf?ref=wheresyoured.at https://www.goldmansachs.com/intelligence/pages/gs-research/...