3 ms·
Hallucinating is a hard problem to solve and likely impossible with the current paradigms and models. Making a bigger GPT is not expected to change this. More c
by verdverm 2y ago
Hallucinating is a hard problem to solve and likely impossible with the current paradigms and models. Making a bigger GPT is not expected to change this. More complex (CoT, RAG, agents) systems help, but still has issues and takes longer to respond. There is another dimension of complexity and optimization that gets introduced when you adopt these systems.
There was a post on HN yesterday about research into how the brain is different when handling human vs programming language. What we have today is good at human language. We probably need to rethink and find new models for programming / engineering tasks.