3 ms·
...yet.
by adocomplete 3y ago
...yet.
- tomrod 3y agoI work a bit in this space. Current LLM architectures are missing a fundamental "state of the world" as well as the ability to counterfactually reason out of error propagation. Simply a limit of the architecture. Instead, we should be giving model architectures like JEPA a go [0], which explicitly perform LLM-like behavior but with a state of the world implemented for ongoing error correction. [0] https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/ https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/
- singularity2001 3y agoRemember Andrew Ngs theses that current architectures are already "AI complete" meaning that given more training data all models outperform other models with less training data, and a model which seems a bit more efficient with some amount of training data can be less efficient with more data or the other way around.
- deleted 3y ago[deleted]
- tomrod 3y agoDo you mind providing a link to your reference?
- deleted 3y ago[deleted]