3 ms·

Strange, I had the same thought about doing this exact exercise this weekend. I think the overall percentage is the wrong approach here. It’s easy to say a lo

Strange, I had the same thought about doing this exact exercise this weekend. I think the overall percentage is the wrong approach here. It’s easy to say a lot of things that are factually true or predictions that are inevitably true. However the more salient point with Gary Marcus is the one unforgivable thing he was wrong about and continues to double down on which is that deep learning is hitting a wall. Starting in early 2022 and going through today, there is still so much low hanging fruit with deep learning. Today’s LLM progress is mostly being made in RL. But world models are also still so early and they’re deep learning all the way down. It would be nice if he would just admit he was wrong.

deleted 7mo ago
[deleted]
cl42 7mo ago
I'll add one more point. If you scroll through his Substack, a lot of his posts are incredibly negative and unproductive. I was (and continue to be) someone who cares deeply about responsible AI... But there's a difference between working on AI responsibly or pushing the debate, versus simply criticizing everything that is done as folly, useless, crap, etc.
davegoldblatt 7mo ago
[flagged]
emp17344 7mo ago
Is “world models” even a real thing, or just the latest AI buzzword?
- downboots 7mo ago
  A world model is an attempt at ensuring your hallucinations are compatible with reality https://www.nvidia.com/en-us/glossary/world-models/ https://www.nvidia.com/en-us/glossary/world-models/ usage of the term seems to be correlated with GPU sales https://news.ycombinator.com/item?id=47232306 https://news.ycombinator.com/item?id=47232306
lambdasquirrel 7mo ago
Depends on how you look at it. In terms of overcoming fundamental limitations, I would argue it has indeed hit a wall. ChatGPT is how old, but LLMs still can't actually count? But then, to your point, what does it matter, if they're still as useful as they are? Even at this stage, Claude Code makes Jira halfway bearable. Of course, we have to consider the devil's advocate as well. Most CEOs don't seem to be reporting great ROI on their "AI" investments.

deleted 7mo ago

[deleted]

cl42 7mo ago

I'll add one more point. If you scroll through his Substack, a lot of his posts are incredibly negative and unproductive. I was (and continue to be) someone who cares deeply about responsible AI... But there's a difference between working on AI responsibly or pushing the debate, versus simply criticizing everything that is done as folly, useless, crap, etc.

davegoldblatt 7mo ago

[flagged]

emp17344 7mo ago

Is “world models” even a real thing, or just the latest AI buzzword?

downboots 7mo ago
A world model is an attempt at ensuring your hallucinations are compatible with reality https://www.nvidia.com/en-us/glossary/world-models/ https://www.nvidia.com/en-us/glossary/world-models/ usage of the term seems to be correlated with GPU sales https://news.ycombinator.com/item?id=47232306 https://news.ycombinator.com/item?id=47232306

lambdasquirrel 7mo ago

Depends on how you look at it. In terms of overcoming fundamental limitations, I would argue it has indeed hit a wall. ChatGPT is how old, but LLMs still can't actually count? But then, to your point, what does it matter, if they're still as useful as they are? Even at this stage, Claude Code makes Jira halfway bearable. Of course, we have to consider the devil's advocate as well. Most CEOs don't seem to be reporting great ROI on their "AI" investments.