5 ms·
I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.0154
by astrobiased 29d ago
I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.01547
Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area.
It seems more about coverage-driven competence. Somewhat analogous to overfitting at scale.
The harder question, in Chollet’s framing, is: how efficiently can a system learn to do something genuinely new?
With our current AI architectures and training in place, I think we will only continue on skill acquisition optimization vs. truly novel intelligence.
- vessenes 29d agoPretty efficiently, apparently, since it saturated ARC-AGI-3 in half of the predicted time, and according to the Chollet blog post on the fly created dense DSLs to describe and analyze individual games.
- ex-aws-dude 29d agoThey can do new tasks with in-context learning but its obviously limited by context window
- deleted 29d ago[deleted]
- falcor84 29d agoBut what does it mean in practice? Obviously we humans also have a limited cognitive capacity. Let me offer a thought experiment: Let's say that tomorrow we discover Atlantis, with a treasure trove of books about their culture and science, written in a dialect of ancient Greek that we know how to start to analyze, but no one can read fluently. And let's say that you are a billionaire really curious about their culture and want to converse with an "Atlantean expert" as soon as possible. Would you invest your money in a "we-hate-ai-slop(tm)" group of researchers who would abhor AI and instead delegate the books to a massive number of human grad students? Or in a small group of researchers who are willing to use AI agents to go over these? Or maybe just open a chat session with GPT-6 yourself immediately? What would most effectively assuage your curiosity?
- z7 29d agoChollet writes he expects AGI now sooner than 2030, "given progress is happening faster than I expected." https://x.com/fchollet/status/2095607046129463577 https://x.com/fchollet/status/2095607046129463577
- gtirloni 29d agoHow is that an argument to the comment you replied to?
- JacobAsmuth 29d agoAstra is clearly able to acquire new knowledge in context and apply it. It was the whole thing that his ARC-AGI benchmarks have been measuring. It's a direct refutation of the original comment.
- fny 29d agoNone of these LLMs are plastic. They lack neurodiversity. Their thought space and their traversal are likely constrained in someway that humanity's isn't as a collective.
- angusturner 29d agoWhat does this even mean. There is strong evidence of LLMs doing in context learning. Some of the linear RNN layers in recent models are provably doing SGD in hidden space during inference
- oblio 29d ago> What does this even mean. There is strong evidence of LLMs doing in context learning. 1. Is this learning persistent? 2. Do they verify these new lessons against core principles? 3. Do they and protect themselves/ignore requests if these new lessons contradict those core principles? Humans do that from the time they're 3 years old (not that well, but they do do it).
- Jean-Papoulos 29d ago>The harder question, in Chollet’s framing, is: how efficiently can a system learn to do something genuinely new? The more diverse stuff it knows, the easier it will be to learn something new.
- ben_w 29d agoWhile humans can do this, it has historically been difficult for machine learning models to manage it. It is unclear if Transformers are "it" or not, because while they are much more general, they are also very spikey intelligences despite having read almost everything the trainers can get their hands on.
- jeffy29 29d agoDefine novel intelligence in a way that would not exclude 95% of humans, yourself included.
- willis936 29d agoWhen I infer I also train.
- deleted 29d ago[deleted]
- oblio 29d agoBrilliant. We keep pretending LLMs learn. No, they're smart idiots/stupid geniuses. They're turn based intelligence in a real time world. 1. Each time someone talks to me they don't have to repeat the entire conversation from the beginning with each reply. 2. If my boss/partner/whoever gives me some mandates/orders (basically), I don't just forget about them because they were at the beginning of the conversation. 3. If during the conversation I access external data sources to get new info or refresh stale info (a presentation, a book, whatever), I don't instantly forget about it after the conversation ends and forget to incorporate this information if 10 000 other people ask me again. 4. I verify new inputs/lessons against my core principles. 5. I protect myself/ignore requests if new inputs/lessons contradict my core principles. 6. Etc, etc.
- scotty79 29d ago> If my boss/partner/whoever gives me some mandates/orders (basically), I don't just forget about them because they were at the beginning of the conversation. Your context window is 80 years. You are forgetting plenty before you reach the end of it.
- 10xDev 29d agoYou generally forget things you don't retrieve. It is not really a bug but a way to declutter for efficiency. That's not the same as it not fitting your context window because it was at the beginning.
- scotty79 29d ago> Somewhat analogous to overfitting at scale. Sounds like entirety of human education.
- calinet6 29d agoAgree. Token predicting machines will continue to be token predicting machines by nature. Continued size and tuning will have the effect of making them more and more perfect at being average.
- breuleux 29d agoNext-token prediction is a general paradigm, though. In principle, there isn't really anything a sufficiently advanced token predictor couldn't do.
- anthonyrstevens 29d agoThis. People treat "token prediction" dismissively, as if it were a limit and not a foundational skill. Human brains do "token prediction" in all sorts of contexts.
- Dlemlo 29d agoThey learned generic concepts like our brain does to optimize for this particular surprisngly perfect task: You have to be able to respond to a very generic question in a way that the other entity thinks this is good, comprehensive, etc. You can call us situation predicting machines as well if you want. But you undermine what the latent space of an LLM is representing.
- chriskanan 29d agoYou are conflating multiple things. 1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up to focus on that. By forward transfer, what I mean is weights update over time and past learning improves future learning such that we get better sample efficiency. 2) Psychologists distinguish among different kinds of intelligence for Spearman's g (IQ). Crystalized intelligence is using already acquired knowledge (frontier models probably have maxed out that). Fluid intelligence is reasoning and finding solutions in novel situations or without the necessary crystalized knowledge. [Giving colloquial definitions] 3) Now, interestingly, neither of those are correlated with _creativity_ (just they are independent, note some have this threshold theory but it hasn't held up in recent papers). That's what the AI's really are terrible at -- creativity. But I'd argue the vast majority of humans aren't very creative, with truly out-of-the-box ideas. Given that this is HN, and a non-trivial number of us have ADHD, creativity is positively correlated with ADHD. I did a bunch of research on these topics for my AGI course that I teach each Spring (where I then point out conflicting definitions and start using multiple alternative terms rather than AGI to distinguish among the different definitions).
- somethingsome 29d agoVery interesting! I was looking at your website and wondering if there is a way to have access to the course material/videos? In particular: Spring 2025 @ UR : CSC 209/409 Seminar on Artificial General Intelligence Fall 2024 @ UR : CSC 277/477 End-to-End Deep Learning Spring 2023 @ UR : CSC 266/466 Frontiers in Deep Learning Spring 2022 @ Cornell Tech : CS 5787 – Deep Learning Fall 2021 @ RIT : IMGS 684 – Deep Learning for Vision
- monkeydust 29d ago> That's what the AI's really are terrible at -- creativity Good thought piece here "We Are Losing the Ability to Discover What We Didn’t Know to Ask[1]" By Anne-Laure Le Cunff It keeps playing on my mind as I see people at work follow some predetermined AI workflow to get their jobs done, the art of being curious and exploring around the problem is so important to the really big innovations. Been thinking about how to address this through some of the harnesses we are developing in the knowledge working space. [1] https://archive.is/IAxf9 https://archive.is/IAxf9
- tim333 29d agoThey do new stuff all the time. Ask your AI to draw a gerbil riding a unicycle around Pluto and you'd get an image that hasn't been there before. If by genuinely new you mean without any help from past culture, do humans do that? For significant pushing the boundaries of knowledge stuff you maybe need different algorithms like AlphaGo move 37 or Alpha Fold protein folding. Though again how often do humans do that?
- stackbutterflow 29d agoHumans are not prompted.
- fifilura 29d agoAnyone can prompt an AI. Also an AI.
- heaney-555 29d agoMost employed (and contracting) humans are prompted repeatedly every day.
- deathanatos 29d agoYes, but also no. If I prompt a graphics artist, "draw me a picture, any style, of a calendar with this day circled in red", you do not have to tell them what a calendar looks like, what the days of the week are, or how many there are, or that days in a month increment sequentially. Yet if you prompt an AI: https://share.gemini.google/TyUxSnrKmq9h https://share.gemini.google/TyUxSnrKmq9h Also, you probably don't need to tell an artist "don't violate other people's copyrights" while you're at it, though that's perhaps somewhat more debatable than "does the artist know that the day after Thursday is Friday". This applies equally to all disciplines: AI generated code regularly contains wtfisms that a human would not need to be guided from, or at the very least (more similar to the copyright problems) that experience and knowledge would drive them away from — and permit me to entrust, particularly more experienced — humans with a vague outline of the idea, and trust that the details will get filled in sensibly. "Filling in details sensibly" is where AI hallucinates the hardest. (And just to head off, "it's a one-off mistake!" Another example: https://share.gemini.google/yE6axJvBDKYE https://share.gemini.google/yE6axJvBDKYE ; another example: https://share.gemini.google/WG6TEBjlyro7 https://share.gemini.google/WG6TEBjlyro7 (though admittedly, the calendar is pretty good here, I think "humans have 2 arms" is well within the point I'm making of "stuff I don't need to prompt human artists with") ; and another example: https://share.gemini.google/CIH4QM2teQKf https://share.gemini.google/CIH4QM2teQKf ; and another example: https://share.gemini.google/n76c9eJq1dGe https://share.gemini.google/n76c9eJq1dGe)
- harrouet 29d agoI have the same feeling. It is not unlike old-Siri receiving hard-coded workflows for each type of question. It will not scale.
- pmarreck 29d agoMy take on it is that even if there was no "novel" discovery (leaving that up to the reader to define), if you consider human knowledge to be a sphere in N-dimensional state space, "within that surface area" is Swiss cheese, and AI seems to at minimum be able to fill in some of those holes. And those hole-fillings, for all intents and purposes, look to us like novelty, even if much of it was simply overlooked by us, or, perhaps, unable to attain due to time or other constraints. Now if you want to talk beyond the sphere, let's call it the "novel novel discovery of the unknown unknowns", then you may have a point, and AI may be more limited than humans in discovering the things that we don't know we don't know. Especially the as-yet-unmodelable things i.e. intuition. Plenty of discovery left just working from first principles, however. Which I cautiously suggest current frontier AI is good enough to model to a significant enough extent that it is useful for discovery.
- drnick1 29d agoWhy do you expect models to do "genuinely new" things? 99.9% of real world tasks are extremely repetitive.
- nharziro 28d agoI don't understand what you mean by novel intelligence or what people actually expect from these kinds of "Ai" but what novel intelligence can humans claim? Everything we know or learn is based on what someone else figured out. How are llms any different in that respect?