4 ms·
The irony in your statement is immense. Yes, Kurzeweil has been saying this for decades. No it doesn't mean AGI is close. These llms do nothing to advance AGI.
by ggm 3y ago
The irony in your statement is immense. Yes, Kurzeweil has been saying this for decades. No it doesn't mean AGI is close. These llms do nothing to advance AGI. There is no theoretical basis to the belief in emergent intelligence from statistical language models and the answers are amazingly good, highly unreliable and parrot meaning at best. There is no inductance, and no inteospection and no understanding of the deep semantic meaning of the language presented. There's no intelligence.
- coolspot 3y ago> the answers are amazingly good, highly unreliable and parrot meaning at best. There is no inductance, and no inteospection and no understanding of the deep semantic meaning of the language presented. There's no intelligence. Can say the same about a half of population, tbh
- binary132 3y agoSo what you’re saying is.... we’re going to have human-level AI, and it’s going to be incredibly stupid
- bugglebeetle 3y agoBAAGI (Below Average Artificial General Intelligence).
- djmips 3y agoOr more simply not AGI but AGS
- glimshe 3y agoWhy do you say they do nothing to advance AGI? Do you know what it takes to advance AGI? It's hard to state that without knowing how AGI would work yourself. LLMs would be considered magic just a couple years ago. Sure, not AGI but behaves just like one for certain workloads. I find hard to believe we're not a bit closer now - or maybe even a lot closer.
- shrimpx 3y agoAGI should have morals, opinions, self-reflection, learn continuously from sensor data, reason, realize when they’re proven wrong and update their model of the world, and be creative. So far LLMs exhibit none of those. But LLMs exhibit a digestible distillation of a very large body of data which may be a component of an AGI. But you can have an AGI that doesn’t have encyclopedic knowledge but it’s still highly intelligent, so I don’t think LLMs have to be an intrinsic component.
- zarzavat 3y agoThat is not what AGI means. AGI = Artificial General Intelligence. 1. Artificial = we made it 2. General = it can solve problems in any field 3. Intelligence = the ability to solve problems A chess engine is a very strong Artificial Intelligence. But it’s not very General, it can only evaluate chess positions. GPT-4 is very General, you can ask it about any question and get a somewhat reasonable answer. But it’s not very intelligent, often the answer is wrong. You’re talking about an Artificial Human. That’s a different problem. Intelligence is not species dependent. Dolphins are intelligent (a bit), aliens can be intelligent and have zero emotions or conception of self. There’s certainly plenty of amoral intelligent serial killers.
- PaulDavisThe1st 3y agoGPT-4 solves only 1 problem: what is the most likely stream of tokens to follow what we already have. It is remarkably good at this. But there's absolutely zero reason to believe it can solve any other problem at all.
- zarzavat 3y agoHow do humans write if not by intuiting what word comes after another? Intelligence is the ability of that next word decision procedure to determine a next word that is aligned with our human intuition and model of truth. I believe what you’re getting at is modality, that GPT-4 only provides responses in text. You can’t ask it to drive a car, or paint like Dall-e. And that’s a fair criticism, but it’s mostly just because it would make the models too large and slow, not because we don’t know how to do it. The thing we don’t know how to do is make a model reason as well as a human, and it makes sense to try to solve that in the text domain first rather than making highly multimodal models that reason poorly in all domains.
- circuit10 3y agoBy most measures you could think of for intelligence languages models are improving, so I don’t see why you think this wouldn’t lead to something at least almost human-level if you scaled it up enough Of course there could be some wall somewhere but I don’t see why there would be
- croes 3y agoBecause you need more training data for better results and they are running out of new training data.
- ddingus 3y agoI don't think so. While it may be true that new data is coming in at a trickle these days, due to things like Discord, Slack, et al. all locking conversation and context up, as well as the daily volume of chapter is small relative to what is out there now. The fact is that training data can be used in many different ways and I bet you we see the products of that fairly quickly as those who see this same as I do reach a point where they want to show n tell and test.
- croes 3y ago>The fact is that training data can be used in many different ways and I bet you we see the products of that fairly quickly as those who see this same as I do reach a point where they want to show n tell and test. Sounds like wishful thinking to overcome the limitations of LLMs. At the same time we get more and more texts generated by LLMs so it gets harder to get actual man made texts.
- circuit10 3y agoThat’s true for LLMs but not necessarily for reinforcement learning
- ggm 3y agoThat's "we need a larger cowbell" thinking. It's not a theory of mind, it's wishful thinking that it will.. emerge. Absent theory I don't think moar will make it emerge, no.
- panarky 3y agoHis prediction was that one human brain's worth of computing power could be acquired for $1000 by 2029. That still seems reasonable. That's not the same as AGI or the singularity.
- ggm 3y agoYou have a metric for human brains worth of computing power which hasn't already been exceeded? I can't do infinite precision arithmetic or the RSA algorithm in my head, or index a billion strings into lexical sort order. But I am human, I am conscious and no visible VLSI work or algorithmic model will lead to AGI or a human equivalent computing power by 2029. Let alone for $1000.
- dekhn 3y agowell, you could just be hallucinating your own consciousness. By 2029 it seems not unreasonable to expect that the most sophisticated models will carry out visual and auditory interactions which could fool even the most sophisticated viewer. At that point, what really does consciousness mean? If the robot insisted to me it was conscious, how can I really say no?
- ggm 3y ago> it seems not unreasonable This is where we differ.
- kimixa 3y agoThe lack of concept of "knowledge" is a big one for me - if that's an emergent thing it hasn't even shown hints of this yet. This to me seems a pretty hard line right now, as it limits their capability of things even inexperienced humans can do - namely decide if they actually know something, and identify when they don't know something and attempt to fix that - e.g. asking for clarification on vague inputs, or deciding if something is actually truth or fiction. That then ties into another limitation right now - how after a training the model is pretty static, so cannot learn and has no state outside it's context buffer. This could just be another point where a few orders of magnitude more computing power can "fix" it, doing whole training steps between each input to actually incorporate new information and corrections into itself instead of relying on a fixed size small context. But I'm not deep enough into things to say if they're fundamental issues, or current techniques will start displaying those effects as emergent characteristics as the complexity and training increases. There's been a few other examples when "known" techniques start to show unexpected characteristics down the line as they are scaled up, so can't really say for sure they'll /never/ be shown, just that the current examples don't seem to show even the beginnings of that sort of thing.