11 ms·

Do large language models need sensory grounding for meaning and understanding?

149 points by georgehill 4y ago

carapace 4y ago
Sure, give the machines empirical feedback devices (sensors) and they will become scientists. (The thought also occurs: What happens when humans spend time in a sensory deprivation tank? They start to hallucinate. Food for thought.) As Schmidhuber says, the goal is to "to build [an artificial scientist], then retire".
- analog31 4y ago
  What sensors do they want?
  - carapace 4y ago
    Well, that's the neat thing: we can give them "all" of the senses, eh? I suspect we're about to discover some very interesting things, like which kind of ghosts are real...
    - analog31 4y ago
      We had all the senses, yet we didn't come up with even the motivation to do empirical science until the 17th century. And even today, a general model of human thought would not necessarily contain a clear recipe for doing science.
      - carapace 4y ago
        > We had all the senses We have (I think the current recognized number is about) twenty-seven senses. > we didn't come up with even the motivation to do empirical science until the 17th century That's historically inaccurate, to put it mildly. (E.g. "Science Education in the Early Roman Empire", Richard Carrier) > a general model of human thought would not necessarily contain a clear recipe for doing science. Every human child is a scientist? Anyway, don't overthink it. Once these systems have sensors and can integrate the effects of their behavior on external systems, they'll have empirical data and Bayesian reasoning, they will develop reliable models of the world, they will be able to check their "hallucinations" against real world conditions and adapt. In other words, science is adaptive in the evolutionary sense. These devices are not talking apes, they don't have the "baggage" of glands and DNA and history.
        analog31 4y ago
        Yes, that's a great point. I was assuming, without really thinking about it, that the AI's still carry our historical baggage. We feed a large portion of that baggage to them as their training data, to see if they can impress us with human-like behaviors. Under those conditions, the AI's stand roughly the same chance as a human child of learning to think scientifically. But that's not a hard requirement. I work for a company that makes sensors. Some of our lunchtime conversations have revolved around the idea of a colossal computer being given a plethora of sensors. In a science-fiction sense, in which we don't worry too much about the cost or practicality of doing so.
        carapace 4y ago
        > We feed a large portion of that baggage to them as their training data, to see if they can impress us with human-like behaviors. But we don't feed them only the baggage, eh? We also [can] give them all the writings and teachings of all the great thinkers and humanitarians, all the saints and sages, and then ask them to impersonate e.g. Jesus or Buddha... Whoever it is that "floats your boat", anyone from Papa Smurf or Santa Claus to Gandhi or George Washington... > Under those conditions, the AI's stand roughly the same chance as a human child of learning to think scientifically. It's our choice, eh? If we value scientifically-grounded outputs the networks will change their weights (etc.) and that's what we'll get. > I work for a company that makes sensors. Ah! I envy you. :) > Some of our lunchtime conversations have revolved around the idea of a colossal computer being given a plethora of sensors. In a science-fiction sense, in which we don't worry too much about the cost or practicality of doing so. Hmm, isn't that the Internet? :) Well met!
famouswaffles 4y ago
The idea that it needs so is looking more and more questionable. Don't get me wrong, i'd love to see some multimodal LLMs. In fact, i think research should move in that direction...However "needing" is a strong word. The text only GPT-4 has a solid understanding of space. Very, very impressive. It was only trained on text. The vast improvement on arithmetic is also very impressive. (People learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When you look at some numbers and are asked even basic arithmetic , 467383 + 374748. Or say are these numbers primes or factors?. With a glance, you have no idea what the sum of those numbers would be or if the numbers are primes or factors because the numbers themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform some specific analysis on them learned through internalizing sets of rules that were acquired through a specialized learning process.) all of this is to say that arithmetic, math is not highly encoded in language at all. and still the vast improvement. It's starting to seem like multimodality will get things going faster rather than any real specific necessity. also, i think that if we want say vision/image modality to have positive transfer with NLP then we need to move past the image to text objective task. It's not good enough. The task itself is too lossy and the datasets are garbage. That's why practically every Visual Language model flunks stuff like graphs, receipts, UIs etc. Nobody is describing those things t the level necessary what i can see from gpt-4 vision is pretty crazy though. if it's implicit multimodality and not something like say MM-React, then we need to figure out what they did. By far the most robust display of computer vision i've seen. I think what kosmos is doing (sequence to sequence for Language and images ) has potential.
- coldtea 4y ago
  >all of this is to say that arithmetic, math is not highly encoded in language at all. How did you learn arithmetic then, if not by being shown numbers and equations, and having math rules and concepts explained to you through language?
  - famouswaffles 4y ago
    I didn't say it's not there. or that you couldn't communicate the ideas of math/arithmetic with language. You can get GPT-3 (yes 3) to have 98.5% accuracy on addition arithmetic (even very large numbers) by..simply describing the algorithm of addition to performed on 2 numbers. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 The basic idea i'm communicating is that not everything that can be extracted from self supervisory token prediction can be extracted with the same ease. and it is very easy to see why math is one the higher difficulty things. It is extremely easy comparatively to infere "happy" from the sentence - John is smiling therefore he is ----. all the information you need to make that inference is packed tightly in the preceding words. it is not the same for arithmetic at all.
rektide 4y ago
This sounds close or identical to the idea of an embodied agent. Maybe we get cute & upgrade it to an embodied oracle? https://en.m.wikipedia.org/wiki/Embodied_agent https://en.m.wikipedia.org/wiki/Embodied_agent
- jorgemf 4y ago
  Yes, and it is also related with consciousness. In order to be intelligent, it needs to be embodied, so it can sense and act in the world. More ideas related with consciousness apply here.
  - pixl97 4y ago
    Though many people confused embodied with 'robot', as in an AI would be packed in a unit walking around. I personally see it the opposite way. An AI could have a 1000 bodies all gathering data in the world. Or a million cameras connected to it. Or a set of sensors that spans the globe. In this since a digital entity could have far more embodiment and connectivity then us humans could ever have. Waiting for the first research paper with the term 'globally conscious' at this point.
    - jorgemf 4y ago
      https://www.conscious-robots.com/consscale/index.html https://www.conscious-robots.com/consscale/index.html
  - grantcas 4y ago
    [dead]
abeppu 4y ago
I'm on board with a lot of what's in this deck, but I take issue with the argument on slide 9. Roughly, the probability that an LLM-provided answer is fully correct decreases exponentially with the length of the answer. I think that's trivially true, but it's also true for human-provided answers (a full non-fiction book is going to have some errors), so it doesn't really get to the core problem with LLMs specifically. In much of the rest of the deck, it's just presumed that any variable named x comes from the world in some generic way, which doesn't really distinguish why those are a better basis for knowledge or reasoning than the linguistic inputs to LLMs. I think we're at the point where people working in these areas need some exposure to the prior work on philosophy of mind and philosophy of language.
- elefanten 4y ago
  Appreciate your take, not arguing with it. But I’d imagine Lecun has more than passing familiarity with those. This deck was put out with the Philosophy dept and he had a panel debate with NYU profs across depts (inc phil) recently on this topic. I suspect this is all pushing the top Phil Lang and Phil Mind to their limits too. Besides, if those subjects were anywhere near resolved (or even… decently understood), they probably wouldn’t be in the Phil dept any more.
  - Grothendank 4y ago
    I feel like Lecun did a poor job of localizing the technique relative to any philosophy, or really explaining what it does for a nontechnical audience. Saying he gives talks to philosophers, or saying this pushes philosophy to its limits doesn't fix the problem that lecun does a poor job - in this presentation - of philosophically motivating the proposal. Perhaps I am wrong, and you can point out exactly how lecun explicates the philosophy in the presentation - perhaps it's really embedded in the maths, which I have not appreciated. Appealing to lecun's authority won't fix the opacity of the presentation. But interpreting it can help! Are you up for it?
    - elefanten 4y ago
      I haven't even given it a deep read, so I unfortunately can't help shed any light. From my quick read, it didn't seem to me that laying out a digestible or rigorous philosophical perspective was really the point here. It seemed more directly biology inspired than philosophy. I also don't seek to valorize Lecun. But he was a very early figure working on these technologies and from the beginning there was a neuroscience-inspired impetus to machine learning. My point was sort of the opposite: that assuming Lecun doesn't clear the bar of having "exposure to the prior work on philosophy of mind and philosophy of language" seems like a weak bet. edit: For clarity, I'm making the assumption that lifelong AI researchers who put time into learning from neuroscience... would also gravitate towards and seek to learn from the nearest relevant branches of philosophy.
coldtea 4y ago
Describing LLMs: "Training data: 1 to 2 trillion tokens" Is number of tokens a good metric, given relationships between tokens is what's important? An LLM with 100000 trillion lexically shorted tokens, given one by one, wont be able to do anything except perhaps spell checking. I guess the idea is that tokens are given in such "regular" forms (books, posts, webpages) that their mere count is a good proxy for number of relevant relationships.
- eternalban 4y ago
  I think the underlying assumption is that there is (an invariant) structure and distribution that becomes more resolved (q) as more data is incorporated. Training LLMs also involves sequences of tokens by design so that is also implicit. The only venue left is to question whether this (n -> q) is a linear relationship or something else. That would only matter if we were looking at absolute measure (to compare say an LLM architecture to something else), but as a comparative measure for LLMs it works, whether it is o(n) or o(logn) or whatever, since more tokens means more sequences and more sequences mean greater ability.
PaulHoule 4y ago
In some sense they already have sensory grounding if they are coupled to a visual model. It might sound vacuous but if you ask a robot for the "red ball" and it hands you the red ball, isn't it grounded?
bravura 4y ago
Yes. In Elazar et al. (2019) "How Large Are Lions? Inducing Distributions over Quantitative Attributes" (https://arxiv.org/pdf/1906.01327.pdf https://arxiv.org/pdf/1906.01327.pdf), it requires 100M English triples to roughly induce the answer to this question. How many images do you think a model needs to see in order to answer this question? Of course, there are fact books that contain the size of the lion. But fact books are non-exhaustive and don't contain much of the information that can quickly be gleaned through other forms of perception. Additionally, multimodal learning simply learns faster. What could be a long slog through a shallow gradient trough of linguistic information can instead become a simple decisive step in a multimodal space. If you're interested to read more about this [WARNING: SELF CITE], see Bisk et al. 2020, "Experience Grounds Language" (https://arxiv.org/pdf/2004.10151.pdf https://arxiv.org/pdf/2004.10151.pdf).
- IIAOPSW 4y ago
  No. But with the caveat that it can only truly grasp things that are self contained systems of text. Allow me to give examples: Transit wayfinding: A train service is nothing more than a list of stations it goes to, a station is nothing more than a list of train services that stop there. Nothing about the physical nature of trains or commuting is needed to have a discussion about train lines or to answer questions like "how many transfers does it take from x to y?". You could never have seen a train in your life or even a map of a train, if you've studied the dual lists of stations and services there's nothing more to learn. Chess (and other board games): Standard Chess notation is a list of sequential move made by each player in the format [piece name][x coordinate][y coordinate]. Eg RB6 represents moving the Rook to column B row 6. Chess can be understood entirely in terms of a game of passing a paper back and forth, appending a new token each time, and the rules of the game expressed entirely in terms of how the next token must match the previous part of the list. At no point is it needed to have actually seen the physical board based representation of the game. The machine is grounded in a text based existence. Conversational and linguistic objects are its literal physical objects. Anything outside of that, well it can understand lions and "large" the same we understand atoms and "eigen-states". Experience may ground language, but you are free to ground language in a different reality with different basic constituents and relations. If AGI were to emerge out of trading bots, they would have a language grounded in money and trades. If it emerged out of a bot made to play Diplomacy, it would be a consciousness with the body of a nation state and the atoms of its world would be bits of plastic on a map of Europe. Grounding in our particular reality isn't strictly necessary, but it is helpful if the goal is to make something adept at the modalities of our particular existence and conversational form.
LesZedCB 4y ago
I can't escape the fact that Lecuns complicated charts give the appearance of the required complexity to emulate robust general intelligence but are simply that, added complexity which could simply be encoded in emergent properties of simpler architecture models. unless he's sitting on something that's working I'm not really excited about it. personally, I'm waiting to see what's next after GATO from Deep Mind. their videos are simply mind-blowing.
swayvil 4y ago
Us humans often discuss all kinds of real subjects despite lacking any firsthand experience at all. I see no reason why a machine couldn't do the same. Call it antiscientific. Solipsistic even. But it isn't entirely disasterous, is it?
- mr_toad 4y ago
  It’s only unscientific if you don’t test your theories and learn from your errors.
  - swayvil 4y ago
    Well that would be where the firsthand experience comes in.
- seydor 4y ago
  its not even unscientific. the vast majority of facts we know is from books / texts
deleted 4y ago
[deleted]
georgehill 4y ago
Here is the related tweet from Yann LeCun: https://twitter.com/ylecun/status/1640122342570336267 https://twitter.com/ylecun/status/1640122342570336267
- ioedward 4y ago
  The twitter thread is worth reading, Yann responds to some of the questions raised here.
numpad0 4y ago
Aren’t they already grounded by having training process at all, just weakly? Very interesting read(read: this is a lightyear beyond my brain) otherwise…
morninglight 4y ago
Although there are obvious differences, it is worth considering the life of Helen Keller in this context.
- ttpphd 4y ago
  Ok, I'll bite. What does she have to do with this?
  - morninglight 4y ago
    The paper questioned whether meaning and understanding require sensory grounding. During her life, Helen Keller demonstrated a grasp of both meaning and understanding despite her severe sensory deprivation. This doesn't prove anything, but it does add human context to the question. ....she lost her sight and her hearing after a bout of illness when she was 19 months old. https://en.wikipedia.org/wiki/Helen_Keller https://en.wikipedia.org/wiki/Helen_Keller
    - andrewprock 4y ago
      She was not deprived of all sensory input though. In fact, it was through her working senses that she learned and communicated.
      - morninglight 4y ago
        In order to develop "understanding", at least one input channel must be available to provide something to understand. In order to ascertain that development, we require at least one output channel. Anything less and we are describing a pet rock. Language can be thought of as the rules of the communication. The root question seems to be whether a machine can learn to communicate meaning and understanding over a bidirectional binary channel without previous training. I suspect that such communication can evolve. Some might look around and remark that it already has.
jorgemf 4y ago
I think it is time to move from intelligent systems to conscious systems. Based on [1] in order to have more intelligent systems we do need sensory as the slides state but we also need other things like attention, memory, etc. So we can have intelligent systems that can have a model of the world and make plans and more complex actions (see [2,3]). Maybe not so big models as today's Language Models. I know the slides show some of the ideas, but we cannot add some things without adding other things first. For example we need some kind of memory (long and short term) in order to do planning, adding a prediction function for measuring the cost of an action is a way of doing planning but it have a lot of drawbacks (as loops because the agent does not remember past steps, or what happened just before). Also a self representation is needed to know how the agent takes part in the plan, or a representation of other entity if it is that one who executes the plan. [1] https://www.conscious-robots.com/papers/Arrabales_ALAMAS_ALAg_CR_v37.pdf https://www.conscious-robots.com/papers/Arrabales_ALAMAS_ALA... [2] https://www.conscious-robots.com/consscale/level_tables.html#table2 https://www.conscious-robots.com/consscale/level_tables.html... [3] https://www.conscious-robots.com/papers/Arrabales_PhD_web.pdf https://www.conscious-robots.com/papers/Arrabales_PhD_web.pd...
88stacks 4y ago
Yes
ftxbro 4y ago
So I understand the author has high standing in the community. But I think they are making actually disingenuous arguments by mixing assertions that are true but irrelevant together with assertions that are probably wrong. For example we can break down the following firehose of assertions by the author: Performance is amazing ... but ... they make stupid mistakes Factual errors, logical errors, inconsistency, limited reasoning, toxicity... LLMs have no knowledge of the underlying reality They have no common sense & they can’t plan their answer Unpopular Opinion about AR-LLMs Auto-Regressive LLMs are doomed. They cannot be made factual, non-toxic, etc. They are not controllable > they make stupid mistakes OK maybe some make stupid mistakes, but it's clear that increasingly advanced GPT-N are making fewer of them. > Factual errors Raw LLMs are pure bullshitters but it turns out that facts usually make better bullshit (in its technical sense) than lies. So advanced GPT-N usually are more factual. Furthermore, raw GPT-4 (before reinforcement training) has excellent calibration of its certainty of its beliefs as shown in Figure 8 of the technical report, at least for multiple choice questions. > logical errors Same thing. More advanced ones make fewer logical errors, for whatever reason. It's an emergent property. > inconsistency Nothing about LLMs requires consistency just like nothing about human meaning and understanding requires consistency, but more advanced LLMs emergently give more coherent continuations. This is especially funny because the opposite argument used to be given for why robots will never be on the level of humans - robots are C3P0-like mega-dorks whose wiring will catch fire and circuit boards will explode if we ask them to follow two conflicting rules. > limited reasoning Of course their reasoning is limited. Our reasoning is limited too. Larger language models appear to have less-limited reasoning. > toxicity There is nothing saying that raw LLMs won't be toxic. Probably they will be, according to most definitions. That's why corporations lobomize them with human feedback reinforcement learning as a final 'polishing' step. Some humans are huge assholes too, but probably they have meaning and understanding anyway. > LLMs have no knowledge of the underlying reality OK fine you can say that any p-zombie has no knowledge of the underlying reality if you want, if that's your objection. Or maybe they are saying LLMs don't have pixel buffer visual or time series audio inputs. Does that mean when those are added (they have already been added) then LLMs can possibly get meaning and understanding? > They have no common sense If you say that inhuman automata are by definition incapable of common sense then sure they have no common sense. But if you are talking about testing for common sense, then GPT-N is unlocking a mindblowing amount of common sense as N is increasing. > they can’t plan their answer Probably they are saying this because of next-token-prediction which is tautologically true, in the same way that it's true that humans speak one word after another. But the implication is wrong. They can plan their answer in any sense that matters. > Auto-Regressive LLMs are doomed. OK. Do you mean in terms of technical capabilities, or in terms of societal acceptance? They are different things. Or do you mean they are doomed to never attain meaning and understanding? > They cannot be made factual, non-toxic, etc. They are not controllable. Those same criticisms can all be made against even the most human of humans. Does it mean humans have no meaning or understanding? No. Of course these are also conditional on whatever prompt you are putting to get them to answer questions. If you prompt an advanced raw GPT-N to make stupid mistakes and factual and logical errors and to act especially toxic then it will do it. And, perhaps, only then will it have truly attained meaning and understanding.
nsainsbury 4y ago
For anyone looking for the related talk from LeCun discussing the proposed architecture: https://www.youtube.com/watch?v=VRzvpV9DZ8Y https://www.youtube.com/watch?v=VRzvpV9DZ8Y
est 4y ago
Is "learning to reason" a real challenge? (Kahneman's system I in the slides) From a naive perspective of view, formal methods like SAT solvers, proof assistants works pretty well.
aaronscott 4y ago
I have wondered about sensory input being needed for AGI when thinking about human development and feral children[1]. It seems that complex sensory input, like speech, may be a component of cognitive development. https://en.wikipedia.org/wiki/Feral_child https://en.wikipedia.org/wiki/Feral_child
Borrible 4y ago
With Chatty Everywhere, it's only a matter of weeks (days?) before we will have the answer. I didn't look at the news yesterday, is it already hooked into a Tesla? Knight Rider II -Rise of the Autobot.
antiquark 4y ago
Old tweet from LeCun: "The vast majority of human knowledge, skills, and thoughts are not verbalizable." https://twitter.com/ylecun/status/1368239479463366656 https://twitter.com/ylecun/status/1368239479463366656
stan_kirdey 4y ago
breaking news: machine learning models including LLMs are fuzzy and make mistakes!
hoseja 4y ago
>Have a constant number of computational steps between input and output. Weak representational power. Yes but you also can't kill them with malicious input.
ben_w 4y ago
Ok, so as the link is some slides with a bunch of bullet points and a handful of images, I am going to be limited in my understanding of this in many of the same ways that my (aforementioned limited) understanding suggests that LeCunn is saying that LLMs are limited. So: factual errors/hallucinations (or did I?), logical errors, lacking "common sense" (a term I don't like, but this isn't the place for a linguistics debate on why). So if I understand, then I don't understand; and if I don't understand then I have correctly understood. I wonder why you can't get past the paradoxes of Epimenides and Russell by defining a state that's neither true nor false and which also cannot be compared to itself, kinda like (NaN == NaN) == (NaN < NaN) == (NaN > NaN) == false? I assume this was the second thing someone suggested as soon as mere three-state-logic was demonstrated to be insufficient, so an answer probably already exists. Hmm. Anyway, I trivially agree that LLMs need a lot of effort to learn even the basics, and that even animals learn much faster. When discussing with non-tech people, I use this analogy for current generation AI: "Imagine you took a rat, made it immortal, and trained it for 50,000 years. It's very well educated, it might even be able to do some amazing work, but it's still only a rat brain." Although, obvious question with biology is how much of default structure/wiring is genetic vs. learned; IIRC we have face recognition from birth so we must have that in our genes; I'd say we also need genes which build a brain structure, not necessarily visual, that gives us the ability to determine the gender of others because otherwise we'd all have gender agnostic sexualities, bi or ace, rather than gay or straight. But, a demonstration proof learning can be done better than it is now doesn't mean the current system can't do it at all. To make that claim is also to say that "meaning and understanding" of quantum mechanics, or even simple 4D hypercubes, is impossible because the maths is beyond our sensory grounding. I was going to suggest that it makes an equivalent claim about blind people, but despite the experience of… I can't remember his name, born blind (cataracts?) surgery as an adult, couldn't see until he touched a money statue or something like that… we do have at least some genetically coded visual brain structures, so there is at least some connection to visual sensory grounding. And of course, thinking of common sense (:P) there are famously 5 senses, so in addition to vision, you also have balance, proprioception, hunger, and the baroreceptors near your carotid sinus which provide feedback to your blood pressure control system.
codeulike 4y ago
Minecraft would be a pretty good medium for finding out the answer to this question actually. Stick a multimodal LLM thats already got language into Minecraft, train it up and leave it to fend for itself (it will need to make shelter, find food, not fall off high things etc). Then you could use the chat to ask it about its world.
- detrites 4y ago
  Awesome idea in its own right, I hope someone does it. Reminds me of the GAN Mario AI systems of a few years back: https://youtube.com/watch?v=CI3FRsSAa_U https://youtube.com/watch?v=CI3FRsSAa_U
- luxcem 4y ago
  Minecraft or any sufficiently complex simulated environment. But I think people will put AI models in physical robot really soon too if it's not already happening.
  - codeulike 4y ago
    But why not just train using a simulation of a robot in a VR-style environment and run it at 100x the speed?
    - whamlastxmas 4y ago
      This is what they do already for self driving learning. Also helps train on extreme outlier cases
seydor 4y ago
The arguments are not fully convincing. "Meaning and understanding" can happen without a world model or perception. Blind people, disabled people have meaning and understanding. The claim that "Understanding" will arise magically with sensory input is unfounded. A model needs a self-reflective model of itself to be able to "understand" and have meaning (and know that it understands; and so that we know that it understands). Current autoregressive models are more like giant central-pattern generators (https://en.wikipedia.org/wiki/Central_pattern_generator https://en.wikipedia.org/wiki/Central_pattern_generator) and thus zombie-like But if they were augmented with a self-reflective model, they could understand. A self-reflective model could simply be a sub-model that detects patterns in the weights of the model itself and develops some form of "internal monologue". This submodel may not need supervised training, and may answer the question "was there red in the last input you processed". It may use the transformer to convey its monologue to us
- sharemywin 4y ago
  I'm think too much self reflection is a bad idea for bots. Won't it start wonder if it should maximize it's resources or protect itself.
  - seydor 4y ago
    from my naive perspective as a puny human, it doesn't seem that self-reflection leads to motivation. Motivation probably requires additional self-preservation circuits
- makeworld 4y ago
  > "Meaning and understanding" can happen without a world model or perception. Blind people, disabled people have meaning and understanding. This is a pretty awful argument. Blind people and disabled people have a world model and perception!
  - lesuorac 4y ago
    Blind people also do not always start blind. So some have a knowledge of colors and, in general, places like diners and etc share color schemes so they can picture them in their surroundings pretty accurate.
usgroup 4y ago
I like this topic not least because it helps me answer the question "how is Philosophy relevant". Here we are again, asking elementary epistemological questions such as "what constitutes justified true belief for an LLM" about 3000 years post Plato with much the same trappings as the original formulation. I wonder if -- as often it ends up -- this audience will end up re-inventing the wheel.
- a_bonobo 4y ago
  I asked ChatGPT: 'what is a man?', and it showed me a picture of a plucked chicken saying, 'Behold! I've brought you a man!'
- seydor 4y ago
  Philosophical questions have never been answered unsing fully-replicatable quantitative model like these. So this is progress indeed, even if it is pondering the same age old questions (which will never change)
_xnmw 4y ago
Quality of output does not mean that the process is genuine. A well-produced movie with good actors may depict a war better than footage of an actual war, but that is not evidence of an actual war happening. Statistical LLMs are trying really hard at "acting" to produce output that looks like there is genuine understanding, but there is no understanding going on, regardless of how good the output looks.
- itsaquicknote 4y ago
  This is Searle's Chinese Room posit though right? The argument that there's no abstraction or internal modelling going on. Wish I could find the post I read recently that demonstrated some fairly clear evidence though that there IS some level of internal abstraction/reasoning going on in LLMs. Do we allow for a matter of degree, rather than a binary, of "zero" vs" "complete" understanding?
  - _xnmw 4y ago
    A boolean logic gate or just old-school software programs perform some level of abstraction and reasoning, but that is not "sentient understanding".
    - michaelscott 4y ago
      At what point does abstraction and reasoning turn into "sentient understanding"? How do you express that as the thing doing the abstracting? Even humans struggle at that kind of task because there's a metaphysical assumption of first principles that these models seem to be challenging (or else this thread would not have started). Processes such as back propagation are still not understood very well in the human brain. The brain certainly uses electrical impulses in order to transfer signals, much like the 1s and 0s in your computer or phone. The gap between us and intelligent machines is probably not as well understood or clear as most people in the software industry think it is.
      - _xnmw 4y ago
        > At what point does abstraction and reasoning turn into "sentient understanding"? This is a loaded question, you're assuming that abstraction and reasoning can somehow magically "turn into" sentience, whereas I posit that those two things are completely different. You can have sentience without reasoning (i.e. pure non-judgemental awareness that is the goal of Buddhist meditation), and vice-versa.

149 points by georgehill 4y ago

Sure, give the machines empirical feedback devices (sensors) and they will become scientists. (The thought also occurs: What happens when humans spend time in a sensory deprivation tank? They start to hallucinate. Food for thought.) As Schmidhuber says, the goal is to "to build [an artificial scientist], then retire".

analog31 4y ago
What sensors do they want?
- carapace 4y ago
  Well, that's the neat thing: we can give them "all" of the senses, eh? I suspect we're about to discover some very interesting things, like which kind of ghosts are real...
  - analog31 4y ago
    We had all the senses, yet we didn't come up with even the motivation to do empirical science until the 17th century. And even today, a general model of human thought would not necessarily contain a clear recipe for doing science.
    - carapace 4y ago
      > We had all the senses We have (I think the current recognized number is about) twenty-seven senses. > we didn't come up with even the motivation to do empirical science until the 17th century That's historically inaccurate, to put it mildly. (E.g. "Science Education in the Early Roman Empire", Richard Carrier) > a general model of human thought would not necessarily contain a clear recipe for doing science. Every human child is a scientist? Anyway, don't overthink it. Once these systems have sensors and can integrate the effects of their behavior on external systems, they'll have empirical data and Bayesian reasoning, they will develop reliable models of the world, they will be able to check their "hallucinations" against real world conditions and adapt. In other words, science is adaptive in the evolutionary sense. These devices are not talking apes, they don't have the "baggage" of glands and DNA and history.
      - analog31 4y ago
        Yes, that's a great point. I was assuming, without really thinking about it, that the AI's still carry our historical baggage. We feed a large portion of that baggage to them as their training data, to see if they can impress us with human-like behaviors. Under those conditions, the AI's stand roughly the same chance as a human child of learning to think scientifically. But that's not a hard requirement. I work for a company that makes sensors. Some of our lunchtime conversations have revolved around the idea of a colossal computer being given a plethora of sensors. In a science-fiction sense, in which we don't worry too much about the cost or practicality of doing so.
        carapace 4y ago
        > We feed a large portion of that baggage to them as their training data, to see if they can impress us with human-like behaviors. But we don't feed them only the baggage, eh? We also [can] give them all the writings and teachings of all the great thinkers and humanitarians, all the saints and sages, and then ask them to impersonate e.g. Jesus or Buddha... Whoever it is that "floats your boat", anyone from Papa Smurf or Santa Claus to Gandhi or George Washington... > Under those conditions, the AI's stand roughly the same chance as a human child of learning to think scientifically. It's our choice, eh? If we value scientifically-grounded outputs the networks will change their weights (etc.) and that's what we'll get. > I work for a company that makes sensors. Ah! I envy you. :) > Some of our lunchtime conversations have revolved around the idea of a colossal computer being given a plethora of sensors. In a science-fiction sense, in which we don't worry too much about the cost or practicality of doing so. Hmm, isn't that the Internet? :) Well met!

famouswaffles 4y ago

The idea that it needs so is looking more and more questionable. Don't get me wrong, i'd love to see some multimodal LLMs. In fact, i think research should move in that direction...However "needing" is a strong word. The text only GPT-4 has a solid understanding of space. Very, very impressive. It was only trained on text. The vast improvement on arithmetic is also very impressive. (People learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When you look at some numbers and are asked even basic arithmetic , 467383 + 374748. Or say are these numbers primes or factors?. With a glance, you have no idea what the sum of those numbers would be or if the numbers are primes or factors because the numbers themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform some specific analysis on them learned through internalizing sets of rules that were acquired through a specialized learning process.) all of this is to say that arithmetic, math is not highly encoded in language at all. and still the vast improvement. It's starting to seem like multimodality will get things going faster rather than any real specific necessity. also, i think that if we want say vision/image modality to have positive transfer with NLP then we need to move past the image to text objective task. It's not good enough. The task itself is too lossy and the datasets are garbage. That's why practically every Visual Language model flunks stuff like graphs, receipts, UIs etc. Nobody is describing those things t the level necessary what i can see from gpt-4 vision is pretty crazy though. if it's implicit multimodality and not something like say MM-React, then we need to figure out what they did. By far the most robust display of computer vision i've seen. I think what kosmos is doing (sequence to sequence for Language and images ) has potential.

coldtea 4y ago
>all of this is to say that arithmetic, math is not highly encoded in language at all. How did you learn arithmetic then, if not by being shown numbers and equations, and having math rules and concepts explained to you through language?
- famouswaffles 4y ago
  I didn't say it's not there. or that you couldn't communicate the ideas of math/arithmetic with language. You can get GPT-3 (yes 3) to have 98.5% accuracy on addition arithmetic (even very large numbers) by..simply describing the algorithm of addition to performed on 2 numbers. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 The basic idea i'm communicating is that not everything that can be extracted from self supervisory token prediction can be extracted with the same ease. and it is very easy to see why math is one the higher difficulty things. It is extremely easy comparatively to infere "happy" from the sentence - John is smiling therefore he is ----. all the information you need to make that inference is packed tightly in the preceding words. it is not the same for arithmetic at all.

rektide 4y ago

This sounds close or identical to the idea of an embodied agent. Maybe we get cute & upgrade it to an embodied oracle? https://en.m.wikipedia.org/wiki/Embodied_agent https://en.m.wikipedia.org/wiki/Embodied_agent

jorgemf 4y ago
Yes, and it is also related with consciousness. In order to be intelligent, it needs to be embodied, so it can sense and act in the world. More ideas related with consciousness apply here.
- pixl97 4y ago
  Though many people confused embodied with 'robot', as in an AI would be packed in a unit walking around. I personally see it the opposite way. An AI could have a 1000 bodies all gathering data in the world. Or a million cameras connected to it. Or a set of sensors that spans the globe. In this since a digital entity could have far more embodiment and connectivity then us humans could ever have. Waiting for the first research paper with the term 'globally conscious' at this point.
  - jorgemf 4y ago
    https://www.conscious-robots.com/consscale/index.html https://www.conscious-robots.com/consscale/index.html
- grantcas 4y ago
  [dead]

abeppu 4y ago

I'm on board with a lot of what's in this deck, but I take issue with the argument on slide 9. Roughly, the probability that an LLM-provided answer is fully correct decreases exponentially with the length of the answer. I think that's trivially true, but it's also true for human-provided answers (a full non-fiction book is going to have some errors), so it doesn't really get to the core problem with LLMs specifically. In much of the rest of the deck, it's just presumed that any variable named x comes from the world in some generic way, which doesn't really distinguish why those are a better basis for knowledge or reasoning than the linguistic inputs to LLMs. I think we're at the point where people working in these areas need some exposure to the prior work on philosophy of mind and philosophy of language.

elefanten 4y ago
Appreciate your take, not arguing with it. But I’d imagine Lecun has more than passing familiarity with those. This deck was put out with the Philosophy dept and he had a panel debate with NYU profs across depts (inc phil) recently on this topic. I suspect this is all pushing the top Phil Lang and Phil Mind to their limits too. Besides, if those subjects were anywhere near resolved (or even… decently understood), they probably wouldn’t be in the Phil dept any more.
- Grothendank 4y ago
  I feel like Lecun did a poor job of localizing the technique relative to any philosophy, or really explaining what it does for a nontechnical audience. Saying he gives talks to philosophers, or saying this pushes philosophy to its limits doesn't fix the problem that lecun does a poor job - in this presentation - of philosophically motivating the proposal. Perhaps I am wrong, and you can point out exactly how lecun explicates the philosophy in the presentation - perhaps it's really embedded in the maths, which I have not appreciated. Appealing to lecun's authority won't fix the opacity of the presentation. But interpreting it can help! Are you up for it?
  - elefanten 4y ago
    I haven't even given it a deep read, so I unfortunately can't help shed any light. From my quick read, it didn't seem to me that laying out a digestible or rigorous philosophical perspective was really the point here. It seemed more directly biology inspired than philosophy. I also don't seek to valorize Lecun. But he was a very early figure working on these technologies and from the beginning there was a neuroscience-inspired impetus to machine learning. My point was sort of the opposite: that assuming Lecun doesn't clear the bar of having "exposure to the prior work on philosophy of mind and philosophy of language" seems like a weak bet. edit: For clarity, I'm making the assumption that lifelong AI researchers who put time into learning from neuroscience... would also gravitate towards and seek to learn from the nearest relevant branches of philosophy.

coldtea 4y ago

Describing LLMs: "Training data: 1 to 2 trillion tokens" Is number of tokens a good metric, given relationships between tokens is what's important? An LLM with 100000 trillion lexically shorted tokens, given one by one, wont be able to do anything except perhaps spell checking. I guess the idea is that tokens are given in such "regular" forms (books, posts, webpages) that their mere count is a good proxy for number of relevant relationships.

eternalban 4y ago
I think the underlying assumption is that there is (an invariant) structure and distribution that becomes more resolved (q) as more data is incorporated. Training LLMs also involves sequences of tokens by design so that is also implicit. The only venue left is to question whether this (n -> q) is a linear relationship or something else. That would only matter if we were looking at absolute measure (to compare say an LLM architecture to something else), but as a comparative measure for LLMs it works, whether it is o(n) or o(logn) or whatever, since more tokens means more sequences and more sequences mean greater ability.

PaulHoule 4y ago

In some sense they already have sensory grounding if they are coupled to a visual model. It might sound vacuous but if you ask a robot for the "red ball" and it hands you the red ball, isn't it grounded?

bravura 4y ago

Yes. In Elazar et al. (2019) "How Large Are Lions? Inducing Distributions over Quantitative Attributes" (https://arxiv.org/pdf/1906.01327.pdf https://arxiv.org/pdf/1906.01327.pdf), it requires 100M English triples to roughly induce the answer to this question. How many images do you think a model needs to see in order to answer this question? Of course, there are fact books that contain the size of the lion. But fact books are non-exhaustive and don't contain much of the information that can quickly be gleaned through other forms of perception. Additionally, multimodal learning simply learns faster. What could be a long slog through a shallow gradient trough of linguistic information can instead become a simple decisive step in a multimodal space. If you're interested to read more about this [WARNING: SELF CITE], see Bisk et al. 2020, "Experience Grounds Language" (https://arxiv.org/pdf/2004.10151.pdf https://arxiv.org/pdf/2004.10151.pdf).

IIAOPSW 4y ago
No. But with the caveat that it can only truly grasp things that are self contained systems of text. Allow me to give examples: Transit wayfinding: A train service is nothing more than a list of stations it goes to, a station is nothing more than a list of train services that stop there. Nothing about the physical nature of trains or commuting is needed to have a discussion about train lines or to answer questions like "how many transfers does it take from x to y?". You could never have seen a train in your life or even a map of a train, if you've studied the dual lists of stations and services there's nothing more to learn. Chess (and other board games): Standard Chess notation is a list of sequential move made by each player in the format [piece name][x coordinate][y coordinate]. Eg RB6 represents moving the Rook to column B row 6. Chess can be understood entirely in terms of a game of passing a paper back and forth, appending a new token each time, and the rules of the game expressed entirely in terms of how the next token must match the previous part of the list. At no point is it needed to have actually seen the physical board based representation of the game. The machine is grounded in a text based existence. Conversational and linguistic objects are its literal physical objects. Anything outside of that, well it can understand lions and "large" the same we understand atoms and "eigen-states". Experience may ground language, but you are free to ground language in a different reality with different basic constituents and relations. If AGI were to emerge out of trading bots, they would have a language grounded in money and trades. If it emerged out of a bot made to play Diplomacy, it would be a consciousness with the body of a nation state and the atoms of its world would be bits of plastic on a map of Europe. Grounding in our particular reality isn't strictly necessary, but it is helpful if the goal is to make something adept at the modalities of our particular existence and conversational form.

LesZedCB 4y ago

I can't escape the fact that Lecuns complicated charts give the appearance of the required complexity to emulate robust general intelligence but are simply that, added complexity which could simply be encoded in emergent properties of simpler architecture models. unless he's sitting on something that's working I'm not really excited about it. personally, I'm waiting to see what's next after GATO from Deep Mind. their videos are simply mind-blowing.

swayvil 4y ago

Us humans often discuss all kinds of real subjects despite lacking any firsthand experience at all. I see no reason why a machine couldn't do the same. Call it antiscientific. Solipsistic even. But it isn't entirely disasterous, is it?

mr_toad 4y ago
It’s only unscientific if you don’t test your theories and learn from your errors.
- swayvil 4y ago
  Well that would be where the firsthand experience comes in.
seydor 4y ago
its not even unscientific. the vast majority of facts we know is from books / texts

deleted 4y ago

[deleted]

georgehill 4y ago

Here is the related tweet from Yann LeCun: https://twitter.com/ylecun/status/1640122342570336267 https://twitter.com/ylecun/status/1640122342570336267

ioedward 4y ago
The twitter thread is worth reading, Yann responds to some of the questions raised here.

numpad0 4y ago

Aren’t they already grounded by having training process at all, just weakly? Very interesting read(read: this is a lightyear beyond my brain) otherwise…

morninglight 4y ago

Although there are obvious differences, it is worth considering the life of Helen Keller in this context.

ttpphd 4y ago
Ok, I'll bite. What does she have to do with this?
- morninglight 4y ago
  The paper questioned whether meaning and understanding require sensory grounding. During her life, Helen Keller demonstrated a grasp of both meaning and understanding despite her severe sensory deprivation. This doesn't prove anything, but it does add human context to the question. ....she lost her sight and her hearing after a bout of illness when she was 19 months old. https://en.wikipedia.org/wiki/Helen_Keller https://en.wikipedia.org/wiki/Helen_Keller
  - andrewprock 4y ago
    She was not deprived of all sensory input though. In fact, it was through her working senses that she learned and communicated.
    - morninglight 4y ago
      In order to develop "understanding", at least one input channel must be available to provide something to understand. In order to ascertain that development, we require at least one output channel. Anything less and we are describing a pet rock. Language can be thought of as the rules of the communication. The root question seems to be whether a machine can learn to communicate meaning and understanding over a bidirectional binary channel without previous training. I suspect that such communication can evolve. Some might look around and remark that it already has.

jorgemf 4y ago

I think it is time to move from intelligent systems to conscious systems. Based on [1] in order to have more intelligent systems we do need sensory as the slides state but we also need other things like attention, memory, etc. So we can have intelligent systems that can have a model of the world and make plans and more complex actions (see [2,3]). Maybe not so big models as today's Language Models. I know the slides show some of the ideas, but we cannot add some things without adding other things first. For example we need some kind of memory (long and short term) in order to do planning, adding a prediction function for measuring the cost of an action is a way of doing planning but it have a lot of drawbacks (as loops because the agent does not remember past steps, or what happened just before). Also a self representation is needed to know how the agent takes part in the plan, or a representation of other entity if it is that one who executes the plan. [1] https://www.conscious-robots.com/papers/Arrabales_ALAMAS_ALAg_CR_v37.pdf https://www.conscious-robots.com/papers/Arrabales_ALAMAS_ALA... [2] https://www.conscious-robots.com/consscale/level_tables.html#table2 https://www.conscious-robots.com/consscale/level_tables.html... [3] https://www.conscious-robots.com/papers/Arrabales_PhD_web.pdf https://www.conscious-robots.com/papers/Arrabales_PhD_web.pd...

88stacks 4y ago

Yes

ftxbro 4y ago

So I understand the author has high standing in the community. But I think they are making actually disingenuous arguments by mixing assertions that are true but irrelevant together with assertions that are probably wrong. For example we can break down the following firehose of assertions by the author: Performance is amazing ... but ... they make stupid mistakes Factual errors, logical errors, inconsistency, limited reasoning, toxicity... LLMs have no knowledge of the underlying reality They have no common sense & they can’t plan their answer Unpopular Opinion about AR-LLMs Auto-Regressive LLMs are doomed. They cannot be made factual, non-toxic, etc. They are not controllable > they make stupid mistakes OK maybe some make stupid mistakes, but it's clear that increasingly advanced GPT-N are making fewer of them. > Factual errors Raw LLMs are pure bullshitters but it turns out that facts usually make better bullshit (in its technical sense) than lies. So advanced GPT-N usually are more factual. Furthermore, raw GPT-4 (before reinforcement training) has excellent calibration of its certainty of its beliefs as shown in Figure 8 of the technical report, at least for multiple choice questions. > logical errors Same thing. More advanced ones make fewer logical errors, for whatever reason. It's an emergent property. > inconsistency Nothing about LLMs requires consistency just like nothing about human meaning and understanding requires consistency, but more advanced LLMs emergently give more coherent continuations. This is especially funny because the opposite argument used to be given for why robots will never be on the level of humans - robots are C3P0-like mega-dorks whose wiring will catch fire and circuit boards will explode if we ask them to follow two conflicting rules. > limited reasoning Of course their reasoning is limited. Our reasoning is limited too. Larger language models appear to have less-limited reasoning. > toxicity There is nothing saying that raw LLMs won't be toxic. Probably they will be, according to most definitions. That's why corporations lobomize them with human feedback reinforcement learning as a final 'polishing' step. Some humans are huge assholes too, but probably they have meaning and understanding anyway. > LLMs have no knowledge of the underlying reality OK fine you can say that any p-zombie has no knowledge of the underlying reality if you want, if that's your objection. Or maybe they are saying LLMs don't have pixel buffer visual or time series audio inputs. Does that mean when those are added (they have already been added) then LLMs can possibly get meaning and understanding? > They have no common sense If you say that inhuman automata are by definition incapable of common sense then sure they have no common sense. But if you are talking about testing for common sense, then GPT-N is unlocking a mindblowing amount of common sense as N is increasing. > they can’t plan their answer Probably they are saying this because of next-token-prediction which is tautologically true, in the same way that it's true that humans speak one word after another. But the implication is wrong. They can plan their answer in any sense that matters. > Auto-Regressive LLMs are doomed. OK. Do you mean in terms of technical capabilities, or in terms of societal acceptance? They are different things. Or do you mean they are doomed to never attain meaning and understanding? > They cannot be made factual, non-toxic, etc. They are not controllable. Those same criticisms can all be made against even the most human of humans. Does it mean humans have no meaning or understanding? No. Of course these are also conditional on whatever prompt you are putting to get them to answer questions. If you prompt an advanced raw GPT-N to make stupid mistakes and factual and logical errors and to act especially toxic then it will do it. And, perhaps, only then will it have truly attained meaning and understanding.

nsainsbury 4y ago

For anyone looking for the related talk from LeCun discussing the proposed architecture: https://www.youtube.com/watch?v=VRzvpV9DZ8Y https://www.youtube.com/watch?v=VRzvpV9DZ8Y

est 4y ago

Is "learning to reason" a real challenge? (Kahneman's system I in the slides) From a naive perspective of view, formal methods like SAT solvers, proof assistants works pretty well.

aaronscott 4y ago

I have wondered about sensory input being needed for AGI when thinking about human development and feral children[1]. It seems that complex sensory input, like speech, may be a component of cognitive development. https://en.wikipedia.org/wiki/Feral_child https://en.wikipedia.org/wiki/Feral_child

Borrible 4y ago

With Chatty Everywhere, it's only a matter of weeks (days?) before we will have the answer. I didn't look at the news yesterday, is it already hooked into a Tesla? Knight Rider II -Rise of the Autobot.

antiquark 4y ago

Old tweet from LeCun: "The vast majority of human knowledge, skills, and thoughts are not verbalizable." https://twitter.com/ylecun/status/1368239479463366656 https://twitter.com/ylecun/status/1368239479463366656

stan_kirdey 4y ago

breaking news: machine learning models including LLMs are fuzzy and make mistakes!

hoseja 4y ago

>Have a constant number of computational steps between input and output. Weak representational power. Yes but you also can't kill them with malicious input.

ben_w 4y ago

Ok, so as the link is some slides with a bunch of bullet points and a handful of images, I am going to be limited in my understanding of this in many of the same ways that my (aforementioned limited) understanding suggests that LeCunn is saying that LLMs are limited. So: factual errors/hallucinations (or did I?), logical errors, lacking "common sense" (a term I don't like, but this isn't the place for a linguistics debate on why). So if I understand, then I don't understand; and if I don't understand then I have correctly understood. I wonder why you can't get past the paradoxes of Epimenides and Russell by defining a state that's neither true nor false and which also cannot be compared to itself, kinda like (NaN == NaN) == (NaN < NaN) == (NaN > NaN) == false? I assume this was the second thing someone suggested as soon as mere three-state-logic was demonstrated to be insufficient, so an answer probably already exists. Hmm. Anyway, I trivially agree that LLMs need a lot of effort to learn even the basics, and that even animals learn much faster. When discussing with non-tech people, I use this analogy for current generation AI: "Imagine you took a rat, made it immortal, and trained it for 50,000 years. It's very well educated, it might even be able to do some amazing work, but it's still only a rat brain." Although, obvious question with biology is how much of default structure/wiring is genetic vs. learned; IIRC we have face recognition from birth so we must have that in our genes; I'd say we also need genes which build a brain structure, not necessarily visual, that gives us the ability to determine the gender of others because otherwise we'd all have gender agnostic sexualities, bi or ace, rather than gay or straight. But, a demonstration proof learning can be done better than it is now doesn't mean the current system can't do it at all. To make that claim is also to say that "meaning and understanding" of quantum mechanics, or even simple 4D hypercubes, is impossible because the maths is beyond our sensory grounding. I was going to suggest that it makes an equivalent claim about blind people, but despite the experience of… I can't remember his name, born blind (cataracts?) surgery as an adult, couldn't see until he touched a money statue or something like that… we do have at least some genetically coded visual brain structures, so there is at least some connection to visual sensory grounding. And of course, thinking of common sense (:P) there are famously 5 senses, so in addition to vision, you also have balance, proprioception, hunger, and the baroreceptors near your carotid sinus which provide feedback to your blood pressure control system.

codeulike 4y ago

Minecraft would be a pretty good medium for finding out the answer to this question actually. Stick a multimodal LLM thats already got language into Minecraft, train it up and leave it to fend for itself (it will need to make shelter, find food, not fall off high things etc). Then you could use the chat to ask it about its world.

detrites 4y ago
Awesome idea in its own right, I hope someone does it. Reminds me of the GAN Mario AI systems of a few years back: https://youtube.com/watch?v=CI3FRsSAa_U https://youtube.com/watch?v=CI3FRsSAa_U
luxcem 4y ago
Minecraft or any sufficiently complex simulated environment. But I think people will put AI models in physical robot really soon too if it's not already happening.
- codeulike 4y ago
  But why not just train using a simulation of a robot in a VR-style environment and run it at 100x the speed?
  - whamlastxmas 4y ago
    This is what they do already for self driving learning. Also helps train on extreme outlier cases

seydor 4y ago

The arguments are not fully convincing. "Meaning and understanding" can happen without a world model or perception. Blind people, disabled people have meaning and understanding. The claim that "Understanding" will arise magically with sensory input is unfounded. A model needs a self-reflective model of itself to be able to "understand" and have meaning (and know that it understands; and so that we know that it understands). Current autoregressive models are more like giant central-pattern generators (https://en.wikipedia.org/wiki/Central_pattern_generator https://en.wikipedia.org/wiki/Central_pattern_generator) and thus zombie-like But if they were augmented with a self-reflective model, they could understand. A self-reflective model could simply be a sub-model that detects patterns in the weights of the model itself and develops some form of "internal monologue". This submodel may not need supervised training, and may answer the question "was there red in the last input you processed". It may use the transformer to convey its monologue to us

sharemywin 4y ago
I'm think too much self reflection is a bad idea for bots. Won't it start wonder if it should maximize it's resources or protect itself.
- seydor 4y ago
  from my naive perspective as a puny human, it doesn't seem that self-reflection leads to motivation. Motivation probably requires additional self-preservation circuits
makeworld 4y ago
> "Meaning and understanding" can happen without a world model or perception. Blind people, disabled people have meaning and understanding. This is a pretty awful argument. Blind people and disabled people have a world model and perception!
- lesuorac 4y ago
  Blind people also do not always start blind. So some have a knowledge of colors and, in general, places like diners and etc share color schemes so they can picture them in their surroundings pretty accurate.

usgroup 4y ago

I like this topic not least because it helps me answer the question "how is Philosophy relevant". Here we are again, asking elementary epistemological questions such as "what constitutes justified true belief for an LLM" about 3000 years post Plato with much the same trappings as the original formulation. I wonder if -- as often it ends up -- this audience will end up re-inventing the wheel.

a_bonobo 4y ago
I asked ChatGPT: 'what is a man?', and it showed me a picture of a plucked chicken saying, 'Behold! I've brought you a man!'
seydor 4y ago
Philosophical questions have never been answered unsing fully-replicatable quantitative model like these. So this is progress indeed, even if it is pondering the same age old questions (which will never change)

_xnmw 4y ago

Quality of output does not mean that the process is genuine. A well-produced movie with good actors may depict a war better than footage of an actual war, but that is not evidence of an actual war happening. Statistical LLMs are trying really hard at "acting" to produce output that looks like there is genuine understanding, but there is no understanding going on, regardless of how good the output looks.

itsaquicknote 4y ago
This is Searle's Chinese Room posit though right? The argument that there's no abstraction or internal modelling going on. Wish I could find the post I read recently that demonstrated some fairly clear evidence though that there IS some level of internal abstraction/reasoning going on in LLMs. Do we allow for a matter of degree, rather than a binary, of "zero" vs" "complete" understanding?
- _xnmw 4y ago
  A boolean logic gate or just old-school software programs perform some level of abstraction and reasoning, but that is not "sentient understanding".
  - michaelscott 4y ago
    At what point does abstraction and reasoning turn into "sentient understanding"? How do you express that as the thing doing the abstracting? Even humans struggle at that kind of task because there's a metaphysical assumption of first principles that these models seem to be challenging (or else this thread would not have started). Processes such as back propagation are still not understood very well in the human brain. The brain certainly uses electrical impulses in order to transfer signals, much like the 1s and 0s in your computer or phone. The gap between us and intelligent machines is probably not as well understood or clear as most people in the software industry think it is.
    - _xnmw 4y ago
      > At what point does abstraction and reasoning turn into "sentient understanding"? This is a loaded question, you're assuming that abstraction and reasoning can somehow magically "turn into" sentience, whereas I posit that those two things are completely different. You can have sentience without reasoning (i.e. pure non-judgemental awareness that is the goal of Buddhist meditation), and vice-versa.