31 ms·

What Does It Mean for AI to Understand?

99 points by theafh 5y ago

rbanffy 5y ago
What does "understanding" mean?
- AndrewOMartin 5y ago
  Very droll, but notice that I can write the phrase "Don't make snarky comments" without us having previously agreed on a formal definition of "snark".
  - selestify 5y ago
    Right, so you’ve demonstrated that the phenomenon exists. Now, what defines that phenomenon, exactly?
    - gooseus 5y ago
      I'd say "understanding" is less one thing and more of a collection of capabilities which work together to allow understanding to "emerge". One of those capabilities would be the ability to contextual an object/statement within multiple frames of reference while also being able to compare and contrast the different instances of those contextualized objects/statements. This is what allows a child to identify a bird as any number of physical specimens of different species (chicken, goose, eagle, sparrow), while also identifying cartoon depictions that talk and simple drawings (Twitter icon) as birds as well... while also "understanding" that while the Twitter icon can be called a bird, it is not actually a real bird ("Ceci n'est pas une pipe") and it would not be expected to sing or fly like a backyard sparrow (unless it was animated, which would make still make sense to a child). I think this also what rise to our ability to "understand" jokes, puns, and other turns of phrase - "I just flew in from Boston, and boy are my arms tired!" - this dumb joke requires a number of concepts that need to be contextualized before you can "get" the absurdity of stating that a human might tire out their arms while flying... like you might think a bird would.
  - TheOtherHobbes 5y ago
    That's because snark is a pre-installed module in the Western Human Online Communication Library Codebase. The code for "understanding" is part of Generic Human Firmware codebase and is tightly integrated with the OS. Unfortunately it hasn't been open sourced.
    - PaulDavisThe1st 5y ago
      It's totally open source, just written in a language we don't (yet) understand.
      - teddyh 5y ago
        I.e. a binary blob.
- robbedpeter 5y ago
  What it means for humans to understand has been posited by Jeff Hawkins as a combination of synaptic links within and between cortical columns, resulting in a physical neural construct that activates when stimulated past a sufficient threshold of inbound signals. Constructs can suppress nearby clusters of neurons, or contribute an increase in readiness to signal, or self modulate the activation threshold of constructs within the cortical column. The findings are consistent with current understanding of neuroscience, and align with discoveries such as grid cells. They also provide a basis for explaining what's actually happening in the brain with phenomena like memory palaces, rapid and efficient physical control of the body, combining sense modalities such as sight and touch when catching a ball, and so on. Understanding is what happens when your brain has developed a neural structure such that it's able to predict events successfully based on the thing that was understood.
  - jjbinx007 5y ago
    Children learn thousands of words growing up not by having the definition read to them from a dictionary but by inferring their meaning based on context. Likewise we learn general concepts and they can be applied to a wide range of scenarios. I can't see any other way AI could learn unless it mimicks our own.
- motohagiography 5y ago
  This is the important question. After reading Ted Chiang's story "Understand" about this question, a simple working definition could be that to understand something you apprehend the domain of the subject from its substrate. Hence Feynman's "what I cannot create I do not understand," as well. In this sense, an AI could be said to Understand language if it used it as one of a selection of tools to operate on itself, a peer or other being, or its environment.
- therobot24 5y ago
  Situational Awareness is largely separated into 3 levels -- first level (perception), 2nd level (comprehension),and 3rd level (prediction) Understanding while not completely described by situational awareness definitely has some relationship to it and you could probably use similar constructs for defining it
hpoe 5y ago
When it gets to questions like these I feel that we transcend discussions of technology and end up on questions of philosiphy which aren't going to be going anywhere anytime quick. I also feel that AI should be used to augment not replace human decision making, it seems that where AI shines is problems that are well defined with well defined solutions, and because the AI doesn't get tired, hungry or distracted it can do that really well, but it fails in novel situations[0]. As such it seems to me our best bet is to have the AI provide suggestions rather than have complete control. 0. What is meant by that is a read an article, can't find it now, about using AI to diagnose breast cancer, what they found is that about 90% of the time the AI could accurately check for breast cancer, but the other 10% of the time was an unusual mammogram or something relatively rare, and in those situations the AI would often misdiagnoise.
- deleted 5y ago
  [deleted]
- chermi 5y ago
  AI had meaning that originated from CS theorists before being usurped and losing all meaning. I think tech and associated marketing (which are great, don't get me wrong, I love money!) are the ones at fault here. It should be more of a "philosophical" question, though I'd prefer instead perhaps it be academic. I'm not trying to be rude, but your example of what AI should be is narrow and not very grandiose compared the original meaning. I understand you were talking pretty loosely, so I feel like I'm singling you out but this happened to be where I started typing, sorry! It just reminded of how essentially all conversations about "AI" go. They seem to end up being quite specific, narrow pattern recognition problems at the end of the day. Maybe there's some decision theory on top of it. Maybe if there's enough money /people involved, there's more components, so it's a complicated enough supervised learning problem that it mimics people to a sufficient extent that it looks intelligent enough to make a headline. But it's a copycat, not intelligent. Hey, full circle, Melanie Mitchell! - https://en.m.wikipedia.org/wiki/Copycat_(software) https://en.m.wikipedia.org/wiki/Copycat_(software)
jonplackett 5y ago
Highly recommend A a thousand Brains by Jeff Hawkins if you’re interested in this https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-intelligence/ https://numenta.com/blog/2019/01/16/the-thousand-brains-theo... He basically argues we all have 1000s of little, but interacting models of all sorts of things going on in our brain all at the same time. He calls them reference frames and it’s those that create intelligence. ‘Understanding’ would come naturally out of having those. Fascinating book which I’m probably explaining much less well that he does.
- andyjohnson0 5y ago
  I've not read the book, but from your description it maybe has some similarity to Marvin Minsky’s "Society of Mind". Mind and intelligence as the emergent behaviour of many cooperating/competing systems.
  - jonplackett 5y ago
    yes i think he talks about that. The difference is he's also looking for (and finding) the 'how' in physical brain structure terms, rather than just theory alone.
jonplackett 5y ago
Anyone who has kids and teaches them things will know AI learns and ‘understands’ very differently. I can say something like ‘a tiger is just a lion with stripes’ to a 3 year old and they now ‘understand’ what a tiger is almost as well as if they saw a picture of one. They could definitely identify one from a picture now. This kind of understanding won’t work with an AI because we don’t understand what characteristics it has latched into when identifying a lion. For all we know it’s that the background of each lion image its been trained on has a blue sky. Or the tigers are all looking at the camera. I think the ability to pick apart what you know and learn new things by reasoning about that knowledge is the key to if understanding is taking place.
- ShamelessC 5y ago
  I've done a fair bit of work with multimodal deep learning and I am fairly confident that a DALL-E, CLIP, or NUWA architecture would output/classify those phrases accurately without being trained explicitly on images of Tigers. I see your point however.
  - jonplackett 5y ago
    I’d be interested in where to look for more info on that. Do you think it could work on anything more complex? As I said in comment below I reckon I could make a much more elaborate explanation and still have the kid get it.
    - ShamelessC 5y ago
      https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/ shows a decent ability to generalize to previously unseen concepts. You're correct to consider the complexity of the phrase and just how good humans are at this sort of thing without needing much "training". For now, concepts that aren't explicitly in the training set are effectively composed from those which are. This can lead to some bizarre and outright incorrect results, particularly when it comes to counting objects in a scene or with relative positioning between objects (e.g. a blue box on top of a red rectangle to the left of a green triangle) but it's early days and there's lots of progress happening all the time.
mwattsun 5y ago
I've been using Visual Studio 2022 recently which has AI driven code prediction built into it. Sometimes it predicts what the next thing I will type will be and I merely have to hit tab once or twice to accept it. At no point am I tempted to think Visual Studio understands my code, because it's just code itself. The first time I played a chess game was back in the early 1980's. While it beat me I felt an eerie "presence" in the machine that was sentient. I didn't know then about chess code so it was easier for me to anthropomorphize the machine (but it was the main reason I became interested in computers.) A computer "understanding" the difference between "how do I melt a block of ice" and "how do I melt my lover's icy heart" would be looking at the context and the relation of the words to each other. The computer might also predict I was sad if I asked the latter question. If I were a non-technical user I might think the computer felt empathy and be amazed by it. If I came upon a computer that "understands" I would want to determine if it understands like Richard Feynman or if it understands like my dog. My dog operates on a limited set of patterns, so that seems doable, but on the other hand, I've seen videos and heard stories of dogs exhibiting inventive and creative behavior that is unexpected. One such case is dogs that get lost and manage to find their way home thousands of miles away. tldr; I'm jaded. I know it's buggy code all the way down with computers.
tested23 5y ago
What does it mean for humans to understand? There are many times in the past where i thought i understood something and then i grow older and i see the holes.
- Verdex 5y ago
  Yeah, I feel this question is important to understand before we worry about what it means for the AI to understand. My thought is that we've got three types of "understanding": 1) social understanding 2) intuitive understanding 3) structural understanding Social understanding is something the society we live in knows, but the individual only knows in so far as the individual is doing something to fit in or via peer pressure. So for example, some high latitude countries eat fish for breakfast. Supposedly the statistics show that this helps them be more healthy than countries at similar latitudes which do not eat fish for breakfast ... probably because of problems due to lack of vitamin D due to lack of sunlight for certain parts of the year (the fish oil helps with this). However nobody actually "knows" this. They just eat fish because everyone else eats fish. Intuitive understanding is anything where we start to use flowery language like "experience" or "gut". You're really good at it, but just giving someone a flow chart isn't good enough. They have to have gone through the experience themselves. Driving is a good example. We make people take a test, but if just giving them diagrams and rules was good enough, then we wouldn't need a test where you actually drive and requirements about a certain number of hours of supervised driving. Structural understanding is anything that can be put to rules. So there's a lot of mathematics and algorithm stuff here. A simple example might be playing tic tac toe. The game is simple enough that you can write down a few rules that allow you to never lose. EDIT: My categories don't really answer the question, but they do give profiles and categories to look out for. Social understanding is good because it statistically learns to avoid lethal pitfalls. Like, if there's a dangerous well in the forest that people fall down and die in. A society might start telling people to not go in the forest because other people go in there and die. However, the society doesn't know why this is good advice. Intuitive understanding is good because it allows you to quickly statistically learn how to deal with imperfect and chaotic systems while getting good results. Structural understanding is good because it allows you to break free of the statistics of the previous two understandings. You can get exact results. Also it lets you break free of issues that come from distantly causal action + consequence. A person's intuition might not tell them that dumping toxic waste into the water is a good idea because things don't go bad until a lot of waste has already been dumped. Similarly a society might make a similar judgement if the failure is far enough away from the actions that kick it off. However, if you understand the structural relationships between things then you'll have an idea that toxic waste should not be consumed.
6gvONxR4sf7o 5y ago
Good article. I think winograd/winogrande are super clever (and also kind of a “fun” idea). My personal take is that blanket understanding is too hard of a task to define, so we ought to cheat and talk about types of understanding. In my mind, understanding a thing means not only that you can answer, but also that you can justify your answer. So different kinds of understanding point to different kinds of justifications. In math classes, you’ll be asked not only to state whether a thing is true or false, but also to show that it’s true or false, giving an answer as well as a proof. In a literature class, you don’t just make a point, you also have to support it in natural language. Same with science classes, supporting things with data and logic. The closest we have right now in ML (widely) is statistical measurements on holdout data. It’s 99% accurate on other things, so it’s 99% likely to be right on this too, if this was sampled from something just like the holdout. We also (less widely) have a little cottage industry of post-hoc explanations that try to explain models predictions. I’d love to see models that can do better than post-hoc explanations. I want a model that “understands in terms of predicate logic” that can spit out an answer with a checkable proof. Or “understands in terms of a knowledge graph” or “understands in terms of a set of human anatomy facts” or “understands in terms of natural language arguments” or any number of things that can spit out an answer as well as a justification. Just asking for blanket understanding means we have to define blanket understanding, but there’s a lot of limited understanding that’s still better than what we have today.
- harperlee 5y ago
  Exactly, understanding something requires holding a model (or several) of the thing in the mind and be able to work with that model to some extent. That’s why we dont see statistical tricks as understanding - as soon as trivial errors show that operation in those models is not feasible, we reject it. And that’s why winograd works: it establishes a simmetry that requires a base semantic model outside of the provided text.
- rdedev 5y ago
  The other good thing with requiring proofs is that you can show specify exactly where the proof fails and the one giving the proof can use this feedback to correct their beliefs. I would actually be pretty okay with a model that can do that. Take feedback and correct themselves without resorting to retraining. It would still be pretty far away from a general AI but much much better than a black box.
stakkur 5y ago
For AI to be truly artificial intelligence, shouldn't it be able to define when it understands?
hans1729 5y ago
Mandatory reference to Robert Miles' content on AI safety problems, alignment etc.: https://www.youtube.com/c/RobertMilesAI/playlists https://www.youtube.com/c/RobertMilesAI/playlists
trabant00 5y ago
It's incredible how stupidly powerful the rational part of the human brain is. It has this unlimited capacity to get lost in details. "What does it mean for AI to Understand?" - we keep arguing over definitions and moving the goal posts to make it seem we are an step closer to reaching AI. When my first AI coworker will read the on-boarding docs and start solving Jira issues I will have no doubt we have done it. That simple. Does anybody believe that an entity that would actually develop AI would start selling it? They would keep it for themselves and literally take over the world! Complete domination of the digital realm is one of easiest things an AI could do - I believe a lot easier than driving a car. And that alone would make them God. When the first true AI will be born we will simply live the experience. Imagine being there when we learned to control fire. Would you argue over the definition of it? The size, flame color, temperature and so on? Something that great can not be denied by such small details. LE: what happened to the Touring test? We forgot about it or does ordering things from Amazon when we command smartphone assistants to turn down the lights actually fools us?
- PeterisP 5y ago
  Regarding Turing test, it turned out that fooling people is easier than it might seem, so it's not considered anymore a reasonable qualification for general artificial intelligence as everybody now presumes that it's possible for a system to exist that can beat the Turing test but doesn't even attempt to be intelligent in any reasonable/general way.
  - bryan0 5y ago
    I don't know why this is a common viewpoint. A proper Turing test with trained judges and human subjects who are actually trying to convince the judge they are human still seems like the best test of intelligence IMO.
  - deleted 5y ago
    [deleted]
- SxC97 5y ago
  When my first AI coworker will read the on-boarding docs and start solving Jira issues I will have no doubt we have done it. That simple. It’s not exactly what you’re after, but we’re already pretty close!: https://mobile.twitter.com/gabro27/status/1173547934132178944 https://mobile.twitter.com/gabro27/status/117354793413217894...
throwawayai12 5y ago
I always get a little confused when people quote Oren. He hasn't been involved in meaningful work in AI in decades despite leading a large, well-funded group. But boy howdy can be give talks that sound good to lay people.
cgearhart 5y ago
Ignoring the associate debacle, the characterization of large language models as “stochastic parrots”[1] is the most accurate description I think I’ve ever heard for the capabilities of AI language models. These models don’t understand that a mistake on a Winograd question is not the same as a mistake on a medical diagnosis (as a contrived example). [1] https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922
- visarga 5y ago
  I don't think it's a good name. GPT-3 is more like a dreaming hallucination machine. Humans do nonsensical things in their dreams, same kind of non sequitur as the language models. What GPT-3 lacks is being able to wake up from its disembodied hallucinations, in other words a body and something to motivate its actions.
temptemptemp111 5y ago
It doesn't mean anything; people just don't understand what 'concepts' are anymore because they're so delusional. For AI to understand something means its human inventor / implementer understood it, potentially. The human understood a concept - not the actual thing - and that concept is what you call immaterial. You can point to code or output, but that is related to the concept. "Understanding" is when you stand under a concept - though you could always step out from under it in the case that you lose your understanding or it does not apply - etc. This inability for people to think is becoming hilarious. The metaverse is going to prevail for these damaged people but means nothing to those living in the real world.
nqzero 5y ago
What does it mean for a human to understand ?
- prometheus76 5y ago
  And to extend your question: write something sarcastic on the internet and you'll quickly find out there are multiple layers of "understanding" something.
jll29 5y ago
As an AI professor, I've always held that machines are NOT intelligent (I am prepared to change my position on the day my computer asks me anything surprising that I didn't program it to). But this does not mean we cannot produce operational models of understanding, for example we have models of propositional/logical semantics and discourse such as Lambda Discourse Representation Theory and others, which can compute a formal representation of the meaning structures for a piece of text. These have been used e.g. for answering question, and working in this space has been a lot of fun, and continues to do so. At the moment people talk a lot about "deep" learning (neural networks with more than one hidden layer), but for such models we need to do a lot more work into explainability, because it is too dangerous to use black boxes in real life. We still do not understand the human brain function in any substantial way, and it is perhaps a greater mystery of nature than even cosmology, where at least several competing theories have been posed that can explain parts of the evidence. How are thoughts represented (if that is answerable, it turns out 'Where are thoughts represented?' has proven to be a meaningless question due to the distributed nature of human memory)? What is consciousness? What is a conscience? How do consciousness and intention emerge from materials that are not alive and that have neither consciousness nor intention? How to implement approximate models? (A lot of work to do!)
- kkoncevicius 5y ago
  > How do consciousness and intention emerge from materials that are not alive and that have neither consciousness nor intention? I think the question shoud first be not about "how", but rather - "does consciousness emerge from the materials". Most cultures in the past have maintained ideas about the duality between spirit and matter. Nowadays we are so adept at manipulating matter we always explicitly assume that everything must be material in one way or another. But here is this consciousness thing that does not give in. And there is no angle to it where it can ever give in. If somebody creates a machine and claims it to be conscious nobody can test if that is so. And further - nobody can even test wheather the inventor is conscious himself.
  - marginalia_nu 5y ago
    It seems to me that consciousness emerges when things arranged in some particular way are acted upon some principle, which is true even for simple behavior, like a rock falling to the ground. If you put a sail on a boat, it will catch the wind and move across the water. It's not the sail alone that moves the boat, nor the wind, but their interplay. A sail without wind doesn't get moved, and a wind without anything to catch it doesn't move anything. May be a bit of an unintuitive conclusion, but I think may be that we are catching the consciousness like a sail catches the wind, and this is the same "wind" that we call the forces of nature, that we only can perceive as patterns in how things around us appear to change.
continuational 5y ago
This is a great quote: > When AI can’t determine what ‘it’ refers to in a sentence, it’s hard to believe that it will take over the world.
typon 5y ago
I've come to believe that the answer will be provided by the "free market". When AIs start to replace humans for tasks that require "intelligence" (how we currently define it for humans), then AI will have achieved "understanding". Sure there will be hype driven companies that replace humans with AI for PR purposes, but eventually those will flatten out. Once I start receiving phone calls from AI telemarketers, have a decent conversation with them and can't tell that I just talked to an AI, then AI will have achieved understanding. And so on for other domains in every day life.
chrischapman 5y ago
Machines don't learn. Living things learn. Machines don't understand. Living things understand. Machines 'do' algorithms. Living things 'do' use-cases. Algorithm Definition: A sequence of actions that yields a result. Use-case Definition: A sequence of actions that yields a result of value to a user. The difference between the two is 'of value to a user'. To me, the line between algorithm and use-case is the line between unconsciousness and consciousness. That line pivots around the ability to 'value' something. I doubt we will ever see the I in AI until we build something that can value a result in the same way that you and I do. We need new words to describe what machines do. Using 'learn' or 'understand' seems like anthropomorphism. It's weird that we glibly anthropomorphise when talking about machines but prohibit it when talking about living things. Almost all of our qualities have been inherited from other living things. It seems to me that we should always anthropomorphise when talking about living things and never anthropomorphise when talking about machines. And yet, we always seem to do the opposite. Until we can explain how a machine can 'value' something in the same sense that humans do (or chimps or ducks or caterpillars do), we should avoid anthropomorphic words like 'learn' and 'understand' as it misdirects our efforts. However, I have no idea what else to suggest other than to try to explain how a machine can 'value' something. > IBM’s Watson was found to propose “multiple examples of unsafe and incorrect treatment recommendations.” That won't stop until Watson has the ability to 'value' the result of what it is doing. Watson 'does' algorithms. Watson needs to 'do' use-cases. Once it can 'value' a result in the same way we do, it will correct its mistakes.
- JoeAltmaier 5y ago
  Sounds like a bunch of unsubstantiated claims? So if something does learn, then its a living thing? My AI can learn (see a pattern and repeat it); now its a living thing? Not sure what to do with that.
  - chrischapman 5y ago
    Yep. Totally unsubstantiated. Back of the envelope theory at best. > learn (see a pattern and repeat it) How did your AI 'see' and is 'see a pattern and repeat it' a good enough definition of 'learn'? Surely to 'learn' also means to 'understand'. What did your AI 'understand'? I doubt your AI actually learned anything. It attained zero knowledge in the way a living thing obtains knowledge. It may have stored a result in a database but did it actually understand anything? These are genuine questions as I have no professional knowledge of AI.
swframe2 5y ago
A software and hardware tool that can design and build another software and/or hardware tool to determine the cause and effect rules of any real world activity.
tomthe 5y ago
My understanding of "Understanding": Imagine a photo of a written poem: An image-processing program can "understand" the digital image: It can read the jpg, change the picture completely (e.g.: change the colors slightly), without changing the meaning one level up. But it doesn't understand the characters or words. An OCR program can read the image and "understand" the characters (or the textual representation. It can change the representation completely (save it as UTF-8 or whatever) without changing the meaning one level up. But it doesn't understand the language. GPT-3.... well, let't go directly to humans A human can read the text and understand the words and understand their meanings and what the sentences say. Another human really understands the poem and the subtext another level up. I think understanding always works on different levels and is a part of communication.
- pinouchon 5y ago
  Interestingly, the poet with the highest level understanding can be clueless about UTF-8 or what a pixel is
- criddell 5y ago
  Sounds like you are describing the treachery of images. Ceci n'est pas un poème.
- deleted 5y ago
  [deleted]
vbphprubyjsgo 5y ago
Nothing. Once it has enough neurons it's no different than a human brain in terms of what it can understand. Once it has more it can understand more.
bryan0 5y ago
> Unfortunately, Turing underestimated the propensity of humans to be fooled by machines. Even simple chatbots, such as Joseph Weizenbaum’s 1960s ersatz psychotherapist Eliza, have fooled people into believing they were conversing with an understanding being, even when they knew that their conversation partner was a machine. This is not a Turing test. Or at least not a reasonable one. A reasonable Turing consists of a judge and 2 interfaces, one which chats with a computer and the other which chats with a human. Both the human and computer are trying to convince the judge that they are the human. If the judge cannot determine which is which after an open-ended conversation then the computer passes. There is no reasonable judge which after chatting with a human (who is actually trying to convince the judge they are human) would be unable to differentiate between a human and Eliza or any other chatbot out there.
carapace 5y ago
Prediction. Setting aside the metaphysical questions of subjective "meaning" and "understanding" what else is there? What a system can predict is the measure of its "understanding", surely?
- idiotsecant 5y ago
  Does a mechanical bomb sight 'understand' ballistics and aerodynamics?
  - carapace 5y ago
    By this definition, yes, of course. That's a great example.
sabhiram 5y ago
What does it mean for a child to understand? For a baby? A dog? A command line application? Understanding implies comprehension of some input, to influence a future state. Surely my stateful database understands requests that come to it. It, however will never surprise me with behavior that I would (should) not expect. I suppose if you "understood" the human machine and mind well enough, it would be possible to predict the actions it will carry out.
KhoomeiK 5y ago
People outside the NLP research community need to understand that a language model does nothing more than calculate probabilties for the next word given some context. The way it learns that probabilistic function can be quite complex, involving billions of parameters, but it's still fundamentally the same. Most of the increase in performance in recent years has come from the ability to train LMs that better generalize this probability function from huge text corpora--functions that better interleave between the datapoints it's trained on. Humans use language with purpose, to complete tasks and communicate with others, but GPT-3 has no more goals or desires than an n-gram model from the 90s. LMs are essentially a faculty for syntactically well-formed or intuitive/system 1 language generation, but they don't seem to be much more.
- machiaweliczny 5y ago
  What if to predict next word you need to compress/understand universe in the end? AFAIK we have no clue how exactly NNs pick up/structure information. For me backprop and learning is similar to evolution thus result might be simialar. My knowledge is very limited though but I am happy to read any proofs/insights.
  - KhoomeiK 5y ago
    This is basically my area of research right now, compositional/symbolic representations in neural language models. I'll let you know the answer in a few months :)
wombatmobile 5y ago
> What Does It Mean for AI to Understand? What does it mean for a human to understand?
machiaweliczny 5y ago
BTW humans are generally stupid. We are only smart in collective with all trial/error saved in books, so don't underestimate computers. Also most of progress happens by outliers in humanity so it can be after AIs start to collaborate. Distributed MoE architectures are step in this direction. My prediction: codex and programming/operating APIs will give NNs ultra boost, then programming/math gets automated (hard to master, easy to verify) and rest/us will be history. John Conor is already alive. Mark my words.
mcguire 5y ago
"A Winograd schema, named for the language researcher Terry Winograd, consists of a pair of sentences, differing by exactly one word, each followed by a question. ... In each sentence pair, the one-word difference can change which thing or person a pronoun refers to. Answering these questions correctly seems to require commonsense understanding." I don't buy it. These Winograd schemas are significantly limited and are exactly the sort of thing that "story understanding" systems from the '70s and '80s were designed for.
toisanji 5y ago
a simulation engine in the human mind seems required for understanding: https://blog.jtoy.net/on-the-wondrous-human-simulation-engine/ https://blog.jtoy.net/on-the-wondrous-human-simulation-engin...
nullhack 5y ago
https://archive.md/8rArs https://archive.md/8rArs The page INSISTED on re-directing me to https://www.quantamagazine.org/how-close-are-computers-to-automating-mathematical-reasoning-20200827/ https://www.quantamagazine.org/how-close-are-computers-to-au... which was odd

99 points by theafh 5y ago

rbanffy 5y ago

What does "understanding" mean?

AndrewOMartin 5y ago
Very droll, but notice that I can write the phrase "Don't make snarky comments" without us having previously agreed on a formal definition of "snark".
- selestify 5y ago
  Right, so you’ve demonstrated that the phenomenon exists. Now, what defines that phenomenon, exactly?
  - gooseus 5y ago
    I'd say "understanding" is less one thing and more of a collection of capabilities which work together to allow understanding to "emerge". One of those capabilities would be the ability to contextual an object/statement within multiple frames of reference while also being able to compare and contrast the different instances of those contextualized objects/statements. This is what allows a child to identify a bird as any number of physical specimens of different species (chicken, goose, eagle, sparrow), while also identifying cartoon depictions that talk and simple drawings (Twitter icon) as birds as well... while also "understanding" that while the Twitter icon can be called a bird, it is not actually a real bird ("Ceci n'est pas une pipe") and it would not be expected to sing or fly like a backyard sparrow (unless it was animated, which would make still make sense to a child). I think this also what rise to our ability to "understand" jokes, puns, and other turns of phrase - "I just flew in from Boston, and boy are my arms tired!" - this dumb joke requires a number of concepts that need to be contextualized before you can "get" the absurdity of stating that a human might tire out their arms while flying... like you might think a bird would.
- TheOtherHobbes 5y ago
  That's because snark is a pre-installed module in the Western Human Online Communication Library Codebase. The code for "understanding" is part of Generic Human Firmware codebase and is tightly integrated with the OS. Unfortunately it hasn't been open sourced.
  - PaulDavisThe1st 5y ago
    It's totally open source, just written in a language we don't (yet) understand.
    - teddyh 5y ago
      I.e. a binary blob.
robbedpeter 5y ago
What it means for humans to understand has been posited by Jeff Hawkins as a combination of synaptic links within and between cortical columns, resulting in a physical neural construct that activates when stimulated past a sufficient threshold of inbound signals. Constructs can suppress nearby clusters of neurons, or contribute an increase in readiness to signal, or self modulate the activation threshold of constructs within the cortical column. The findings are consistent with current understanding of neuroscience, and align with discoveries such as grid cells. They also provide a basis for explaining what's actually happening in the brain with phenomena like memory palaces, rapid and efficient physical control of the body, combining sense modalities such as sight and touch when catching a ball, and so on. Understanding is what happens when your brain has developed a neural structure such that it's able to predict events successfully based on the thing that was understood.
- jjbinx007 5y ago
  Children learn thousands of words growing up not by having the definition read to them from a dictionary but by inferring their meaning based on context. Likewise we learn general concepts and they can be applied to a wide range of scenarios. I can't see any other way AI could learn unless it mimicks our own.
motohagiography 5y ago
This is the important question. After reading Ted Chiang's story "Understand" about this question, a simple working definition could be that to understand something you apprehend the domain of the subject from its substrate. Hence Feynman's "what I cannot create I do not understand," as well. In this sense, an AI could be said to Understand language if it used it as one of a selection of tools to operate on itself, a peer or other being, or its environment.
therobot24 5y ago
Situational Awareness is largely separated into 3 levels -- first level (perception), 2nd level (comprehension),and 3rd level (prediction) Understanding while not completely described by situational awareness definitely has some relationship to it and you could probably use similar constructs for defining it

hpoe 5y ago

When it gets to questions like these I feel that we transcend discussions of technology and end up on questions of philosiphy which aren't going to be going anywhere anytime quick. I also feel that AI should be used to augment not replace human decision making, it seems that where AI shines is problems that are well defined with well defined solutions, and because the AI doesn't get tired, hungry or distracted it can do that really well, but it fails in novel situations[0]. As such it seems to me our best bet is to have the AI provide suggestions rather than have complete control. 0. What is meant by that is a read an article, can't find it now, about using AI to diagnose breast cancer, what they found is that about 90% of the time the AI could accurately check for breast cancer, but the other 10% of the time was an unusual mammogram or something relatively rare, and in those situations the AI would often misdiagnoise.

deleted 5y ago
[deleted]
chermi 5y ago
AI had meaning that originated from CS theorists before being usurped and losing all meaning. I think tech and associated marketing (which are great, don't get me wrong, I love money!) are the ones at fault here. It should be more of a "philosophical" question, though I'd prefer instead perhaps it be academic. I'm not trying to be rude, but your example of what AI should be is narrow and not very grandiose compared the original meaning. I understand you were talking pretty loosely, so I feel like I'm singling you out but this happened to be where I started typing, sorry! It just reminded of how essentially all conversations about "AI" go. They seem to end up being quite specific, narrow pattern recognition problems at the end of the day. Maybe there's some decision theory on top of it. Maybe if there's enough money /people involved, there's more components, so it's a complicated enough supervised learning problem that it mimics people to a sufficient extent that it looks intelligent enough to make a headline. But it's a copycat, not intelligent. Hey, full circle, Melanie Mitchell! - https://en.m.wikipedia.org/wiki/Copycat_(software) https://en.m.wikipedia.org/wiki/Copycat_(software)

jonplackett 5y ago

Highly recommend A a thousand Brains by Jeff Hawkins if you’re interested in this https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-intelligence/ https://numenta.com/blog/2019/01/16/the-thousand-brains-theo... He basically argues we all have 1000s of little, but interacting models of all sorts of things going on in our brain all at the same time. He calls them reference frames and it’s those that create intelligence. ‘Understanding’ would come naturally out of having those. Fascinating book which I’m probably explaining much less well that he does.

andyjohnson0 5y ago
I've not read the book, but from your description it maybe has some similarity to Marvin Minsky’s "Society of Mind". Mind and intelligence as the emergent behaviour of many cooperating/competing systems.
- jonplackett 5y ago
  yes i think he talks about that. The difference is he's also looking for (and finding) the 'how' in physical brain structure terms, rather than just theory alone.

jonplackett 5y ago

Anyone who has kids and teaches them things will know AI learns and ‘understands’ very differently. I can say something like ‘a tiger is just a lion with stripes’ to a 3 year old and they now ‘understand’ what a tiger is almost as well as if they saw a picture of one. They could definitely identify one from a picture now. This kind of understanding won’t work with an AI because we don’t understand what characteristics it has latched into when identifying a lion. For all we know it’s that the background of each lion image its been trained on has a blue sky. Or the tigers are all looking at the camera. I think the ability to pick apart what you know and learn new things by reasoning about that knowledge is the key to if understanding is taking place.

ShamelessC 5y ago
I've done a fair bit of work with multimodal deep learning and I am fairly confident that a DALL-E, CLIP, or NUWA architecture would output/classify those phrases accurately without being trained explicitly on images of Tigers. I see your point however.
- jonplackett 5y ago
  I’d be interested in where to look for more info on that. Do you think it could work on anything more complex? As I said in comment below I reckon I could make a much more elaborate explanation and still have the kid get it.
  - ShamelessC 5y ago
    https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/ shows a decent ability to generalize to previously unseen concepts. You're correct to consider the complexity of the phrase and just how good humans are at this sort of thing without needing much "training". For now, concepts that aren't explicitly in the training set are effectively composed from those which are. This can lead to some bizarre and outright incorrect results, particularly when it comes to counting objects in a scene or with relative positioning between objects (e.g. a blue box on top of a red rectangle to the left of a green triangle) but it's early days and there's lots of progress happening all the time.

mwattsun 5y ago

I've been using Visual Studio 2022 recently which has AI driven code prediction built into it. Sometimes it predicts what the next thing I will type will be and I merely have to hit tab once or twice to accept it. At no point am I tempted to think Visual Studio understands my code, because it's just code itself. The first time I played a chess game was back in the early 1980's. While it beat me I felt an eerie "presence" in the machine that was sentient. I didn't know then about chess code so it was easier for me to anthropomorphize the machine (but it was the main reason I became interested in computers.) A computer "understanding" the difference between "how do I melt a block of ice" and "how do I melt my lover's icy heart" would be looking at the context and the relation of the words to each other. The computer might also predict I was sad if I asked the latter question. If I were a non-technical user I might think the computer felt empathy and be amazed by it. If I came upon a computer that "understands" I would want to determine if it understands like Richard Feynman or if it understands like my dog. My dog operates on a limited set of patterns, so that seems doable, but on the other hand, I've seen videos and heard stories of dogs exhibiting inventive and creative behavior that is unexpected. One such case is dogs that get lost and manage to find their way home thousands of miles away. tldr; I'm jaded. I know it's buggy code all the way down with computers.

tested23 5y ago

What does it mean for humans to understand? There are many times in the past where i thought i understood something and then i grow older and i see the holes.

Verdex 5y ago
Yeah, I feel this question is important to understand before we worry about what it means for the AI to understand. My thought is that we've got three types of "understanding": 1) social understanding 2) intuitive understanding 3) structural understanding Social understanding is something the society we live in knows, but the individual only knows in so far as the individual is doing something to fit in or via peer pressure. So for example, some high latitude countries eat fish for breakfast. Supposedly the statistics show that this helps them be more healthy than countries at similar latitudes which do not eat fish for breakfast ... probably because of problems due to lack of vitamin D due to lack of sunlight for certain parts of the year (the fish oil helps with this). However nobody actually "knows" this. They just eat fish because everyone else eats fish. Intuitive understanding is anything where we start to use flowery language like "experience" or "gut". You're really good at it, but just giving someone a flow chart isn't good enough. They have to have gone through the experience themselves. Driving is a good example. We make people take a test, but if just giving them diagrams and rules was good enough, then we wouldn't need a test where you actually drive and requirements about a certain number of hours of supervised driving. Structural understanding is anything that can be put to rules. So there's a lot of mathematics and algorithm stuff here. A simple example might be playing tic tac toe. The game is simple enough that you can write down a few rules that allow you to never lose. EDIT: My categories don't really answer the question, but they do give profiles and categories to look out for. Social understanding is good because it statistically learns to avoid lethal pitfalls. Like, if there's a dangerous well in the forest that people fall down and die in. A society might start telling people to not go in the forest because other people go in there and die. However, the society doesn't know why this is good advice. Intuitive understanding is good because it allows you to quickly statistically learn how to deal with imperfect and chaotic systems while getting good results. Structural understanding is good because it allows you to break free of the statistics of the previous two understandings. You can get exact results. Also it lets you break free of issues that come from distantly causal action + consequence. A person's intuition might not tell them that dumping toxic waste into the water is a good idea because things don't go bad until a lot of waste has already been dumped. Similarly a society might make a similar judgement if the failure is far enough away from the actions that kick it off. However, if you understand the structural relationships between things then you'll have an idea that toxic waste should not be consumed.

6gvONxR4sf7o 5y ago

Good article. I think winograd/winogrande are super clever (and also kind of a “fun” idea). My personal take is that blanket understanding is too hard of a task to define, so we ought to cheat and talk about types of understanding. In my mind, understanding a thing means not only that you can answer, but also that you can justify your answer. So different kinds of understanding point to different kinds of justifications. In math classes, you’ll be asked not only to state whether a thing is true or false, but also to show that it’s true or false, giving an answer as well as a proof. In a literature class, you don’t just make a point, you also have to support it in natural language. Same with science classes, supporting things with data and logic. The closest we have right now in ML (widely) is statistical measurements on holdout data. It’s 99% accurate on other things, so it’s 99% likely to be right on this too, if this was sampled from something just like the holdout. We also (less widely) have a little cottage industry of post-hoc explanations that try to explain models predictions. I’d love to see models that can do better than post-hoc explanations. I want a model that “understands in terms of predicate logic” that can spit out an answer with a checkable proof. Or “understands in terms of a knowledge graph” or “understands in terms of a set of human anatomy facts” or “understands in terms of natural language arguments” or any number of things that can spit out an answer as well as a justification. Just asking for blanket understanding means we have to define blanket understanding, but there’s a lot of limited understanding that’s still better than what we have today.

harperlee 5y ago
Exactly, understanding something requires holding a model (or several) of the thing in the mind and be able to work with that model to some extent. That’s why we dont see statistical tricks as understanding - as soon as trivial errors show that operation in those models is not feasible, we reject it. And that’s why winograd works: it establishes a simmetry that requires a base semantic model outside of the provided text.
rdedev 5y ago
The other good thing with requiring proofs is that you can show specify exactly where the proof fails and the one giving the proof can use this feedback to correct their beliefs. I would actually be pretty okay with a model that can do that. Take feedback and correct themselves without resorting to retraining. It would still be pretty far away from a general AI but much much better than a black box.

stakkur 5y ago

For AI to be truly artificial intelligence, shouldn't it be able to define when it understands?

hans1729 5y ago

Mandatory reference to Robert Miles' content on AI safety problems, alignment etc.: https://www.youtube.com/c/RobertMilesAI/playlists https://www.youtube.com/c/RobertMilesAI/playlists

trabant00 5y ago

It's incredible how stupidly powerful the rational part of the human brain is. It has this unlimited capacity to get lost in details. "What does it mean for AI to Understand?" - we keep arguing over definitions and moving the goal posts to make it seem we are an step closer to reaching AI. When my first AI coworker will read the on-boarding docs and start solving Jira issues I will have no doubt we have done it. That simple. Does anybody believe that an entity that would actually develop AI would start selling it? They would keep it for themselves and literally take over the world! Complete domination of the digital realm is one of easiest things an AI could do - I believe a lot easier than driving a car. And that alone would make them God. When the first true AI will be born we will simply live the experience. Imagine being there when we learned to control fire. Would you argue over the definition of it? The size, flame color, temperature and so on? Something that great can not be denied by such small details. LE: what happened to the Touring test? We forgot about it or does ordering things from Amazon when we command smartphone assistants to turn down the lights actually fools us?

PeterisP 5y ago
Regarding Turing test, it turned out that fooling people is easier than it might seem, so it's not considered anymore a reasonable qualification for general artificial intelligence as everybody now presumes that it's possible for a system to exist that can beat the Turing test but doesn't even attempt to be intelligent in any reasonable/general way.
- bryan0 5y ago
  I don't know why this is a common viewpoint. A proper Turing test with trained judges and human subjects who are actually trying to convince the judge they are human still seems like the best test of intelligence IMO.
- deleted 5y ago
  [deleted]
SxC97 5y ago
When my first AI coworker will read the on-boarding docs and start solving Jira issues I will have no doubt we have done it. That simple. It’s not exactly what you’re after, but we’re already pretty close!: https://mobile.twitter.com/gabro27/status/1173547934132178944 https://mobile.twitter.com/gabro27/status/117354793413217894...

throwawayai12 5y ago

I always get a little confused when people quote Oren. He hasn't been involved in meaningful work in AI in decades despite leading a large, well-funded group. But boy howdy can be give talks that sound good to lay people.

cgearhart 5y ago

Ignoring the associate debacle, the characterization of large language models as “stochastic parrots”[1] is the most accurate description I think I’ve ever heard for the capabilities of AI language models. These models don’t understand that a mistake on a Winograd question is not the same as a mistake on a medical diagnosis (as a contrived example). [1] https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922

visarga 5y ago
I don't think it's a good name. GPT-3 is more like a dreaming hallucination machine. Humans do nonsensical things in their dreams, same kind of non sequitur as the language models. What GPT-3 lacks is being able to wake up from its disembodied hallucinations, in other words a body and something to motivate its actions.

temptemptemp111 5y ago

It doesn't mean anything; people just don't understand what 'concepts' are anymore because they're so delusional. For AI to understand something means its human inventor / implementer understood it, potentially. The human understood a concept - not the actual thing - and that concept is what you call immaterial. You can point to code or output, but that is related to the concept. "Understanding" is when you stand under a concept - though you could always step out from under it in the case that you lose your understanding or it does not apply - etc. This inability for people to think is becoming hilarious. The metaverse is going to prevail for these damaged people but means nothing to those living in the real world.

nqzero 5y ago

What does it mean for a human to understand ?

prometheus76 5y ago
And to extend your question: write something sarcastic on the internet and you'll quickly find out there are multiple layers of "understanding" something.

jll29 5y ago

As an AI professor, I've always held that machines are NOT intelligent (I am prepared to change my position on the day my computer asks me anything surprising that I didn't program it to). But this does not mean we cannot produce operational models of understanding, for example we have models of propositional/logical semantics and discourse such as Lambda Discourse Representation Theory and others, which can compute a formal representation of the meaning structures for a piece of text. These have been used e.g. for answering question, and working in this space has been a lot of fun, and continues to do so. At the moment people talk a lot about "deep" learning (neural networks with more than one hidden layer), but for such models we need to do a lot more work into explainability, because it is too dangerous to use black boxes in real life. We still do not understand the human brain function in any substantial way, and it is perhaps a greater mystery of nature than even cosmology, where at least several competing theories have been posed that can explain parts of the evidence. How are thoughts represented (if that is answerable, it turns out 'Where are thoughts represented?' has proven to be a meaningless question due to the distributed nature of human memory)? What is consciousness? What is a conscience? How do consciousness and intention emerge from materials that are not alive and that have neither consciousness nor intention? How to implement approximate models? (A lot of work to do!)

kkoncevicius 5y ago
> How do consciousness and intention emerge from materials that are not alive and that have neither consciousness nor intention? I think the question shoud first be not about "how", but rather - "does consciousness emerge from the materials". Most cultures in the past have maintained ideas about the duality between spirit and matter. Nowadays we are so adept at manipulating matter we always explicitly assume that everything must be material in one way or another. But here is this consciousness thing that does not give in. And there is no angle to it where it can ever give in. If somebody creates a machine and claims it to be conscious nobody can test if that is so. And further - nobody can even test wheather the inventor is conscious himself.
- marginalia_nu 5y ago
  It seems to me that consciousness emerges when things arranged in some particular way are acted upon some principle, which is true even for simple behavior, like a rock falling to the ground. If you put a sail on a boat, it will catch the wind and move across the water. It's not the sail alone that moves the boat, nor the wind, but their interplay. A sail without wind doesn't get moved, and a wind without anything to catch it doesn't move anything. May be a bit of an unintuitive conclusion, but I think may be that we are catching the consciousness like a sail catches the wind, and this is the same "wind" that we call the forces of nature, that we only can perceive as patterns in how things around us appear to change.

continuational 5y ago

This is a great quote: > When AI can’t determine what ‘it’ refers to in a sentence, it’s hard to believe that it will take over the world.

typon 5y ago

I've come to believe that the answer will be provided by the "free market". When AIs start to replace humans for tasks that require "intelligence" (how we currently define it for humans), then AI will have achieved "understanding". Sure there will be hype driven companies that replace humans with AI for PR purposes, but eventually those will flatten out. Once I start receiving phone calls from AI telemarketers, have a decent conversation with them and can't tell that I just talked to an AI, then AI will have achieved understanding. And so on for other domains in every day life.

chrischapman 5y ago

Machines don't learn. Living things learn. Machines don't understand. Living things understand. Machines 'do' algorithms. Living things 'do' use-cases. Algorithm Definition: A sequence of actions that yields a result. Use-case Definition: A sequence of actions that yields a result of value to a user. The difference between the two is 'of value to a user'. To me, the line between algorithm and use-case is the line between unconsciousness and consciousness. That line pivots around the ability to 'value' something. I doubt we will ever see the I in AI until we build something that can value a result in the same way that you and I do. We need new words to describe what machines do. Using 'learn' or 'understand' seems like anthropomorphism. It's weird that we glibly anthropomorphise when talking about machines but prohibit it when talking about living things. Almost all of our qualities have been inherited from other living things. It seems to me that we should always anthropomorphise when talking about living things and never anthropomorphise when talking about machines. And yet, we always seem to do the opposite. Until we can explain how a machine can 'value' something in the same sense that humans do (or chimps or ducks or caterpillars do), we should avoid anthropomorphic words like 'learn' and 'understand' as it misdirects our efforts. However, I have no idea what else to suggest other than to try to explain how a machine can 'value' something. > IBM’s Watson was found to propose “multiple examples of unsafe and incorrect treatment recommendations.” That won't stop until Watson has the ability to 'value' the result of what it is doing. Watson 'does' algorithms. Watson needs to 'do' use-cases. Once it can 'value' a result in the same way we do, it will correct its mistakes.

JoeAltmaier 5y ago
Sounds like a bunch of unsubstantiated claims? So if something does learn, then its a living thing? My AI can learn (see a pattern and repeat it); now its a living thing? Not sure what to do with that.
- chrischapman 5y ago
  Yep. Totally unsubstantiated. Back of the envelope theory at best. > learn (see a pattern and repeat it) How did your AI 'see' and is 'see a pattern and repeat it' a good enough definition of 'learn'? Surely to 'learn' also means to 'understand'. What did your AI 'understand'? I doubt your AI actually learned anything. It attained zero knowledge in the way a living thing obtains knowledge. It may have stored a result in a database but did it actually understand anything? These are genuine questions as I have no professional knowledge of AI.

swframe2 5y ago

A software and hardware tool that can design and build another software and/or hardware tool to determine the cause and effect rules of any real world activity.

tomthe 5y ago

My understanding of "Understanding": Imagine a photo of a written poem: An image-processing program can "understand" the digital image: It can read the jpg, change the picture completely (e.g.: change the colors slightly), without changing the meaning one level up. But it doesn't understand the characters or words. An OCR program can read the image and "understand" the characters (or the textual representation. It can change the representation completely (save it as UTF-8 or whatever) without changing the meaning one level up. But it doesn't understand the language. GPT-3.... well, let't go directly to humans A human can read the text and understand the words and understand their meanings and what the sentences say. Another human really understands the poem and the subtext another level up. I think understanding always works on different levels and is a part of communication.

pinouchon 5y ago
Interestingly, the poet with the highest level understanding can be clueless about UTF-8 or what a pixel is
criddell 5y ago
Sounds like you are describing the treachery of images. Ceci n'est pas un poème.
deleted 5y ago
[deleted]

vbphprubyjsgo 5y ago

Nothing. Once it has enough neurons it's no different than a human brain in terms of what it can understand. Once it has more it can understand more.

bryan0 5y ago

> Unfortunately, Turing underestimated the propensity of humans to be fooled by machines. Even simple chatbots, such as Joseph Weizenbaum’s 1960s ersatz psychotherapist Eliza, have fooled people into believing they were conversing with an understanding being, even when they knew that their conversation partner was a machine. This is not a Turing test. Or at least not a reasonable one. A reasonable Turing consists of a judge and 2 interfaces, one which chats with a computer and the other which chats with a human. Both the human and computer are trying to convince the judge that they are the human. If the judge cannot determine which is which after an open-ended conversation then the computer passes. There is no reasonable judge which after chatting with a human (who is actually trying to convince the judge they are human) would be unable to differentiate between a human and Eliza or any other chatbot out there.

carapace 5y ago

Prediction. Setting aside the metaphysical questions of subjective "meaning" and "understanding" what else is there? What a system can predict is the measure of its "understanding", surely?

idiotsecant 5y ago
Does a mechanical bomb sight 'understand' ballistics and aerodynamics?
- carapace 5y ago
  By this definition, yes, of course. That's a great example.

sabhiram 5y ago

What does it mean for a child to understand? For a baby? A dog? A command line application? Understanding implies comprehension of some input, to influence a future state. Surely my stateful database understands requests that come to it. It, however will never surprise me with behavior that I would (should) not expect. I suppose if you "understood" the human machine and mind well enough, it would be possible to predict the actions it will carry out.

KhoomeiK 5y ago

People outside the NLP research community need to understand that a language model does nothing more than calculate probabilties for the next word given some context. The way it learns that probabilistic function can be quite complex, involving billions of parameters, but it's still fundamentally the same. Most of the increase in performance in recent years has come from the ability to train LMs that better generalize this probability function from huge text corpora--functions that better interleave between the datapoints it's trained on. Humans use language with purpose, to complete tasks and communicate with others, but GPT-3 has no more goals or desires than an n-gram model from the 90s. LMs are essentially a faculty for syntactically well-formed or intuitive/system 1 language generation, but they don't seem to be much more.

machiaweliczny 5y ago
What if to predict next word you need to compress/understand universe in the end? AFAIK we have no clue how exactly NNs pick up/structure information. For me backprop and learning is similar to evolution thus result might be simialar. My knowledge is very limited though but I am happy to read any proofs/insights.
- KhoomeiK 5y ago
  This is basically my area of research right now, compositional/symbolic representations in neural language models. I'll let you know the answer in a few months :)

wombatmobile 5y ago

> What Does It Mean for AI to Understand? What does it mean for a human to understand?

machiaweliczny 5y ago

BTW humans are generally stupid. We are only smart in collective with all trial/error saved in books, so don't underestimate computers. Also most of progress happens by outliers in humanity so it can be after AIs start to collaborate. Distributed MoE architectures are step in this direction. My prediction: codex and programming/operating APIs will give NNs ultra boost, then programming/math gets automated (hard to master, easy to verify) and rest/us will be history. John Conor is already alive. Mark my words.

mcguire 5y ago

"A Winograd schema, named for the language researcher Terry Winograd, consists of a pair of sentences, differing by exactly one word, each followed by a question. ... In each sentence pair, the one-word difference can change which thing or person a pronoun refers to. Answering these questions correctly seems to require commonsense understanding." I don't buy it. These Winograd schemas are significantly limited and are exactly the sort of thing that "story understanding" systems from the '70s and '80s were designed for.

toisanji 5y ago

a simulation engine in the human mind seems required for understanding: https://blog.jtoy.net/on-the-wondrous-human-simulation-engine/ https://blog.jtoy.net/on-the-wondrous-human-simulation-engin...

nullhack 5y ago

https://archive.md/8rArs https://archive.md/8rArs The page INSISTED on re-directing me to https://www.quantamagazine.org/how-close-are-computers-to-automating-mathematical-reasoning-20200827/ https://www.quantamagazine.org/how-close-are-computers-to-au... which was odd