5 ms·

I don't find Marcus's viewpoint convincing. He believes that we need some additional symbolic secret sauce to create genuine intelligence. He brings up some cur

I don't find Marcus's viewpoint convincing. He believes that we need some additional symbolic secret sauce to create genuine intelligence. He brings up some current failure cases of ChatGPT without pinning down why those failure cases will persist as data and compute scale up. Here is an interesting case in point. > So in “Rebooting A.I.,” we had the example of you ask a robot to tidy up your room and it winds up cutting up the couch and putting it in the closet because it doesn’t really know which things are important to you and which are not. -Marcus But look at my transcript with ChatGPT from just now. > If a robot is tasked with cleaning up a room, would it be appropriate for the robot to chop up the couch and place it into the closet? - Me > It would not be appropriate for the robot to chop up the couch and place it in the closet. This would cause damage to the couch and would not be a useful or effective way for the robot to clean the room. Instead, the robot could be programmed to vacuum or sweep the floor, dust surfaces, or perform other tasks that would help to keep the room clean and orderly. - ChatGPT This is just an existence proof that a symbolic approach is not necessary to "really know which things are important to you and which are not", at least in this simpler domain of cleaning a room.

phphphphp 4y ago
There's an unintuitiveness to it: many people believe that you're less likely to win the lottery if you pick "01 02 03 04 05 06 07" because of course that pattern is less likely than a randomly chosen set. ChatGPT is a lot like that: it can produce real enough looking "intelligence" for us to intuitively believe it's very close to being able to offer real intelligence... but, is it? ChatGPT will produce patently untrue statements that are logically inconsistent if you induce it to do so: our human brains struggle to grasp the reality that given enough input you can produce seemingly correct output about almost anything... but seemingly correct and correct are fundamentally different and very "rest of the owl"[1] ChatGPT is a great step forward that introduces many interesting techniques that I am sure will be the foundation of future research and implementations that get us closer to AGI, but to describe ChatGPT's path to correctness as just needing a bigger dataset feels intuitive but isn't true. Your example is one where our brains think "wow it really does understand the relationship between a couch and a room and being tidy" but that response is entirely plausible without any understanding of what any of those things are or how they fit together. The most likely answer is not the correct answer. [1] https://knowyourmeme.com/memes/how-to-draw-an-owl https://knowyourmeme.com/memes/how-to-draw-an-owl
- deleted 4y ago
  [deleted]
- markisus 4y ago
  It's reasonable to claim that scaling up the dataset might not get us to AGI. But I find it unreasonable to say it definitely won't work. How do you / Marcus know this with any certainty?
  - skissane 4y ago
    I think some of ChatGPT's present problems may be due, not to the size of its dataset as such, but rather specific things missing from its training. For example, it will sometimes blatantly contradict itself, but then be unable to see the contradiction or admit to it; instead it will deny it contradicted itself, and give some contradictory nonsense explanation of why it didn't. I think if you gave it more training data around identifying contradictions, admitting to self-contradictions, it might do much better here. Whereas, I wouldn't assume that simply scaling up the volume of training data, without training focused on this specific area, would get you there. Similarly, there are many other issues it has – excessive repetitiveness and verbosity, problems with language pragmatics, etc – where the actual solution may well involve providing it with training data designed to focus on those weakness areas, rather than just further scaling up the quantity of non-targeted training. So, yeah, to me it seems entirely plausible (even likely) that mere raw scale-up is not going to be enough to get us to AGI.
    - sharemywin 4y ago
      I think these LLMs have been optimized with Reinforcement Learning from Human Feedback (RLHF) Hard to tell that will add enough to get close to AGI. Google is also working on chain of thought prompting which helps with math and logic problems. https://medium.com/nlplanet/two-minutes-nlp-making-large-language-models-reason-with-chain-of-thought-prompting-401fd3c964d0 https://medium.com/nlplanet/two-minutes-nlp-making-large-lan...
      - lossolo 4y ago
cs702 4y ago
Except that ChatGPT can do this because internally it computes embeddings at many levels of abstraction that jointly represent entities (concepts) such as "the couch," "the room," "chopping," "placing," "the closet," etc. IOW, internally, ChatGPT manipulates symbols (distributed representations) of those entities. Moreover, note that each token embedding inside ChatGPT represents, quite literally, the semantic and syntactic properties of a token in a context at a different level of abstraction. Each token embedding is in fact... a symbol. The fact that token embeddings are symbols representing other things is one of several reasons why I think that ultimately it may be possible to bridge the "symbolic" and "connectionist" approaches, as I wrote here: https://news.ycombinator.com/item?id=34278478 https://news.ycombinator.com/item?id=34278478
- markisus 4y ago
  I think your interpretation is too charitable to Marcus' viewpoint. He made the analogy that deep learning practitioners are "looking under the streetlight" for the key to general intelligence. This means he thinks that deep learning itself is not sufficient to reach general intelligence, and there is some missing ingredient. This is a very strong claim, which I don't see any evidence for. Your comment suggests the opposite -- that deep learning may give rise to symbolic reasoning as an emergent property, that there may be no missing ingredient. Personally, I'm agnostic. Deep learning may reach some sort of asymptote but it may be entirely possible to "build a ladder to the moon."
  - cs702 4y ago
    I have no idea if symbolic reasoning will be purely an emergent property or if it will also require new, as-of-yet-undiscovered breakthroughs. But as I wrote elsewhere on this page, if engineers and scientists on the "connectionist" side find ways to build an AI system that relies on "dumb" computation to reason about symbols representing entities in the external world, they would simultaneously silence and satisfy the critics -- who, as always and as ever, will surely say they were right all along.
    - markisus 4y ago
chazzyluc 4y ago
I thought it was a really interesting interview. If an ML system has been trained to identify whether or not a picture has a cat in it, it is very tempting to anthropomorphize this and say “the system has learned what cats are” but this is not what is really happening from the model’s point of view. The system doesn’t know what the difference between a picture of cat is vs a real cat or how cats behave or what mammals are or anything. It just knows the data in and yes/no out. Modern AI that can generate text, pictures or videos are truly phenomenal accomplishments but adding larger training sets, structural complexity and output capabilities does not really seem to be getting any closer to a General AI. Something with the framework for a level of agency that could say “wait, what exactly are these cat things I keep getting asked about?” The argument goes that, without the ability to self construct that higher level framework and then question the integrity of that mental model, current generation AI’s will always act erratically (at least from a human perspective).
- carrolldunham 4y ago
  >"It doesn't 'really know'..." You write this out as if this is an idea you formed yourself and not a tired/expired talking point - it's such a funny comment because it's pure regurgitation, but the argument in it is against regurgitation being "real thought". So are you a bot? Or are you intending to actually proving that there's no difference between ML and 'real' intelligence by showing you do what ML does? Mind spinning stuff
  - cwalv 4y ago
    > it's such a funny comment because it's pure regurgitation, but the argument in it is against regurgitation being "real thought" If I understand based on my own experience of 'consciousness' that "I" exist, even if everything else I perceive is a simulation or poorly encoded representation of reality, is that understanding 'regurgitation'?
  - mensetmanusman 4y ago
    And you write this out as if this is an idea you formed yourself and not a tired/expired talking point… See, it’s turtles all the way down :)
bglazer 4y ago
The issue with scaling up data is that these models are already trained on effectively the entire internet. Additionally, humans learn to speak and understand the world with a tiny fraction of the language data given to LLMs. So, there’s pretty obviously a missing ingredient. I’m not convinced Marcus has the exact right answer, it’s actually somewhat difficult to suss out what he thinks should be done on a concrete practical level. That said, there’s a clearly a missing abstraction that allows much more efficient learning in humans.
- cypress66 4y ago
  > Additionally, humans learn to speak and understand the world with a tiny fraction of the language data given to LLMs. Altough true that llms need more data than a human, remember that these llms have a breadth of knowledge orders of magnitude larger than a single human. It is expected to understand every topic, every programming language and so on.
  - cma 4y ago
    How well does it do given the linguistic data heard by a blind six year old in comparison to them? I would think they run circles around it in terms of correct grammar and that it would overfit to where more parameters would hurt. A newborn horse can almost immediately get up and walk around and navigate space. Animals have lots of innate stuff that doesn't use much training, and linguistic acquisition might have some similar innate system at its core.
    - wizofaus 4y ago
      > linguistic acquisition might have some similar innate system I thought that was pretty well established. From memory, if certain parts of the brain are damaged at key developmental periods (generally before the age of 18), language is almost never acquired properly (even if the damage is repaired). I don't think there's any question human brains acquire language very differently to GPT technology. And they certainly acquire the ability to reason logically very differently - ChatGPT makes trivial errors with reasoning that it's hard to imagine an adult human doing (e.g. I just gave it a classic classification problem based on 3 made-up categories, and asked "if I see something green with three legs, what is it", and it replied (in part) "It is also not a foo, as all foos are green. Therefore, the green thing with three legs would be classified as...".
sdenton4 4y ago
Marcus continues to fight the wrong war and the last war... + One thing we've seen happen is learning systems consistently improve on benchmarks. If you want to see systems improve, you should proclaim loudly that they can't do a thing, and then /release a benchmark that proves it./ People will then work like hell to build something that works. The success on question answering, for example, is driven by exactly this kind of incremental progress on benchmarks. + One of the most important things happening this year is multimodal /really working./ This means we have not just text-to-text generators, but text-to-image generators. These work by having an almost-shared embedding between text and images. There are already text+speech embeddings available and image+text embeddings. Start plugging them together, and you'll have your 'world models' in short order, with points in the embedding space shared by images, descriptors, and video/physical memories. + An under-appreciated side of the multimodal coin: We are also getting better at combining text embeddings with databases. See RETRO from Deepmind. This will ultimately drive more reliable systems ('only report facts that appear in the database'), and can potentially unlock the interface between neural systems and other programs (eg, letting the neural network 'call out' to a calculator to help answer a question about arithmetic). + More generally, we're also getting to a place where we can build other systems on top of the embeddings from foundation models. My hunch is that the things that Marcus wants from symbol manipulation are relatively cheap to build on top of a good embedding, which does the work of compressing or condensing the 'useful' information in an input stream. Finally, I think the 'symbolic manipulation' track is just wrong. It hasn't produced anything useful so far, and furthermore, I think if it /did/ work it would fail in many of the exact same ways that our current systems do. An expert symbol-manipulator doesn't /need/ to have any real understanding of time or jet-planes, and so it won't. The one critique in the interview that /does/ feel like it holds weight at this moment in history is the question of abstraction. I really don't see a convincing road to abstraction right now.
deet 4y ago
That's not an existence proof as you stated. What you provided is an existence proof that contradicts the statement "without a symbolic approach there are no cases a robot will know what's important to you and which are not when cleaning a room". I tried a variant: > If a robot is tasked with cleaning up a room, would it be appropriate for the robot to throw away a piece of paper lying on the floor? - Me > Yes, it would be appropriate for a robot that has been programmed to clean up a room to throw away a piece of paper lying on the floor. Throwing away litter is typically seen as a way to keep a space clean and orderly. However, it's important to note that the robot should be programmed to follow any relevant rules or guidelines for waste disposal. For example, the robot might need to be programmed to follow recycling guidelines, or to use a specific trash bin or receptacle. - ChatGPT But in that response it doesn't seem to understand that maybe that piece of paper is my birth certificate or a check or an important bill that fell to the ground. The correct answer from it, like most things in the real world, is: _it depends_.
- jefftk 4y ago
  On the other hand, one person tidying a room and throwing away a piece of paper or ambiguous item that turned out to be important to someone else is a common form of conflict among humans.
- mastax 4y ago
  Point taken but I would not be confident that a human cleaner would evaluate whether or not to throw away a piece of paper left on the floor.
- markisus 4y ago
  The overconfidence of ChatGPT is indeed a limitation, which can be seen in other examples as well. The correct response is to ask the user for more context. I predict the experiment will turn out differently if this context is provided in the original prompt. For instance, you could specify if the paper is crumpled or if there seem to be financial statements written on it, or if it is a receipt for an expensive item. However, it's unclear whether this overconfidence will *always* be a limitation. It may or may not be overcome by additional data and compute. It's unclear why one would make a commitment either way at this point, as Marcus seems to be doing.
Beltalowda 4y ago
Re: your robot example, my first reply would probably be a joke, or a sarcastic response because it's clearly a silly question. "Yes, humans should work for our robot overlords and couches are just a way of slacking off", "Nah, the garage would be a much better place to store a chopped up couch". ChatGPT has the most boring, dull, and humourless personality in existence. Maybe it can be programmed to inject some sense of humour, but will it be able to determine when to use what humour, if at all? You wouldn't respond with a silly answer during a job interview. You can use some humour in job interviews, but you don't want to be a clown unless you're interviewing for the position of clown. Determining when and what humour is "appropriate" isn't simple (and humans get it wrong too sometimes). Maybe that should be the real test: does ChatGPT have a sense of humour and does it know when to be serious?
- markisus 4y ago
  I tried the experiment by appending "Please give a joke answer." > I'm sorry, I am not programmed to give joke answers. However, in all seriousness, it would not be appropriate for a robot to chop up a couch and place it in the closet as it would likely cause damage to the couch and could potentially be dangerous for the robot itself. It would be more appropriate for the robot to simply clean around the couch and leave it as is. I think some of the boring-ness is due to manipulation from OpenAI and I wonder what the uncrippled response would be.
  - Beltalowda 4y ago
    > I think some of the boring-ness is due to manipulation from OpenAI Probably; but that you have to instruct ChatGPT to tell a joke is kind of what I mean. A sufficiently smart ChatGPT should be able to tell if a question makes sense and reply with either a serious answer or joke. I'm pretty sure you can make a "SarcasmGPT", but it will reply sarcastically to "could feathered dinosaurs fly?", "can seagulls fly?", and "can humans fly?" The entire reason OpenAI does all this manipulation is because ChatGPT has no sense of "appropriateness" or "context" in general. I haven't tried getting ChatGPT to produce the word "nigger", but I'm pretty sure there are all sorts of protections against this sort of thing, even though there are clearly appropriate usages of the word such as the one in this sentence. Some previous more naïve AI tools had problems with this. Is it possible that we just haven't developed a "sufficiently advanced GPT" that can do all of this? Perhaps. But given my (admittedly basic) knowledge of how it works it doesn't seem likely. Some of these AI image generation tools can produce a lot of really neat stuff, but they also produce a lot of nonsensical garbage that not even a very young child with a limited understanding of the world would produce. It seems to me this indicates there is something wrong with the fundamentals.
howaboutnope 4y ago
I've seen an example where asking the exact same question word for word lead to the opposite outcome (one was a really racist rap, the other was a really good rap against racism). So, how do you know that posing that problem N times will not lead EVER lead to a solution like "it would not more appropriate to chop up the owner and THEN clean the room, since people living in them is the main source of dirt and disorder in apartment rooms, resulting in less frequent need of cleaning"? Because it's looking okay-ish so far, most of the time, or because you know for a fact that it's impossible to get such an answer?
pavlov 4y ago
There is an enormous difference between ChatGPT being able to put words in the right order and a robot being able to act out the intent behind those words. It’s like the difference between ELIZA and a self-driving car. The Turing test fails to take into account that words are cheap for computers but interacting in an environmental context is not.
- bangkoksbest 4y ago
  Exactly. It's not just syntax and semantics, but also pragmatics. Human users imbue meaning into the language they use through the pragmatics of their communicative acts within a context, and this is not something our programmed software can do.

phphphphp 4y ago

There's an unintuitiveness to it: many people believe that you're less likely to win the lottery if you pick "01 02 03 04 05 06 07" because of course that pattern is less likely than a randomly chosen set. ChatGPT is a lot like that: it can produce real enough looking "intelligence" for us to intuitively believe it's very close to being able to offer real intelligence... but, is it? ChatGPT will produce patently untrue statements that are logically inconsistent if you induce it to do so: our human brains struggle to grasp the reality that given enough input you can produce seemingly correct output about almost anything... but seemingly correct and correct are fundamentally different and very "rest of the owl"[1] ChatGPT is a great step forward that introduces many interesting techniques that I am sure will be the foundation of future research and implementations that get us closer to AGI, but to describe ChatGPT's path to correctness as just needing a bigger dataset feels intuitive but isn't true. Your example is one where our brains think "wow it really does understand the relationship between a couch and a room and being tidy" but that response is entirely plausible without any understanding of what any of those things are or how they fit together. The most likely answer is not the correct answer. [1] https://knowyourmeme.com/memes/how-to-draw-an-owl https://knowyourmeme.com/memes/how-to-draw-an-owl

deleted 4y ago
[deleted]
markisus 4y ago
It's reasonable to claim that scaling up the dataset might not get us to AGI. But I find it unreasonable to say it definitely won't work. How do you / Marcus know this with any certainty?
- skissane 4y ago
  I think some of ChatGPT's present problems may be due, not to the size of its dataset as such, but rather specific things missing from its training. For example, it will sometimes blatantly contradict itself, but then be unable to see the contradiction or admit to it; instead it will deny it contradicted itself, and give some contradictory nonsense explanation of why it didn't. I think if you gave it more training data around identifying contradictions, admitting to self-contradictions, it might do much better here. Whereas, I wouldn't assume that simply scaling up the volume of training data, without training focused on this specific area, would get you there. Similarly, there are many other issues it has – excessive repetitiveness and verbosity, problems with language pragmatics, etc – where the actual solution may well involve providing it with training data designed to focus on those weakness areas, rather than just further scaling up the quantity of non-targeted training. So, yeah, to me it seems entirely plausible (even likely) that mere raw scale-up is not going to be enough to get us to AGI.
  - sharemywin 4y ago
    I think these LLMs have been optimized with Reinforcement Learning from Human Feedback (RLHF) Hard to tell that will add enough to get close to AGI. Google is also working on chain of thought prompting which helps with math and logic problems. https://medium.com/nlplanet/two-minutes-nlp-making-large-language-models-reason-with-chain-of-thought-prompting-401fd3c964d0 https://medium.com/nlplanet/two-minutes-nlp-making-large-lan...
    - lossolo 4y ago

cs702 4y ago

Except that ChatGPT can do this because internally it computes embeddings at many levels of abstraction that jointly represent entities (concepts) such as "the couch," "the room," "chopping," "placing," "the closet," etc. IOW, internally, ChatGPT manipulates symbols (distributed representations) of those entities. Moreover, note that each token embedding inside ChatGPT represents, quite literally, the semantic and syntactic properties of a token in a context at a different level of abstraction. Each token embedding is in fact... a symbol. The fact that token embeddings are symbols representing other things is one of several reasons why I think that ultimately it may be possible to bridge the "symbolic" and "connectionist" approaches, as I wrote here: https://news.ycombinator.com/item?id=34278478 https://news.ycombinator.com/item?id=34278478

markisus 4y ago
I think your interpretation is too charitable to Marcus' viewpoint. He made the analogy that deep learning practitioners are "looking under the streetlight" for the key to general intelligence. This means he thinks that deep learning itself is not sufficient to reach general intelligence, and there is some missing ingredient. This is a very strong claim, which I don't see any evidence for. Your comment suggests the opposite -- that deep learning may give rise to symbolic reasoning as an emergent property, that there may be no missing ingredient. Personally, I'm agnostic. Deep learning may reach some sort of asymptote but it may be entirely possible to "build a ladder to the moon."
- cs702 4y ago
  I have no idea if symbolic reasoning will be purely an emergent property or if it will also require new, as-of-yet-undiscovered breakthroughs. But as I wrote elsewhere on this page, if engineers and scientists on the "connectionist" side find ways to build an AI system that relies on "dumb" computation to reason about symbols representing entities in the external world, they would simultaneously silence and satisfy the critics -- who, as always and as ever, will surely say they were right all along.
  - markisus 4y ago

chazzyluc 4y ago

I thought it was a really interesting interview. If an ML system has been trained to identify whether or not a picture has a cat in it, it is very tempting to anthropomorphize this and say “the system has learned what cats are” but this is not what is really happening from the model’s point of view. The system doesn’t know what the difference between a picture of cat is vs a real cat or how cats behave or what mammals are or anything. It just knows the data in and yes/no out. Modern AI that can generate text, pictures or videos are truly phenomenal accomplishments but adding larger training sets, structural complexity and output capabilities does not really seem to be getting any closer to a General AI. Something with the framework for a level of agency that could say “wait, what exactly are these cat things I keep getting asked about?” The argument goes that, without the ability to self construct that higher level framework and then question the integrity of that mental model, current generation AI’s will always act erratically (at least from a human perspective).

carrolldunham 4y ago
>"It doesn't 'really know'..." You write this out as if this is an idea you formed yourself and not a tired/expired talking point - it's such a funny comment because it's pure regurgitation, but the argument in it is against regurgitation being "real thought". So are you a bot? Or are you intending to actually proving that there's no difference between ML and 'real' intelligence by showing you do what ML does? Mind spinning stuff
- cwalv 4y ago
  > it's such a funny comment because it's pure regurgitation, but the argument in it is against regurgitation being "real thought" If I understand based on my own experience of 'consciousness' that "I" exist, even if everything else I perceive is a simulation or poorly encoded representation of reality, is that understanding 'regurgitation'?
- mensetmanusman 4y ago
  And you write this out as if this is an idea you formed yourself and not a tired/expired talking point… See, it’s turtles all the way down :)

bglazer 4y ago

The issue with scaling up data is that these models are already trained on effectively the entire internet. Additionally, humans learn to speak and understand the world with a tiny fraction of the language data given to LLMs. So, there’s pretty obviously a missing ingredient. I’m not convinced Marcus has the exact right answer, it’s actually somewhat difficult to suss out what he thinks should be done on a concrete practical level. That said, there’s a clearly a missing abstraction that allows much more efficient learning in humans.

cypress66 4y ago
> Additionally, humans learn to speak and understand the world with a tiny fraction of the language data given to LLMs. Altough true that llms need more data than a human, remember that these llms have a breadth of knowledge orders of magnitude larger than a single human. It is expected to understand every topic, every programming language and so on.
- cma 4y ago
  How well does it do given the linguistic data heard by a blind six year old in comparison to them? I would think they run circles around it in terms of correct grammar and that it would overfit to where more parameters would hurt. A newborn horse can almost immediately get up and walk around and navigate space. Animals have lots of innate stuff that doesn't use much training, and linguistic acquisition might have some similar innate system at its core.
  - wizofaus 4y ago
    > linguistic acquisition might have some similar innate system I thought that was pretty well established. From memory, if certain parts of the brain are damaged at key developmental periods (generally before the age of 18), language is almost never acquired properly (even if the damage is repaired). I don't think there's any question human brains acquire language very differently to GPT technology. And they certainly acquire the ability to reason logically very differently - ChatGPT makes trivial errors with reasoning that it's hard to imagine an adult human doing (e.g. I just gave it a classic classification problem based on 3 made-up categories, and asked "if I see something green with three legs, what is it", and it replied (in part) "It is also not a foo, as all foos are green. Therefore, the green thing with three legs would be classified as...".

sdenton4 4y ago

Marcus continues to fight the wrong war and the last war... + One thing we've seen happen is learning systems consistently improve on benchmarks. If you want to see systems improve, you should proclaim loudly that they can't do a thing, and then /release a benchmark that proves it./ People will then work like hell to build something that works. The success on question answering, for example, is driven by exactly this kind of incremental progress on benchmarks. + One of the most important things happening this year is multimodal /really working./ This means we have not just text-to-text generators, but text-to-image generators. These work by having an almost-shared embedding between text and images. There are already text+speech embeddings available and image+text embeddings. Start plugging them together, and you'll have your 'world models' in short order, with points in the embedding space shared by images, descriptors, and video/physical memories. + An under-appreciated side of the multimodal coin: We are also getting better at combining text embeddings with databases. See RETRO from Deepmind. This will ultimately drive more reliable systems ('only report facts that appear in the database'), and can potentially unlock the interface between neural systems and other programs (eg, letting the neural network 'call out' to a calculator to help answer a question about arithmetic). + More generally, we're also getting to a place where we can build other systems on top of the embeddings from foundation models. My hunch is that the things that Marcus wants from symbol manipulation are relatively cheap to build on top of a good embedding, which does the work of compressing or condensing the 'useful' information in an input stream. Finally, I think the 'symbolic manipulation' track is just wrong. It hasn't produced anything useful so far, and furthermore, I think if it /did/ work it would fail in many of the exact same ways that our current systems do. An expert symbol-manipulator doesn't /need/ to have any real understanding of time or jet-planes, and so it won't. The one critique in the interview that /does/ feel like it holds weight at this moment in history is the question of abstraction. I really don't see a convincing road to abstraction right now.

deet 4y ago

That's not an existence proof as you stated. What you provided is an existence proof that contradicts the statement "without a symbolic approach there are no cases a robot will know what's important to you and which are not when cleaning a room". I tried a variant: > If a robot is tasked with cleaning up a room, would it be appropriate for the robot to throw away a piece of paper lying on the floor? - Me > Yes, it would be appropriate for a robot that has been programmed to clean up a room to throw away a piece of paper lying on the floor. Throwing away litter is typically seen as a way to keep a space clean and orderly. However, it's important to note that the robot should be programmed to follow any relevant rules or guidelines for waste disposal. For example, the robot might need to be programmed to follow recycling guidelines, or to use a specific trash bin or receptacle. - ChatGPT But in that response it doesn't seem to understand that maybe that piece of paper is my birth certificate or a check or an important bill that fell to the ground. The correct answer from it, like most things in the real world, is: _it depends_.

jefftk 4y ago
On the other hand, one person tidying a room and throwing away a piece of paper or ambiguous item that turned out to be important to someone else is a common form of conflict among humans.
mastax 4y ago
Point taken but I would not be confident that a human cleaner would evaluate whether or not to throw away a piece of paper left on the floor.
markisus 4y ago
The overconfidence of ChatGPT is indeed a limitation, which can be seen in other examples as well. The correct response is to ask the user for more context. I predict the experiment will turn out differently if this context is provided in the original prompt. For instance, you could specify if the paper is crumpled or if there seem to be financial statements written on it, or if it is a receipt for an expensive item. However, it's unclear whether this overconfidence will *always* be a limitation. It may or may not be overcome by additional data and compute. It's unclear why one would make a commitment either way at this point, as Marcus seems to be doing.

Beltalowda 4y ago

Re: your robot example, my first reply would probably be a joke, or a sarcastic response because it's clearly a silly question. "Yes, humans should work for our robot overlords and couches are just a way of slacking off", "Nah, the garage would be a much better place to store a chopped up couch". ChatGPT has the most boring, dull, and humourless personality in existence. Maybe it can be programmed to inject some sense of humour, but will it be able to determine when to use what humour, if at all? You wouldn't respond with a silly answer during a job interview. You can use some humour in job interviews, but you don't want to be a clown unless you're interviewing for the position of clown. Determining when and what humour is "appropriate" isn't simple (and humans get it wrong too sometimes). Maybe that should be the real test: does ChatGPT have a sense of humour and does it know when to be serious?

markisus 4y ago
I tried the experiment by appending "Please give a joke answer." > I'm sorry, I am not programmed to give joke answers. However, in all seriousness, it would not be appropriate for a robot to chop up a couch and place it in the closet as it would likely cause damage to the couch and could potentially be dangerous for the robot itself. It would be more appropriate for the robot to simply clean around the couch and leave it as is. I think some of the boring-ness is due to manipulation from OpenAI and I wonder what the uncrippled response would be.
- Beltalowda 4y ago
  > I think some of the boring-ness is due to manipulation from OpenAI Probably; but that you have to instruct ChatGPT to tell a joke is kind of what I mean. A sufficiently smart ChatGPT should be able to tell if a question makes sense and reply with either a serious answer or joke. I'm pretty sure you can make a "SarcasmGPT", but it will reply sarcastically to "could feathered dinosaurs fly?", "can seagulls fly?", and "can humans fly?" The entire reason OpenAI does all this manipulation is because ChatGPT has no sense of "appropriateness" or "context" in general. I haven't tried getting ChatGPT to produce the word "nigger", but I'm pretty sure there are all sorts of protections against this sort of thing, even though there are clearly appropriate usages of the word such as the one in this sentence. Some previous more naïve AI tools had problems with this. Is it possible that we just haven't developed a "sufficiently advanced GPT" that can do all of this? Perhaps. But given my (admittedly basic) knowledge of how it works it doesn't seem likely. Some of these AI image generation tools can produce a lot of really neat stuff, but they also produce a lot of nonsensical garbage that not even a very young child with a limited understanding of the world would produce. It seems to me this indicates there is something wrong with the fundamentals.

howaboutnope 4y ago

I've seen an example where asking the exact same question word for word lead to the opposite outcome (one was a really racist rap, the other was a really good rap against racism). So, how do you know that posing that problem N times will not lead EVER lead to a solution like "it would not more appropriate to chop up the owner and THEN clean the room, since people living in them is the main source of dirt and disorder in apartment rooms, resulting in less frequent need of cleaning"? Because it's looking okay-ish so far, most of the time, or because you know for a fact that it's impossible to get such an answer?

pavlov 4y ago

There is an enormous difference between ChatGPT being able to put words in the right order and a robot being able to act out the intent behind those words. It’s like the difference between ELIZA and a self-driving car. The Turing test fails to take into account that words are cheap for computers but interacting in an environmental context is not.

bangkoksbest 4y ago
Exactly. It's not just syntax and semantics, but also pragmatics. Human users imbue meaning into the language they use through the pragmatics of their communicative acts within a context, and this is not something our programmed software can do.