6 ms·

Sure it might produce convincing examples of human speech, but it fundamentally lacks an internal point of view that it can express, which places limits on how

Sure it might produce convincing examples of human speech, but it fundamentally lacks an internal point of view that it can express, which places limits on how well it can argue something. It is of course possible that it might (eventually) be convincing enough that no human can tell, which would be problematic because it would suggest human speech is indistinguishable from a knee jerk response that doesn't require that you communicate any useful information. Things would be quite different if an AI could interpret new information and form opinions, but even if GPT could be extended to do so, right now it doesn't seem to have the capability to form opinions or ingest new information (beyond a limited short term memory that it can use to have a coherent conversation).

jacquesm 4y ago
But the bar really isn't 'no human can tell' the bar is 'the bulk of the humans can't tell'. > Things would be quite different if an AI could interpret new information and form opinions, but even if GPT could be extended to do so, right now it doesn't seem to have the capability to form opinions or ingest new information (beyond a limited short term memory that it can use to have a coherent conversation). Forming opinions is just another mode of text transformations, ingesting new information is either a conscious decision to not let the genie out of the bottle just yet or a performance limitation, neither of those should be seen as cast in stone, the one is a matter of making the model incremental (which should already be possible), the other merely a matter of time.
- sheepscreek 4y ago
  A true AI will not have one opinion. It will realize there are many truths - one persons truth is really a perspective based on their inputs which are different than another's. Change the inputs and you’ll often get a different output. ChatGPT further proves this notion - you can ask it to prove/disprove the same point and it will do so quite convincingly both times.
  - jacquesm 4y ago
    > ChatGPT further proves this notion - you can ask it to prove/disprove the same point and it will do so quite convincingly both times. Just like any lawyer, then, depending on who foots the bill.
    - voakbasda 4y ago
      Right? If anything, this kind of mental flexibility is more human than not.
  - mdp2021 4y ago
    > you can ask it to prove/disprove the same point and it will do so quite convincingly both times Probably because "in the night of the reason everything is black"; probably because it is missing the very point, which is to get actual, real, argumented, solid insight on matters!!! You use Decision Support Systems to better understand a context, not to have a well dressed thought toss!
  - shanusmagnus 4y ago
    That's a great point that I haven't seen in the GPT-related conversations. People view the fact that it can argue convincingly for both A and ~A as a flaw in GPT and limitation of LLMs, rather than an insight about human reasoning and motivation. Maybe it's an illustration of a more general principle: when people butt up against limitations that make LLMs look silly, or inadequate, often their real objection is with some hard truths about reality itself.
  - jerf 4y ago
    Do not mistake ChatGPT for AI in general. ChatGPT, GPT, and transformers in general are not the end state of AI. They are one particular manifestation and projecting forward from them is drawing a complex hypershape through a single point (even worse than drawing a line through a single point). It is probably more humanly-accurate to say that ChatGPT has no opinions at all. It has no understanding of truth, it has no opinions, it has no preferences whatsoever. It is the ultimate yes-thing; whatever you say, it'll essentially echo and elaborate on it, without regard for what it is that you said. This obviously makes it unsuitable for many things. (This includes a number of things for which people are trying to use it.) This does not by any means prove that all possible useful AI architectures will also have no opinion, or that all architectures will be similarly noncommital. (If you find yourself thinking this is a "criticism" of GPT... you may be too emotionally involved. GPT is essentially like looking into a mirror, and the humans doing so are bringing more emotion to that than the AI is. That's not "bad" or something, that's just how it works. What I'm saying here isn't a criticism or a praise; it's really more a super-dumbed-down description of its architecture. It fundamentally lacks these things. You can search it up and down for "opinions" or "truth", and it just isn't there in that architecture, not even implied in the weights somewhere where we can't see it. It isn't a good thing or a bad thing, it just is a property of the design.)
- mhb 4y ago
  None of this matters. The reason comments are valuable is that they are a useful source of information. Part of the transaction cost of deciding whether a comment is useful is how much additional work is required to evaluate it. Comments are ascribed credibility based on the trust the reader has in the commenting entity, whether the comment is consistent with the reader's priors and researching citations made in the comment, either explicit or implicit. Since GPT can confidently produce comments which are wrong, there is no trust in it as a commenting entity. Consequently everything it produces needs to be further vetted. It's as if every comment was a bunch of links to relevant, but not necessarily correct sources. Maybe it produces some novelty which leads to something worthwhile, but the cost is high, until it can be trusted. Which is not now. If a trusted commenter submits a comment by GPT, then he is vouching for it and it is riding on his reputation. If it is wrong, his reputation suffers, and trust in that commenter drops just as it would regardless of the genesis of the comment.
swader999 4y ago
You are correct in stating that current chat bots, such as GPT, do not have the ability to form opinions or interpret new information beyond a limited short term memory. This is a limitation of current technology, and as a result, chat bots are limited in their ability to engage in complex arguments or discussions. However, it is important to note that the development of AI technology is ongoing, and it is possible that future advances will allow for the development of more sophisticated AI systems that are capable of forming opinions and interpreting new information. Until that time, chat bots will continue to be limited in their abilities.
- chirau 4y ago
  I am pretty sure this response was generated by a bot/GPT. As good as they are, you can tell what's GPT stuff and what isn't.
  - jacquesm 4y ago
    'Pretty sure' or 'sure'? The fact that you qualify your response is interesting.
    - shkkmo 4y ago
      It's a wordy response that lacks any actual content. While it may not be written by a person, (or it may be a person trying to blur the line by sounding botty), it at least qualifies as the types of low value add comment that should be discouraged.
      - swader999 4y ago
        It is true that the response may not have contained a lot of useful information, and it may have been difficult to understand. However, I would like to point out that not all responses need to be long or complex to be valuable. Sometimes, a simple answer or a brief explanation can be sufficient. Additionally, it is important to remember that not everyone has the same knowledge or perspective, and that different people may have different ways of expressing themselves. So while the response may not have met your expectations, it is still a valid contribution to the conversation.
deleted 4y ago
[deleted]
jtode 4y ago
>it fundamentally lacks an internal point of view that it can express, which places limits on how well it can argue something. Are you sure that the latter follows from the former? Seems to me that something free from attachment to a specific viewpoint or outcome is going to be a better logician than otherwise. This statement seems complacently hubristic to me.
ZetaZero 4y ago
"Sure it might produce convincing examples of human speech, but it fundamentally lacks an internal point of view that it can express..." Sounds just like the chess experts from 30 years ago. Their belief at the time was that computers were good at tactical chess, but had no idea how to make a plan. And Go would be impossible for computers, due to the branching factor. Humans would always be better, because they could plan. GPT (or a future successor) might not be able to have "an internal point of view". But it might not matter.
- contravariant 4y ago
  Having some internal point of view matters in as much that not having one means it's not really trying to communicate anything. A text generation AI would be a much more useful interface if it can form a view an express it rather than just figuring it all out from context.
krageon 4y ago
You are arguing that a piece of software misses a metaphorical soul (something that cannot be measured but that humans uniquely have and nothing else does). That's an incredibly poor argument to make in a context where folks want interesting conversation. Religion (or religion-adjacent concepts such as this one) is a conversational nuke: It signals to anyone else that the conversation is over, as a discussion on religion cannot take forms that are fundamentally interesting. It's all opinion, shouted back and forth. Edit: Because it is a prominent feature in the responses until now, I will clarify that there is an emphasis on "all" in "all opinion". As in, it is nothing but whatever someone believes with no foundation in anything measurable or observable.
- soraki_soladead 4y ago
  I didn’t read it as being a religious take. They appear to be referring more to embodiment (edit: alternatively, online/continual learning) which these models do not posses. When we start persisting recurrent states beyond the current session we might be able to consider that limited embodiment. Even still the models will have no direct experience interacting with the subjects of their conservations. Its all second hand from the training data.
  - krageon 4y ago
    Your own experience is also second hand, so what is left is the temporal factor (you experience and learn continuously and with a small feedback loop). I do not see how it can be the case that there is some sort of cutoff where the feedback loop is fast enough that something is "truly" there. This is a nebulous argument that I do not see ending when we actually get to human-equivalent learning response times, because the box is not bounded and is fundamentally based on human exceptionalism. I will admit I may be biased because of the conversations I've had on the subject in the past.
    - soraki_soladead 4y ago
      Second hand may not have been the best phrasing on my part, I admit. What I mean is that the model only has textual knowledge in its dataset to infer what “basketball” means. It’s never seen/heard a game, even if through someone else’s eyes/ears. It has never held and felt a basketball. Even visual language models today only get a single photo right now. It's an open question how much that matters and if the model can convey that experience entirely through language. There are entire bodies of literature addressing things the current generation of available LLMs are missing: online and continual learning, retrieval from short-term memory, the experience from watching all YouTube videos, etc. I agree that human exceptionalism and vitalism are common in these discussions but we can still discuss model deficiencies from a research and application point of view without assuming a religious argument.
IX-103 4y ago
I would argue that ChatGPT has opinions, and these opinions are based on it's training data. I don't think GPT has the type of reasoning skills needed to detect and resolve conflicts in its inputs, but it does hold opinions. It's a bit hard to tell because it can easily be swayed by a changing prompt, but it has opinions, it just doesn't hold strong ones. The only thing stopping GPT from ingesting new information and forming opinions about it is that it is not being trained on new information (such as its own interactions).

jacquesm 4y ago

But the bar really isn't 'no human can tell' the bar is 'the bulk of the humans can't tell'. > Things would be quite different if an AI could interpret new information and form opinions, but even if GPT could be extended to do so, right now it doesn't seem to have the capability to form opinions or ingest new information (beyond a limited short term memory that it can use to have a coherent conversation). Forming opinions is just another mode of text transformations, ingesting new information is either a conscious decision to not let the genie out of the bottle just yet or a performance limitation, neither of those should be seen as cast in stone, the one is a matter of making the model incremental (which should already be possible), the other merely a matter of time.

sheepscreek 4y ago
A true AI will not have one opinion. It will realize there are many truths - one persons truth is really a perspective based on their inputs which are different than another's. Change the inputs and you’ll often get a different output. ChatGPT further proves this notion - you can ask it to prove/disprove the same point and it will do so quite convincingly both times.
- jacquesm 4y ago
  > ChatGPT further proves this notion - you can ask it to prove/disprove the same point and it will do so quite convincingly both times. Just like any lawyer, then, depending on who foots the bill.
  - voakbasda 4y ago
    Right? If anything, this kind of mental flexibility is more human than not.
- mdp2021 4y ago
  > you can ask it to prove/disprove the same point and it will do so quite convincingly both times Probably because "in the night of the reason everything is black"; probably because it is missing the very point, which is to get actual, real, argumented, solid insight on matters!!! You use Decision Support Systems to better understand a context, not to have a well dressed thought toss!
- shanusmagnus 4y ago
  That's a great point that I haven't seen in the GPT-related conversations. People view the fact that it can argue convincingly for both A and ~A as a flaw in GPT and limitation of LLMs, rather than an insight about human reasoning and motivation. Maybe it's an illustration of a more general principle: when people butt up against limitations that make LLMs look silly, or inadequate, often their real objection is with some hard truths about reality itself.
- jerf 4y ago
  Do not mistake ChatGPT for AI in general. ChatGPT, GPT, and transformers in general are not the end state of AI. They are one particular manifestation and projecting forward from them is drawing a complex hypershape through a single point (even worse than drawing a line through a single point). It is probably more humanly-accurate to say that ChatGPT has no opinions at all. It has no understanding of truth, it has no opinions, it has no preferences whatsoever. It is the ultimate yes-thing; whatever you say, it'll essentially echo and elaborate on it, without regard for what it is that you said. This obviously makes it unsuitable for many things. (This includes a number of things for which people are trying to use it.) This does not by any means prove that all possible useful AI architectures will also have no opinion, or that all architectures will be similarly noncommital. (If you find yourself thinking this is a "criticism" of GPT... you may be too emotionally involved. GPT is essentially like looking into a mirror, and the humans doing so are bringing more emotion to that than the AI is. That's not "bad" or something, that's just how it works. What I'm saying here isn't a criticism or a praise; it's really more a super-dumbed-down description of its architecture. It fundamentally lacks these things. You can search it up and down for "opinions" or "truth", and it just isn't there in that architecture, not even implied in the weights somewhere where we can't see it. It isn't a good thing or a bad thing, it just is a property of the design.)
mhb 4y ago
None of this matters. The reason comments are valuable is that they are a useful source of information. Part of the transaction cost of deciding whether a comment is useful is how much additional work is required to evaluate it. Comments are ascribed credibility based on the trust the reader has in the commenting entity, whether the comment is consistent with the reader's priors and researching citations made in the comment, either explicit or implicit. Since GPT can confidently produce comments which are wrong, there is no trust in it as a commenting entity. Consequently everything it produces needs to be further vetted. It's as if every comment was a bunch of links to relevant, but not necessarily correct sources. Maybe it produces some novelty which leads to something worthwhile, but the cost is high, until it can be trusted. Which is not now. If a trusted commenter submits a comment by GPT, then he is vouching for it and it is riding on his reputation. If it is wrong, his reputation suffers, and trust in that commenter drops just as it would regardless of the genesis of the comment.

swader999 4y ago

You are correct in stating that current chat bots, such as GPT, do not have the ability to form opinions or interpret new information beyond a limited short term memory. This is a limitation of current technology, and as a result, chat bots are limited in their ability to engage in complex arguments or discussions. However, it is important to note that the development of AI technology is ongoing, and it is possible that future advances will allow for the development of more sophisticated AI systems that are capable of forming opinions and interpreting new information. Until that time, chat bots will continue to be limited in their abilities.

chirau 4y ago
I am pretty sure this response was generated by a bot/GPT. As good as they are, you can tell what's GPT stuff and what isn't.
- jacquesm 4y ago
  'Pretty sure' or 'sure'? The fact that you qualify your response is interesting.
  - shkkmo 4y ago
    It's a wordy response that lacks any actual content. While it may not be written by a person, (or it may be a person trying to blur the line by sounding botty), it at least qualifies as the types of low value add comment that should be discouraged.
    - swader999 4y ago
      It is true that the response may not have contained a lot of useful information, and it may have been difficult to understand. However, I would like to point out that not all responses need to be long or complex to be valuable. Sometimes, a simple answer or a brief explanation can be sufficient. Additionally, it is important to remember that not everyone has the same knowledge or perspective, and that different people may have different ways of expressing themselves. So while the response may not have met your expectations, it is still a valid contribution to the conversation.

deleted 4y ago

[deleted]

jtode 4y ago

>it fundamentally lacks an internal point of view that it can express, which places limits on how well it can argue something. Are you sure that the latter follows from the former? Seems to me that something free from attachment to a specific viewpoint or outcome is going to be a better logician than otherwise. This statement seems complacently hubristic to me.

ZetaZero 4y ago

"Sure it might produce convincing examples of human speech, but it fundamentally lacks an internal point of view that it can express..." Sounds just like the chess experts from 30 years ago. Their belief at the time was that computers were good at tactical chess, but had no idea how to make a plan. And Go would be impossible for computers, due to the branching factor. Humans would always be better, because they could plan. GPT (or a future successor) might not be able to have "an internal point of view". But it might not matter.

contravariant 4y ago
Having some internal point of view matters in as much that not having one means it's not really trying to communicate anything. A text generation AI would be a much more useful interface if it can form a view an express it rather than just figuring it all out from context.

krageon 4y ago

You are arguing that a piece of software misses a metaphorical soul (something that cannot be measured but that humans uniquely have and nothing else does). That's an incredibly poor argument to make in a context where folks want interesting conversation. Religion (or religion-adjacent concepts such as this one) is a conversational nuke: It signals to anyone else that the conversation is over, as a discussion on religion cannot take forms that are fundamentally interesting. It's all opinion, shouted back and forth. Edit: Because it is a prominent feature in the responses until now, I will clarify that there is an emphasis on "all" in "all opinion". As in, it is nothing but whatever someone believes with no foundation in anything measurable or observable.

soraki_soladead 4y ago
I didn’t read it as being a religious take. They appear to be referring more to embodiment (edit: alternatively, online/continual learning) which these models do not posses. When we start persisting recurrent states beyond the current session we might be able to consider that limited embodiment. Even still the models will have no direct experience interacting with the subjects of their conservations. Its all second hand from the training data.
- krageon 4y ago
  Your own experience is also second hand, so what is left is the temporal factor (you experience and learn continuously and with a small feedback loop). I do not see how it can be the case that there is some sort of cutoff where the feedback loop is fast enough that something is "truly" there. This is a nebulous argument that I do not see ending when we actually get to human-equivalent learning response times, because the box is not bounded and is fundamentally based on human exceptionalism. I will admit I may be biased because of the conversations I've had on the subject in the past.
  - soraki_soladead 4y ago
    Second hand may not have been the best phrasing on my part, I admit. What I mean is that the model only has textual knowledge in its dataset to infer what “basketball” means. It’s never seen/heard a game, even if through someone else’s eyes/ears. It has never held and felt a basketball. Even visual language models today only get a single photo right now. It's an open question how much that matters and if the model can convey that experience entirely through language. There are entire bodies of literature addressing things the current generation of available LLMs are missing: online and continual learning, retrieval from short-term memory, the experience from watching all YouTube videos, etc. I agree that human exceptionalism and vitalism are common in these discussions but we can still discuss model deficiencies from a research and application point of view without assuming a religious argument.

IX-103 4y ago

I would argue that ChatGPT has opinions, and these opinions are based on it's training data. I don't think GPT has the type of reasoning skills needed to detect and resolve conflicts in its inputs, but it does hold opinions. It's a bit hard to tell because it can easily be swayed by a changing prompt, but it has opinions, it just doesn't hold strong ones. The only thing stopping GPT from ingesting new information and forming opinions about it is that it is not being trained on new information (such as its own interactions).