5 ms·

ChatGPT is still just an interface to for text completion. As you "catch it" in some inconsistency and antagonize it, you're guiding it to complete a new docum

ChatGPT is still just an interface to for text completion. As you "catch it" in some inconsistency and antagonize it, you're guiding it to complete a new document where you play the role of accuser and it plays the role of buffoon or denier. The more aggravated or clever you get, the more innane and stubborn it gets. Not because it's an intelligent agent trying to hide something but because that's what this sort of dialog generally looks like in its training data and with the seeded context it starts with ("helpful AI, doesn't know current stuff", etc). It doesn't know what the f--- is going on because it doesn't know anything. It's just trying to complete a dialog that looks the way you're hinting that it should look. Hopefully, experiences like this can be explained to people so that they understand how simple and limited it actually is. It's not "intelligent" or "a liar" or any such thing -- it's just a committed improvisor with a big (enormous) catalog of text samples to reference when composing a creative dialog with you.

gmaster1440 4y ago
Wrote about this here[1], seems relevant to the topic at hand. [1] https://www.markfayngersh.com/posts/the-thinking-placebo https://www.markfayngersh.com/posts/the-thinking-placebo
aaomidi 4y ago
> ChatGPT is still just an interface to for text completion. These are really weird takes tbh. It's a large NLP model. If I want to, I can just say everything we do is to have an interface for idea completion. It's a huge step in technology, and I'm not sure what we get from selling it short.
- swatcoder 4y ago
  It's incredible technology, but that doesn't change that everything it does can and should be treated as text completion. If it's not doing what you want, you think about how you made that happen by shaping the dialog it was trying to write with you. If it commits an error, you think about how that error will impact the remainder of the dialog its trying to write with you and how your own own responses will shape that. Text and ideas are not equivalent. Even if everything we do is "idea completion" (whatever that means), that's not what it does. Maybe it's a step towards that, but for now it just does the text thing and it's really f--ing great at it when you remember that that's what it's doing.
- whalesalad 4y ago
  from a technical standpoint chatgpt is honestly very simple. this is not selling it short. there is a reason that you can run things like stable diffusion on your laptop - current AI is really not that sophisticated.
- retrac 4y ago
  It's precisely because it's so amazing that we need to sell it short. It really is basically a statistical completion engine. Auto-suggest on steroids. By some interpretations, language models are without any semantic properties. It does not reason. It does not do symbolic manipulation. There is no chain of thought, or argument, as we understand those things, on the way to producing its output. There are no easily-identified objects in its model that map to tokens or objects related to something in the outside world. It is predicting, with some very fancy statistics, the most probable completion to a prompt, based on the patterns in its training input. This is not a dismissal. I think it makes what it can do all the more impressive, and perhaps unsettling. Until recently, I believed something like this could never be accomplished without great advances in more traditional AI - stuff like fuzzy symbolic reasoning and programmed-in general knowledge databases. That ChatGPT lacks such should be kept in mind. Because people seem to frequently fall for the illusion that it is reasoning, that it is doing symbolic manipulation of tokens, etc. It is not. It can't even remember or learn anything! There is no state held between prompt invocations. The architecture just feeds in the recent backlog when you run the next prompt, to help guide it with context. So it's little surprise that with the right prompts it will "lie" and then deny it. It doesn't have a concept of truth nor any memory.
  - williamcotton 4y ago
    Large language models do seem to have some deeper analogies with classically human qualities, including a shared metaphysics: https://www.williamcotton.com/articles/chatgpt-and-the-analytic-synthetic-distinction https://www.williamcotton.com/articles/chatgpt-and-the-analy...
- bicx 4y ago
  Text completion is what the underlying mechanism is called. This is from the "Get Started" docs for GPT-3, which is the underlying model for what is used to build ChatGPT (technically they've been calling the model GPT "3.5"): https://platform.openai.com/docs/guides/completion/introduction https://platform.openai.com/docs/guides/completion/introduct...
1lint 4y ago
I'm curious, how do you determine whether ChatGPT "knows" or does not "know" anything? What would an AI model have to demonstrate to convince you that it "knows" something, or that it is "intelligent", or that it is "a liar"?
- swatcoder 4y ago
  That's a great question. I'll admit that I haven't even figured out what it means for anything to "know" something. But everything I've learned about, witnessed, and experienced with ChatGPT is consistent with a far more mechanical process than I experience with myself, my friends, or my community. I find it easy to spot the wires above the marionette and find there workings to be pretty intuitive. I have a comfortable sense of which ones to tug to make it dance the way I want, and when it's producing something unusual I've found that it doesn't take a lot of work to come up with a consistent, mechanical theory of how. With people, that sense of clarity and comprehensibility hasn't really become apparent despite many decades of engaging with a bunch of them. There seems to be something far more sophisticated going on under the hood there, and they consistently surprise me and confound me in ways that ChatGPT doesn't even approach. Maybe some AI model will achieve the same thing someday, and maybe that day will be soon, but for now ChatGPT looks and acts exactly like what the associated research papers would suggest.
  - 1lint 4y ago
    > I'll admit that I haven't even figured out what it means for anything to "know" something. Same! That's the overhanging concern I have when hearing comparisons made between artificial and human intelligence. I don't feel that human knowledge/intelligence is well defined enough to the point where we can meaningfully compare it to alternative systems. But my gut instinct is that our neural architecture is composed of mechanistic biological compounds that gradually evolved sufficiently complex levels of organization to achieve our current level of cognition. Capacity for knowledge/intelligence/consciousness did not suddenly appear at some instant in our evolutionary history, rather it gradually emerged from increasing levels of neural complexity. I would rationalize the development of AI capacity for cognition in a similar light, that it will gradually emerge from increasing levels of complexity.
disqard 4y ago
> it's just a committed improviser with a big (enormous) catalog of text samples to reference when composing a creative dialog with you. That's a great description! When feeling generous, I've called it "a sentence generator and idea exploration assistant". As for my uncharitable take: it's "a first-rate bullshit generator".

gmaster1440 4y ago

Wrote about this here[1], seems relevant to the topic at hand. [1] https://www.markfayngersh.com/posts/the-thinking-placebo https://www.markfayngersh.com/posts/the-thinking-placebo

aaomidi 4y ago

> ChatGPT is still just an interface to for text completion. These are really weird takes tbh. It's a large NLP model. If I want to, I can just say everything we do is to have an interface for idea completion. It's a huge step in technology, and I'm not sure what we get from selling it short.

swatcoder 4y ago
It's incredible technology, but that doesn't change that everything it does can and should be treated as text completion. If it's not doing what you want, you think about how you made that happen by shaping the dialog it was trying to write with you. If it commits an error, you think about how that error will impact the remainder of the dialog its trying to write with you and how your own own responses will shape that. Text and ideas are not equivalent. Even if everything we do is "idea completion" (whatever that means), that's not what it does. Maybe it's a step towards that, but for now it just does the text thing and it's really f--ing great at it when you remember that that's what it's doing.
whalesalad 4y ago
from a technical standpoint chatgpt is honestly very simple. this is not selling it short. there is a reason that you can run things like stable diffusion on your laptop - current AI is really not that sophisticated.
retrac 4y ago
It's precisely because it's so amazing that we need to sell it short. It really is basically a statistical completion engine. Auto-suggest on steroids. By some interpretations, language models are without any semantic properties. It does not reason. It does not do symbolic manipulation. There is no chain of thought, or argument, as we understand those things, on the way to producing its output. There are no easily-identified objects in its model that map to tokens or objects related to something in the outside world. It is predicting, with some very fancy statistics, the most probable completion to a prompt, based on the patterns in its training input. This is not a dismissal. I think it makes what it can do all the more impressive, and perhaps unsettling. Until recently, I believed something like this could never be accomplished without great advances in more traditional AI - stuff like fuzzy symbolic reasoning and programmed-in general knowledge databases. That ChatGPT lacks such should be kept in mind. Because people seem to frequently fall for the illusion that it is reasoning, that it is doing symbolic manipulation of tokens, etc. It is not. It can't even remember or learn anything! There is no state held between prompt invocations. The architecture just feeds in the recent backlog when you run the next prompt, to help guide it with context. So it's little surprise that with the right prompts it will "lie" and then deny it. It doesn't have a concept of truth nor any memory.
- williamcotton 4y ago
  Large language models do seem to have some deeper analogies with classically human qualities, including a shared metaphysics: https://www.williamcotton.com/articles/chatgpt-and-the-analytic-synthetic-distinction https://www.williamcotton.com/articles/chatgpt-and-the-analy...
bicx 4y ago
Text completion is what the underlying mechanism is called. This is from the "Get Started" docs for GPT-3, which is the underlying model for what is used to build ChatGPT (technically they've been calling the model GPT "3.5"): https://platform.openai.com/docs/guides/completion/introduction https://platform.openai.com/docs/guides/completion/introduct...

1lint 4y ago

I'm curious, how do you determine whether ChatGPT "knows" or does not "know" anything? What would an AI model have to demonstrate to convince you that it "knows" something, or that it is "intelligent", or that it is "a liar"?

swatcoder 4y ago
That's a great question. I'll admit that I haven't even figured out what it means for anything to "know" something. But everything I've learned about, witnessed, and experienced with ChatGPT is consistent with a far more mechanical process than I experience with myself, my friends, or my community. I find it easy to spot the wires above the marionette and find there workings to be pretty intuitive. I have a comfortable sense of which ones to tug to make it dance the way I want, and when it's producing something unusual I've found that it doesn't take a lot of work to come up with a consistent, mechanical theory of how. With people, that sense of clarity and comprehensibility hasn't really become apparent despite many decades of engaging with a bunch of them. There seems to be something far more sophisticated going on under the hood there, and they consistently surprise me and confound me in ways that ChatGPT doesn't even approach. Maybe some AI model will achieve the same thing someday, and maybe that day will be soon, but for now ChatGPT looks and acts exactly like what the associated research papers would suggest.
- 1lint 4y ago
  > I'll admit that I haven't even figured out what it means for anything to "know" something. Same! That's the overhanging concern I have when hearing comparisons made between artificial and human intelligence. I don't feel that human knowledge/intelligence is well defined enough to the point where we can meaningfully compare it to alternative systems. But my gut instinct is that our neural architecture is composed of mechanistic biological compounds that gradually evolved sufficiently complex levels of organization to achieve our current level of cognition. Capacity for knowledge/intelligence/consciousness did not suddenly appear at some instant in our evolutionary history, rather it gradually emerged from increasing levels of neural complexity. I would rationalize the development of AI capacity for cognition in a similar light, that it will gradually emerge from increasing levels of complexity.

disqard 4y ago

> it's just a committed improviser with a big (enormous) catalog of text samples to reference when composing a creative dialog with you. That's a great description! When feeling generous, I've called it "a sentence generator and idea exploration assistant". As for my uncharitable take: it's "a first-rate bullshit generator".