3 ms·

> ChatGPT is still just an interface to for text completion. These are really weird takes tbh. It's a large NLP model. If I want to, I can just say everything

> ChatGPT is still just an interface to for text completion. These are really weird takes tbh. It's a large NLP model. If I want to, I can just say everything we do is to have an interface for idea completion. It's a huge step in technology, and I'm not sure what we get from selling it short.

swatcoder 4y ago
It's incredible technology, but that doesn't change that everything it does can and should be treated as text completion. If it's not doing what you want, you think about how you made that happen by shaping the dialog it was trying to write with you. If it commits an error, you think about how that error will impact the remainder of the dialog its trying to write with you and how your own own responses will shape that. Text and ideas are not equivalent. Even if everything we do is "idea completion" (whatever that means), that's not what it does. Maybe it's a step towards that, but for now it just does the text thing and it's really f--ing great at it when you remember that that's what it's doing.
whalesalad 4y ago
from a technical standpoint chatgpt is honestly very simple. this is not selling it short. there is a reason that you can run things like stable diffusion on your laptop - current AI is really not that sophisticated.
retrac 4y ago
It's precisely because it's so amazing that we need to sell it short. It really is basically a statistical completion engine. Auto-suggest on steroids. By some interpretations, language models are without any semantic properties. It does not reason. It does not do symbolic manipulation. There is no chain of thought, or argument, as we understand those things, on the way to producing its output. There are no easily-identified objects in its model that map to tokens or objects related to something in the outside world. It is predicting, with some very fancy statistics, the most probable completion to a prompt, based on the patterns in its training input. This is not a dismissal. I think it makes what it can do all the more impressive, and perhaps unsettling. Until recently, I believed something like this could never be accomplished without great advances in more traditional AI - stuff like fuzzy symbolic reasoning and programmed-in general knowledge databases. That ChatGPT lacks such should be kept in mind. Because people seem to frequently fall for the illusion that it is reasoning, that it is doing symbolic manipulation of tokens, etc. It is not. It can't even remember or learn anything! There is no state held between prompt invocations. The architecture just feeds in the recent backlog when you run the next prompt, to help guide it with context. So it's little surprise that with the right prompts it will "lie" and then deny it. It doesn't have a concept of truth nor any memory.
- williamcotton 4y ago
  Large language models do seem to have some deeper analogies with classically human qualities, including a shared metaphysics: https://www.williamcotton.com/articles/chatgpt-and-the-analytic-synthetic-distinction https://www.williamcotton.com/articles/chatgpt-and-the-analy...
bicx 4y ago
Text completion is what the underlying mechanism is called. This is from the "Get Started" docs for GPT-3, which is the underlying model for what is used to build ChatGPT (technically they've been calling the model GPT "3.5"): https://platform.openai.com/docs/guides/completion/introduction https://platform.openai.com/docs/guides/completion/introduct...

swatcoder 4y ago

It's incredible technology, but that doesn't change that everything it does can and should be treated as text completion. If it's not doing what you want, you think about how you made that happen by shaping the dialog it was trying to write with you. If it commits an error, you think about how that error will impact the remainder of the dialog its trying to write with you and how your own own responses will shape that. Text and ideas are not equivalent. Even if everything we do is "idea completion" (whatever that means), that's not what it does. Maybe it's a step towards that, but for now it just does the text thing and it's really f--ing great at it when you remember that that's what it's doing.

whalesalad 4y ago

from a technical standpoint chatgpt is honestly very simple. this is not selling it short. there is a reason that you can run things like stable diffusion on your laptop - current AI is really not that sophisticated.

retrac 4y ago

It's precisely because it's so amazing that we need to sell it short. It really is basically a statistical completion engine. Auto-suggest on steroids. By some interpretations, language models are without any semantic properties. It does not reason. It does not do symbolic manipulation. There is no chain of thought, or argument, as we understand those things, on the way to producing its output. There are no easily-identified objects in its model that map to tokens or objects related to something in the outside world. It is predicting, with some very fancy statistics, the most probable completion to a prompt, based on the patterns in its training input. This is not a dismissal. I think it makes what it can do all the more impressive, and perhaps unsettling. Until recently, I believed something like this could never be accomplished without great advances in more traditional AI - stuff like fuzzy symbolic reasoning and programmed-in general knowledge databases. That ChatGPT lacks such should be kept in mind. Because people seem to frequently fall for the illusion that it is reasoning, that it is doing symbolic manipulation of tokens, etc. It is not. It can't even remember or learn anything! There is no state held between prompt invocations. The architecture just feeds in the recent backlog when you run the next prompt, to help guide it with context. So it's little surprise that with the right prompts it will "lie" and then deny it. It doesn't have a concept of truth nor any memory.

williamcotton 4y ago
Large language models do seem to have some deeper analogies with classically human qualities, including a shared metaphysics: https://www.williamcotton.com/articles/chatgpt-and-the-analytic-synthetic-distinction https://www.williamcotton.com/articles/chatgpt-and-the-analy...

bicx 4y ago

Text completion is what the underlying mechanism is called. This is from the "Get Started" docs for GPT-3, which is the underlying model for what is used to build ChatGPT (technically they've been calling the model GPT "3.5"): https://platform.openai.com/docs/guides/completion/introduction https://platform.openai.com/docs/guides/completion/introduct...