4 ms·

One of the things that always gives me a little hit of hype is when I tell it to do something ridiculous and it just dutifully starts spitting out the result wi

One of the things that always gives me a little hit of hype is when I tell it to do something ridiculous and it just dutifully starts spitting out the result without complaining or questioning lol

felipemnoa 4y ago
I wonder if that is how our brain produces dreams? The guardrails are down so it will just start producing ridiculous and/or implausible things. Edit: It almost seems like you are anthropomorphizing it. It is just a program doing what it's supposed to be doing: to predict the next token based on its weights. Nothing more, nothing less. It does give the illusion of intelligence. Pretty soon, though, we may not be able to tell the difference.
- JKCalhoun 4y ago
  > It is just a program doing what it's supposed to be doing: to predict the next token based on its weights. Nothing more, nothing less. Every time I see a comment along these lines it gives me pause: there is a built-in assumption that each of us is somehow doing something more than this. I'm not convinced. I've heard people refer to some of our instinctive behaviors as due to "our lizard brain", suggesting that our brains are hierarchical, or comprised of a series of evolutionary steps, a more evolved order of brain wrapping the more primitive. I increasingly suspect that ChatGPT has more or less nailed one of those layers.
  - hnfong 4y ago
    To add to your point, note that a language model that "predicts the next token" accurately (as in predicting what human text would say) would pass the Turing test by definition. One may argue what passing the Turing test means, but at least by definition it is mimicking human intelligence in some way.
  - chankstein38 4y ago
    YES! I feel the same! I frequently see myself make the same kinds of mistakes I see ChatGPT make or I just go overly deep into thought and start thinking about the fact that, especially when communicating, I'm not prethinking my words or anything I'm just coming up with the next word on the fly. Sometimes I lose my train of thought mid conversation and end up seeing a sentence like "it makes me happy that.." and end up finishing it out of context before realizing I was talking about something else. It's the same with speech, sometimes if I'm looking for something nice to say to someone but nothing is top of mind I'll just seed myself by starting a sentence "You are great honestly I was just thinking about how..." and then just hope my brain finishes it for me. Usually it does but sometimes it ends in me explaining why I stopped haha
ojosilva 4y ago
I was thinking exactly the same as I read the OP, right where the dad+kid were answering hypothetical ChatGPT questions with Yes and No. I think LLMs inference training should include teaching it to ask questions back before starting full-fledged generation. You know, make it a little more Socratic. Right now the approach is: ChatGPT starts answering and, if it's going the wrong way, you either hit "Stop Generating" or just wait for it to finish then figure yourself how to improve the prompt. LLMs should be also trained in ranking the prompt and determining what questions would make the prompt statistically stronger to generate. I bet it would result in savings running it too. In fact, one can try this out by configuring a system prompt that tells the model to ask questions before getting started with an answer.
- chankstein38 4y ago
  I've seen people do this with some prompt engineering! One I saw was for tuning prompts, you told GPT to ask questions which could help it better understand the prompt then incorporate your answers into a new prompt which could be better then loop that until it hits a certain score or rating! It does feel like it should be more commonplace or even just the basis of how these things function because sometimes it just goes off on something completely unrelated because my prompt was bad haha

One of the things that always gives me a little hit of hype is when I tell it to do something ridiculous and it just dutifully starts spitting out the result without complaining or questioning lol

felipemnoa 4y ago

I wonder if that is how our brain produces dreams? The guardrails are down so it will just start producing ridiculous and/or implausible things. Edit: It almost seems like you are anthropomorphizing it. It is just a program doing what it's supposed to be doing: to predict the next token based on its weights. Nothing more, nothing less. It does give the illusion of intelligence. Pretty soon, though, we may not be able to tell the difference.

JKCalhoun 4y ago
> It is just a program doing what it's supposed to be doing: to predict the next token based on its weights. Nothing more, nothing less. Every time I see a comment along these lines it gives me pause: there is a built-in assumption that each of us is somehow doing something more than this. I'm not convinced. I've heard people refer to some of our instinctive behaviors as due to "our lizard brain", suggesting that our brains are hierarchical, or comprised of a series of evolutionary steps, a more evolved order of brain wrapping the more primitive. I increasingly suspect that ChatGPT has more or less nailed one of those layers.
- hnfong 4y ago
  To add to your point, note that a language model that "predicts the next token" accurately (as in predicting what human text would say) would pass the Turing test by definition. One may argue what passing the Turing test means, but at least by definition it is mimicking human intelligence in some way.
- chankstein38 4y ago
  YES! I feel the same! I frequently see myself make the same kinds of mistakes I see ChatGPT make or I just go overly deep into thought and start thinking about the fact that, especially when communicating, I'm not prethinking my words or anything I'm just coming up with the next word on the fly. Sometimes I lose my train of thought mid conversation and end up seeing a sentence like "it makes me happy that.." and end up finishing it out of context before realizing I was talking about something else. It's the same with speech, sometimes if I'm looking for something nice to say to someone but nothing is top of mind I'll just seed myself by starting a sentence "You are great honestly I was just thinking about how..." and then just hope my brain finishes it for me. Usually it does but sometimes it ends in me explaining why I stopped haha

ojosilva 4y ago

I was thinking exactly the same as I read the OP, right where the dad+kid were answering hypothetical ChatGPT questions with Yes and No. I think LLMs inference training should include teaching it to ask questions back before starting full-fledged generation. You know, make it a little more Socratic. Right now the approach is: ChatGPT starts answering and, if it's going the wrong way, you either hit "Stop Generating" or just wait for it to finish then figure yourself how to improve the prompt. LLMs should be also trained in ranking the prompt and determining what questions would make the prompt statistically stronger to generate. I bet it would result in savings running it too. In fact, one can try this out by configuring a system prompt that tells the model to ask questions before getting started with an answer.

chankstein38 4y ago
I've seen people do this with some prompt engineering! One I saw was for tuning prompts, you told GPT to ask questions which could help it better understand the prompt then incorporate your answers into a new prompt which could be better then loop that until it hits a certain score or rating! It does feel like it should be more commonplace or even just the basis of how these things function because sometimes it just goes off on something completely unrelated because my prompt was bad haha