6 ms·

If you got an invite for GPT-3, give it a shot. I discounted it at first, but then I gave it a few shots and was actually crept out a bit. Even though it is "ra

by csomar 5y ago

If you got an invite for GPT-3, give it a shot. I discounted it at first, but then I gave it a few shots and was actually crept out a bit. Even though it is "randomly" making things up as it goes, it does show what seems like intelligence just from the sheer amount of data it is trained on. One thing I was amazed by: GPT-3 could be a great autocompletion engine for any programming language or configuration schema. Things like Grub configuration file, xkb file could be intuitively completed by GTP-3. And even more: GTP-3 could build basic "concepts" and apply them to that domain knowledge. This seems to emerge naturally rather than something pre-planned by OpenAI. After all, I don't think OpenAI has planned for GPT-3 to understand xkb keyboard layouts.

lolthishuman 5y ago
You make up things randomly as you go. You never thought ahead of any thought. Every thought you’ve ever had is essentially a procedurally generated prompt based on your biased models.
- still_grokking 5y ago
  > You make up things randomly as you go. You never thought ahead of any thought. I don't think that's true. Usually you need to think a lot about something before coming up with "the right" thought(s). > Every thought you’ve ever had is essentially a procedurally generated prompt based on your biased models. That may be. But the interesting part is that those models change as you use them just by using them.
  - antris 5y ago
    > I don't think that's true. How long did you have to think to produce that thought? Or did it just pop into your head instantly? The point is, you cannot think of an upcoming thought, before you have it in your head. Otherwise you would be seeing into the future. What you are talking about in your comment is reaching a conclusion based on previous thoughts. Yes, often we link our thoughts together into a narrative or a conclusion after we've had the thoughts, but the thoughts themselves? Those seem to come out of nowhere.
    - still_grokking 5y ago
      The mind does a lot of unconscious work before coming up with some conscious results. That's true even for very simple things like motion. You can measure things in the brain before those things become conscious thoughts. (Those experiments caused by the way a lot of fuss whether we have free will or are completely predestined in all we do; but that's another topic). The consciousness only observes a small portion of the thought process. So for it a lot of thoughts seem to come out of nowhere. But the unconscious parts of thinking are very important to the whole process and it's outcomes. I think nobody disputes this by this time.
    - unbroken 5y ago
      I love The Darkness that Comes Before, where this observation is explored and exploited, in case you have not read it.
- nutanc 5y ago
  I have used GPT-3 and it works for most of the time. But it fails some of the time too. And thats the problem for use cases like programming or generating config files. Because if you cant trust the output 100% you are pretty much reading the output every time. So, the only time save is that GPT-3 makes you type less. In any case I don't type much anyway now a days. Its mostly copy paste from stack overflow update parameters etc. GPT-3 will be useful, maybe a year from now.
- FeepingCreature 5y ago
  Sure, but we can think behind our thoughts. And we can sound things out before we say them. And we have mutable long-term memory. There's not much between us and GPT, but there is some distance still.
- zaptrem 5y ago
  I can't help but think of this scene in Westworld (spoiler S1) whenever GPT 3 (or earlier text prediction models) and this topic come up together: https://www.youtube.com/watch?v=ZnxJRYit44k https://www.youtube.com/watch?v=ZnxJRYit44k
- Rioghasarig 5y ago
  That's a pretty bold model for human cognition. It's not something you can just assume.
nerdponx 5y ago
Keep in mind that it's somewhere between "random" and "intelligent". It's more or less very complicated fuzzy pattern matching. I do like the idea of generating configuration files, at least as a starting point for users in applications with big complicated configuration set ups. As with all things fuzzy, the output probably won't be perfect, but it might help users save time in getting set up.
- FeepingCreature 5y ago
  Very complicated multilayer fuzzy pattern matching. If you look at it that way, it's not dissimilar to the brain.
  - chucklenorris 5y ago
    not really, the brain can verify the correctness of the pattern matching, and use that to infer other possibly correct patterns. also this model can't really infer intentionality or discern between variants and weigh in pros and cons. that being said i think we're not far from agi, we just need a few more pieces
    - FeepingCreature 5y ago
      It can sort of discern between variants. And intentionality and pros/cons are just another kind of pattern. What it cannot do is any kind of recursive, reflective reasoning (except by unrolling).
  - deleted 5y ago
    [deleted]
  - mensetmanusman 5y ago
    If only we knew how the brain worked…
macrolime 5y ago
It's the same with gpt-3 though. All demos show only where it works well. Only when you get to try it yourself do you get to explore all the many areas it fails.
- Kiro 5y ago
  The comment you're replying to literally says the opposite thing.

by csomar 5y ago

lolthishuman 5y ago

You make up things randomly as you go. You never thought ahead of any thought. Every thought you’ve ever had is essentially a procedurally generated prompt based on your biased models.

still_grokking 5y ago
> You make up things randomly as you go. You never thought ahead of any thought. I don't think that's true. Usually you need to think a lot about something before coming up with "the right" thought(s). > Every thought you’ve ever had is essentially a procedurally generated prompt based on your biased models. That may be. But the interesting part is that those models change as you use them just by using them.
- antris 5y ago
  > I don't think that's true. How long did you have to think to produce that thought? Or did it just pop into your head instantly? The point is, you cannot think of an upcoming thought, before you have it in your head. Otherwise you would be seeing into the future. What you are talking about in your comment is reaching a conclusion based on previous thoughts. Yes, often we link our thoughts together into a narrative or a conclusion after we've had the thoughts, but the thoughts themselves? Those seem to come out of nowhere.
  - still_grokking 5y ago
    The mind does a lot of unconscious work before coming up with some conscious results. That's true even for very simple things like motion. You can measure things in the brain before those things become conscious thoughts. (Those experiments caused by the way a lot of fuss whether we have free will or are completely predestined in all we do; but that's another topic). The consciousness only observes a small portion of the thought process. So for it a lot of thoughts seem to come out of nowhere. But the unconscious parts of thinking are very important to the whole process and it's outcomes. I think nobody disputes this by this time.
  - unbroken 5y ago
    I love The Darkness that Comes Before, where this observation is explored and exploited, in case you have not read it.
nutanc 5y ago
I have used GPT-3 and it works for most of the time. But it fails some of the time too. And thats the problem for use cases like programming or generating config files. Because if you cant trust the output 100% you are pretty much reading the output every time. So, the only time save is that GPT-3 makes you type less. In any case I don't type much anyway now a days. Its mostly copy paste from stack overflow update parameters etc. GPT-3 will be useful, maybe a year from now.
FeepingCreature 5y ago
Sure, but we can think behind our thoughts. And we can sound things out before we say them. And we have mutable long-term memory. There's not much between us and GPT, but there is some distance still.
zaptrem 5y ago
I can't help but think of this scene in Westworld (spoiler S1) whenever GPT 3 (or earlier text prediction models) and this topic come up together: https://www.youtube.com/watch?v=ZnxJRYit44k https://www.youtube.com/watch?v=ZnxJRYit44k
Rioghasarig 5y ago
That's a pretty bold model for human cognition. It's not something you can just assume.

nerdponx 5y ago

Keep in mind that it's somewhere between "random" and "intelligent". It's more or less very complicated fuzzy pattern matching. I do like the idea of generating configuration files, at least as a starting point for users in applications with big complicated configuration set ups. As with all things fuzzy, the output probably won't be perfect, but it might help users save time in getting set up.

FeepingCreature 5y ago
Very complicated multilayer fuzzy pattern matching. If you look at it that way, it's not dissimilar to the brain.
- chucklenorris 5y ago
  not really, the brain can verify the correctness of the pattern matching, and use that to infer other possibly correct patterns. also this model can't really infer intentionality or discern between variants and weigh in pros and cons. that being said i think we're not far from agi, we just need a few more pieces
  - FeepingCreature 5y ago
    It can sort of discern between variants. And intentionality and pros/cons are just another kind of pattern. What it cannot do is any kind of recursive, reflective reasoning (except by unrolling).
- deleted 5y ago
  [deleted]
- mensetmanusman 5y ago
  If only we knew how the brain worked…

macrolime 5y ago

It's the same with gpt-3 though. All demos show only where it works well. Only when you get to try it yourself do you get to explore all the many areas it fails.

Kiro 5y ago
The comment you're replying to literally says the opposite thing.