4 ms·

It is, though, by the same argument also collectively dumber than the dumbest person who ever lived. No single human moron's brain could accommodate so much mis

by _dwt 3y ago

It is, though, by the same argument also collectively dumber than the dumbest person who ever lived. No single human moron's brain could accommodate so much misinformation, useless fluff, spurious reasoning, etc. etc. etc. So I think time will tell. My money is on a sort of regression to the mean: these models will capture the style and "creativity" of the average 2020s Reddit, StackOverflow, etc. user. I can't say I'm terribly excited.

jstx1 3y ago
> It is, though, by the same argument also collectively dumber than the dumbest person who ever lived. Not at all, it's not symmetrical. You're ignoring the training data and all the additional RLHF fine tuning. The model is being actively penalized for being dumb which is why it isn't that dumb in a lot of cases.
- grayhatter 3y ago
  but the popular models plastered across HN are in fact, dumb in a lot of cases. Everyone is impressed when the model does something they don't understand deeply. But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply. I do think it's slightly better than the mean across all topics. But I also strongly suspect it'll soon serve as a great example for regressing to the mean.
  - jstx1 3y ago
    > But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply. This statement sounds out of date. And you can see this sentiment a lot on HN. I don’t know if the people who say this haven’t tried GPT-4 or they have and are just stubbornly refusing to change their mind about something when presented with new evidence.
    - grayhatter 3y ago
      > s haven’t tried GPT-4 or they have and are just stubbornly refusing to change their mind about something when presented with new evidence. Have you considered that the evidence just isn't convincing yet? If you view popular llms as text prediction machines, they are in fact much better than spell check or auto correct from a few years ago. But if you actually ask it to solve the problem with nuance it will not use nuance. That's the part that would impress me. As a recent example if you ask chat GPT how to use ffmpeg to slice out a video. and you tell it that you only want 3 seconds of video. somebody who deeply understands how ffmpeg works, (or even someone who deeply read the documentation) would point out that you have to be aware that it can only cut to keyframes. so if the keyframe is not aligned to the time you ask for you will not get the video that you expect. another example ask it to play 20 questions with you, it will cheat at the end. even if you give it very specific instructions it still is unable to follow them to the fair conclusion of the game. (or it will make a mistake and understanding about some of the semantics of the question, but I don't fault it for a difference in context) I'd caution you that just because you're impressed for the subjects that you understand deeply does not mean that chat gpt is good at all subjects. and therefore I assert that it is disrespectful to be so dismissive of people who have different opinions than yours. I believe the default should be to assume good faith rather than dismissiveness "they just don't understand"
      - jstx1 3y ago
        ChatGPT with GPT-4 with just "how to use ffmpeg to slice out a video" pasted directly from your comment without any additional prompting or clarification brought up your point about the keyframes. So whatever you think you're criticising isn't the same thing that I'm using. Kind of demonstrates my point that you're either using an older version or choosing to ignore evidence for some reason.
        grayhatter 3y ago
        It doesn't impress me that once corrected (which it has been multiple times), that it stops giving wrong answers. If it was able to detect and supply nuance to something novel. Then I might be impressed. BTW: when I posed this question > how do I use ffmpeg to slice out a video, I only want 3 seconds of video from the middle to chatgpt4, it did not mention anything about accounting for keyframes. So I'm not sure what version you're using, but it's not the one I have access to :/ Another reason to assume good faith, because you seem to have access to something that I don't. And, this is now (well before this specific conversation) an expired example because openai's model has been specifically training on this nuance, and thus even if it never made a mistake around it, it still wouldn't be impressive to me; simply because this nuance was directly added to it's model. Remember, we're talking about if the abilities of available models are impressive, what would be impressive to me is if they're able to generalize well enough to know things they haven't been directly taught.

by _dwt 3y ago

jstx1 3y ago

> It is, though, by the same argument also collectively dumber than the dumbest person who ever lived. Not at all, it's not symmetrical. You're ignoring the training data and all the additional RLHF fine tuning. The model is being actively penalized for being dumb which is why it isn't that dumb in a lot of cases.

grayhatter 3y ago
but the popular models plastered across HN are in fact, dumb in a lot of cases. Everyone is impressed when the model does something they don't understand deeply. But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply. I do think it's slightly better than the mean across all topics. But I also strongly suspect it'll soon serve as a great example for regressing to the mean.
- jstx1 3y ago
  > But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply. This statement sounds out of date. And you can see this sentiment a lot on HN. I don’t know if the people who say this haven’t tried GPT-4 or they have and are just stubbornly refusing to change their mind about something when presented with new evidence.
  - grayhatter 3y ago
    > s haven’t tried GPT-4 or they have and are just stubbornly refusing to change their mind about something when presented with new evidence. Have you considered that the evidence just isn't convincing yet? If you view popular llms as text prediction machines, they are in fact much better than spell check or auto correct from a few years ago. But if you actually ask it to solve the problem with nuance it will not use nuance. That's the part that would impress me. As a recent example if you ask chat GPT how to use ffmpeg to slice out a video. and you tell it that you only want 3 seconds of video. somebody who deeply understands how ffmpeg works, (or even someone who deeply read the documentation) would point out that you have to be aware that it can only cut to keyframes. so if the keyframe is not aligned to the time you ask for you will not get the video that you expect. another example ask it to play 20 questions with you, it will cheat at the end. even if you give it very specific instructions it still is unable to follow them to the fair conclusion of the game. (or it will make a mistake and understanding about some of the semantics of the question, but I don't fault it for a difference in context) I'd caution you that just because you're impressed for the subjects that you understand deeply does not mean that chat gpt is good at all subjects. and therefore I assert that it is disrespectful to be so dismissive of people who have different opinions than yours. I believe the default should be to assume good faith rather than dismissiveness "they just don't understand"
    - jstx1 3y ago
      ChatGPT with GPT-4 with just "how to use ffmpeg to slice out a video" pasted directly from your comment without any additional prompting or clarification brought up your point about the keyframes. So whatever you think you're criticising isn't the same thing that I'm using. Kind of demonstrates my point that you're either using an older version or choosing to ignore evidence for some reason.
      - grayhatter 3y ago
        It doesn't impress me that once corrected (which it has been multiple times), that it stops giving wrong answers. If it was able to detect and supply nuance to something novel. Then I might be impressed. BTW: when I posed this question > how do I use ffmpeg to slice out a video, I only want 3 seconds of video from the middle to chatgpt4, it did not mention anything about accounting for keyframes. So I'm not sure what version you're using, but it's not the one I have access to :/ Another reason to assume good faith, because you seem to have access to something that I don't. And, this is now (well before this specific conversation) an expired example because openai's model has been specifically training on this nuance, and thus even if it never made a mistake around it, it still wouldn't be impressive to me; simply because this nuance was directly added to it's model. Remember, we're talking about if the abilities of available models are impressive, what would be impressive to me is if they're able to generalize well enough to know things they haven't been directly taught.