6 ms·

How to think about the OpenAI Q* rumors

54 points by breckenedge 3y ago

swyx 3y ago
the sheer volume of content creator speculation based on a "leak" of a single letter of the alphabet from openai connected to "clues" all self referentially feeding themselves into a media frenzy based on absolutely no other confirmatory information from openai... could provide enough training data for GPT5. as a writer this scares me bc of how un-self-aware every q* anon seems to be right now. how many times have i myself been in this position?
- Nthringas 3y ago
  ah, so your comment leads me to conclude that this is part of some hype marketing campaign to get high-level investors to throw money at openAI ...which faintly smells rotten because of them "lack of a moat" arguments
  - swyx 3y ago
    if you genuinely think either of your sentences is an accurate world model, i think you are miscalibrated.
    - Nthringas 3y ago
      I'm most certainly calibrated differntly than you what I actually think is way crazier than that. if you asked me (which you didn't) I'd say that I think that behind the scenes what all these llm and ai companies are doing is licensing chinese tech i regard the timing for the openai fiasco right after the apec meeting in sf as too good for this all to be true
      - stevenhuang 3y ago
        That sounds even more miscalibrated
andsoitis 3y ago
> How to think about the OpenAI Q* rumors Are rumors worth thinking about?
- andrewflnr 3y ago
  Rumors are weak evidence. Weak evidence is still evidence.
  - nitwit005 3y ago
    I hear andreflnr is a giraffe.
    - andrewflnr 3y ago
      You don't actually, is the thing. No humans exist who think that. If they did, that would imply something, wouldn't it, if only that there are some really weird people obsessed with me?
      - nitwit005 3y ago
        Yes, the person lying to create a rumor doesn't think it's true, but the people who believe the lie do. They treat it as credible, just as you suggested. Who wouldn't be interested in a giraffe that can type? You deserve your fame.
        andrewflnr 3y ago
        You're being obtuse. Not all rumors are deliberate lies. Those that are are evidence of something else. Remember that the only question was "are rumors worth thinking about", which includes meta-level concerns.
  - ranguna 3y ago
    Not sure how you jumped from "rumor" to "evidence". Prefixing it with "weak" does nothing when you are trying to link "evidence" with "rumor". A rumor is a rumor. If you want to talk about it, have fun, you might be lucky and it's true, but more often than not when marketing is involved, it's far from being the truth.
aaron695 3y ago
[dead]
tracerbulletx 3y ago
I think if you've seen this section of https://youtu.be/zjkBMFhNj_g?si=GPUxt1eHbQJ8HFj-&t=2113 https://youtu.be/zjkBMFhNj_g?si=GPUxt1eHbQJ8HFj-&t=2113 this talk you'll probably generally agree that Q* is something like what's described in this article.
- benzible 3y ago
  The video hints at why achieving artificial general intelligence (AGI) is so difficult. As an example, it points out that AlphaGo was able to surpass human Go players by playing against itself, which worked because winning Go games provides a clear reward function: winning the game. Similarly, the rumored Q* breakthrough involves solving simple math problems, another closed domain with a well-defined reward function. This has likely gotten people overly excited about the prospects for rapid advances in domains like open-ended text generation. Defining, much less evaluating, a reward function that maps to the "quality" of generated text in general for use during the inference phase is hard to imagine. So while grafting planning onto a language model works for constrained use cases like math, it is hard to see how it could work for generating high quality, coherent text in general.
say_it_as_it_is 3y ago
Why is this blog called Understanding AI when every single post is an opinion piece about anything but AI?
- binarybits 3y ago
  Author here. What do you mean? The article this thread is about has hardly any opinion in it. Or consider this piece explaining how LLMs work: https://www.understandingai.org/p/large-language-models-explained-with https://www.understandingai.org/p/large-language-models-expl...
  - say_it_as_it_is 3y ago
    the entire front page of blog posts as of the present discussion is all speculative opinion pieces the post you shared is from July, 5 months ago, and does seem to be more on-topic
QuantumGood 3y ago
The density of advances considered "breakthroughs" (whether they are or not) is high right now, and whatever Q may or may not be, it is very likely to be eclipsed shortly, and almost certain to be eclipsed before long, and is otherwise just being used for clickbait, so I find it easy to ignore

54 points by breckenedge 3y ago

swyx 3y ago

the sheer volume of content creator speculation based on a "leak" of a single letter of the alphabet from openai connected to "clues" all self referentially feeding themselves into a media frenzy based on absolutely no other confirmatory information from openai... could provide enough training data for GPT5. as a writer this scares me bc of how un-self-aware every q* anon seems to be right now. how many times have i myself been in this position?

Nthringas 3y ago
ah, so your comment leads me to conclude that this is part of some hype marketing campaign to get high-level investors to throw money at openAI ...which faintly smells rotten because of them "lack of a moat" arguments
- swyx 3y ago
  if you genuinely think either of your sentences is an accurate world model, i think you are miscalibrated.
  - Nthringas 3y ago
    I'm most certainly calibrated differntly than you what I actually think is way crazier than that. if you asked me (which you didn't) I'd say that I think that behind the scenes what all these llm and ai companies are doing is licensing chinese tech i regard the timing for the openai fiasco right after the apec meeting in sf as too good for this all to be true
    - stevenhuang 3y ago
      That sounds even more miscalibrated

andsoitis 3y ago

> How to think about the OpenAI Q* rumors Are rumors worth thinking about?

andrewflnr 3y ago
Rumors are weak evidence. Weak evidence is still evidence.
- nitwit005 3y ago
  I hear andreflnr is a giraffe.
  - andrewflnr 3y ago
    You don't actually, is the thing. No humans exist who think that. If they did, that would imply something, wouldn't it, if only that there are some really weird people obsessed with me?
    - nitwit005 3y ago
      Yes, the person lying to create a rumor doesn't think it's true, but the people who believe the lie do. They treat it as credible, just as you suggested. Who wouldn't be interested in a giraffe that can type? You deserve your fame.
      - andrewflnr 3y ago
        You're being obtuse. Not all rumors are deliberate lies. Those that are are evidence of something else. Remember that the only question was "are rumors worth thinking about", which includes meta-level concerns.
- ranguna 3y ago
  Not sure how you jumped from "rumor" to "evidence". Prefixing it with "weak" does nothing when you are trying to link "evidence" with "rumor". A rumor is a rumor. If you want to talk about it, have fun, you might be lucky and it's true, but more often than not when marketing is involved, it's far from being the truth.

aaron695 3y ago

[dead]

tracerbulletx 3y ago

I think if you've seen this section of https://youtu.be/zjkBMFhNj_g?si=GPUxt1eHbQJ8HFj-&t=2113 https://youtu.be/zjkBMFhNj_g?si=GPUxt1eHbQJ8HFj-&t=2113 this talk you'll probably generally agree that Q* is something like what's described in this article.

benzible 3y ago
The video hints at why achieving artificial general intelligence (AGI) is so difficult. As an example, it points out that AlphaGo was able to surpass human Go players by playing against itself, which worked because winning Go games provides a clear reward function: winning the game. Similarly, the rumored Q* breakthrough involves solving simple math problems, another closed domain with a well-defined reward function. This has likely gotten people overly excited about the prospects for rapid advances in domains like open-ended text generation. Defining, much less evaluating, a reward function that maps to the "quality" of generated text in general for use during the inference phase is hard to imagine. So while grafting planning onto a language model works for constrained use cases like math, it is hard to see how it could work for generating high quality, coherent text in general.

say_it_as_it_is 3y ago

Why is this blog called Understanding AI when every single post is an opinion piece about anything but AI?

binarybits 3y ago
Author here. What do you mean? The article this thread is about has hardly any opinion in it. Or consider this piece explaining how LLMs work: https://www.understandingai.org/p/large-language-models-explained-with https://www.understandingai.org/p/large-language-models-expl...
- say_it_as_it_is 3y ago
  the entire front page of blog posts as of the present discussion is all speculative opinion pieces the post you shared is from July, 5 months ago, and does seem to be more on-topic

QuantumGood 3y ago

The density of advances considered "breakthroughs" (whether they are or not) is high right now, and whatever Q may or may not be, it is very likely to be eclipsed shortly, and almost certain to be eclipsed before long, and is otherwise just being used for clickbait, so I find it easy to ignore