5 ms·

Even as an immigrant starving student I would have paid the $20 to run Knuth's questions. But surely there was someone else with access to GPT-4 in Stanford, of

Even as an immigrant starving student I would have paid the $20 to run Knuth's questions. But surely there was someone else with access to GPT-4 in Stanford, of all places?

Filligree 3y ago
The conclusion seemed dismissive of the entire field, while simultaneously being ignorant. (“Temperature, whatever that means.”) I think they believe they can continue to ignore it, but given the huge leaps in performance just from 3.5 to 4 — or 3 to 3.5, for those who’ve tried 3 — I very much doubt that is the case.
- wilg 3y ago
  I think that quote just indicates he doesn't know the details of what that setting means and hasn't looked, which is okay.
  - CamperBob2 3y ago
    And that he doesn't care, which isn't okay. Then again, Knuth is how old -- in his 80s? He's right in that it makes sense to budget his available research time carefully.
    - wilg 3y ago
      I think it's okay for people of any age to not care about GPT's temperature parameter.
      - CamperBob2 3y ago
        Not if you're going to criticize it in public, though. It's okay for us to think less of people who do things like that, even when -- or especially when -- the guilty parties are respected computer scientists. "Science advances one funeral at a time" is a cynical commentary, not an ideal to strive for.
- gtirloni 3y ago
  Yeah, this particular piece got me wondering. -- I myself shall certainly continue to leave such research to others, and to devote my time to developing concepts that are authentic and trustworthy. And I hope you do the same. Best regards, Don PS: Please reply only with respect to binomial coefficients, because I've already spent way too much time on the topic above! The topic is timely, and important enough not to ignore completely, but it's emphatically not for me.
  - bombcar 3y ago
    Knuth knows enough to poke the fate bear AI with a stick for his own curiosity but also knows he doesn’t know enough to really pontificate about it, and doesn’t want to spend his remaining time getting to know enough about it. But he does know his binomials.
  - ekidd 3y ago
    Knuth has only so many productive years left on this planet and he wants to spend them all on TAOCP (and a few other projects). He had given up email by the 90s and he is incredibly disciplined on what he devotes his time to. If you want to get in touch, send him an actual letter. Last I checked, he reads paper mail once a quarter or so. He's also an utter perfectionist in his own work, in a way that few people ever achieve. So he's curious enough to try ChatGPT. But then he's smart enough to walk away.
    - gtirloni 3y ago
      He did pass judgment though by implying GPT is not "authentic and trustworthy". That what got me thinking, not his desire to focus on other things (it's his time, he can do whatever he wants).
      - grugagag 3y ago
        I think he nailed it on that, chatGPT is not trustworthy. Not sure about authentic, halucinations could be authentic ideas in their own right
      - 3y ago
- doetoe 3y ago
  He didn't say "temperature, whatever that means", he said "temperature 0.7, whatever that means". Do you know what it means? In the API reference it only says the value you can specify for temperature is between 0 and 2, higher values for more random output
  - fauxpause_ 3y ago
    Temperature is a measure of capriciousness. How likely is the model to choose a token that is not “the most likely” next token. It’s not a big ask to look this up. But even if you don’t, making a point to show that you don’t know it seems bad.
    - dragonwriter 3y ago
      > Temperature is a measure of capriciousness. Yes, that’s what “temperature” means, what does a temperature of 0.7 mean? > It’s not a big ask to look this up. But even if you don’t, making a point to show that you don’t know it seems bad. Well, no, making a point of highlighting the points of your ignorance when discussing something is good. Especially when you are a notable expert in the broad field being discussed.
      - fauxpause_ 3y ago
        https://lukesalamone.github.io/posts/what-is-temperature/ https://lukesalamone.github.io/posts/what-is-temperature/ > Well, no, making a point of highlighting the points of your ignorance when discussing something is good. Especially when you are a notable expert in the broad field being discussed. I disagree. Stating “whatever that means” indicates dismissiveness, not a transparent lack of expertise. Also, you should know what it means if you’re an expert. This quote implies to me that he is actually a beginner when it comes to this technology but is expecting to be treated like an expert whose experience generalizes
        copperx 3y ago
        Absolutely disagree. I don't think anyone, except someone with access to the source code, knows exactly what temperature 0.7 means. Knuth is a world expert in randomized algorithms. Do you think he doesn't have a good intuition for what could be happening? But he's a stickler for detail, and temperature is an obfuscation.

Even as an immigrant starving student I would have paid the $20 to run Knuth's questions. But surely there was someone else with access to GPT-4 in Stanford, of all places?

Filligree 3y ago

The conclusion seemed dismissive of the entire field, while simultaneously being ignorant. (“Temperature, whatever that means.”) I think they believe they can continue to ignore it, but given the huge leaps in performance just from 3.5 to 4 — or 3 to 3.5, for those who’ve tried 3 — I very much doubt that is the case.

wilg 3y ago
I think that quote just indicates he doesn't know the details of what that setting means and hasn't looked, which is okay.
- CamperBob2 3y ago
  And that he doesn't care, which isn't okay. Then again, Knuth is how old -- in his 80s? He's right in that it makes sense to budget his available research time carefully.
  - wilg 3y ago
    I think it's okay for people of any age to not care about GPT's temperature parameter.
    - CamperBob2 3y ago
      Not if you're going to criticize it in public, though. It's okay for us to think less of people who do things like that, even when -- or especially when -- the guilty parties are respected computer scientists. "Science advances one funeral at a time" is a cynical commentary, not an ideal to strive for.
gtirloni 3y ago
Yeah, this particular piece got me wondering. -- I myself shall certainly continue to leave such research to others, and to devote my time to developing concepts that are authentic and trustworthy. And I hope you do the same. Best regards, Don PS: Please reply only with respect to binomial coefficients, because I've already spent way too much time on the topic above! The topic is timely, and important enough not to ignore completely, but it's emphatically not for me.
- bombcar 3y ago
  Knuth knows enough to poke the fate bear AI with a stick for his own curiosity but also knows he doesn’t know enough to really pontificate about it, and doesn’t want to spend his remaining time getting to know enough about it. But he does know his binomials.
- ekidd 3y ago
  Knuth has only so many productive years left on this planet and he wants to spend them all on TAOCP (and a few other projects). He had given up email by the 90s and he is incredibly disciplined on what he devotes his time to. If you want to get in touch, send him an actual letter. Last I checked, he reads paper mail once a quarter or so. He's also an utter perfectionist in his own work, in a way that few people ever achieve. So he's curious enough to try ChatGPT. But then he's smart enough to walk away.
  - gtirloni 3y ago
    He did pass judgment though by implying GPT is not "authentic and trustworthy". That what got me thinking, not his desire to focus on other things (it's his time, he can do whatever he wants).
    - grugagag 3y ago
      I think he nailed it on that, chatGPT is not trustworthy. Not sure about authentic, halucinations could be authentic ideas in their own right
    - 3y ago
doetoe 3y ago
He didn't say "temperature, whatever that means", he said "temperature 0.7, whatever that means". Do you know what it means? In the API reference it only says the value you can specify for temperature is between 0 and 2, higher values for more random output
- fauxpause_ 3y ago
  Temperature is a measure of capriciousness. How likely is the model to choose a token that is not “the most likely” next token. It’s not a big ask to look this up. But even if you don’t, making a point to show that you don’t know it seems bad.
  - dragonwriter 3y ago
    > Temperature is a measure of capriciousness. Yes, that’s what “temperature” means, what does a temperature of 0.7 mean? > It’s not a big ask to look this up. But even if you don’t, making a point to show that you don’t know it seems bad. Well, no, making a point of highlighting the points of your ignorance when discussing something is good. Especially when you are a notable expert in the broad field being discussed.
    - fauxpause_ 3y ago
      https://lukesalamone.github.io/posts/what-is-temperature/ https://lukesalamone.github.io/posts/what-is-temperature/ > Well, no, making a point of highlighting the points of your ignorance when discussing something is good. Especially when you are a notable expert in the broad field being discussed. I disagree. Stating “whatever that means” indicates dismissiveness, not a transparent lack of expertise. Also, you should know what it means if you’re an expert. This quote implies to me that he is actually a beginner when it comes to this technology but is expecting to be treated like an expert whose experience generalizes
      - copperx 3y ago
        Absolutely disagree. I don't think anyone, except someone with access to the source code, knows exactly what temperature 0.7 means. Knuth is a world expert in randomized algorithms. Do you think he doesn't have a good intuition for what could be happening? But he's a stickler for detail, and temperature is an obfuscation.