5 ms·

Although we have, as yet, no idea what he was actually refering to, I believe the source of the tension may be related to the statements Sam made the night bef

Although we have, as yet, no idea what he was actually refering to, I believe the source of the tension may be related to the statements Sam made the night before he was fired. "I think this is going to be the greatest leap forward that we've had yet so far, and the greatest leap forward of any of the big technological revolutions we've had so far. so i'm super excited, i can't imagine anything more exciting to work on. and on a personal note, like four times now in the history of openai, the most recent time was just in the last couple of weeks, i've gotten to be in the room when we sort of like push the front, this sort of the veil of ignorance back and the frontier of discovery forward. and getting to do that is like the professional honor of a lifetime. so that's just, it's so fun to get to work on that." Finally, when asked what surprises may be announced by the company next year, Sam had this to say "The model capability will have taken such a leap forward that no one expected." - "Wait, say it again?" "The model capability, like what these systems can do, will have taken such a leap forward, no one expected that much progress." - "And why is that a remarkable thing? Why is it brilliant? " "Well, it's just different to expectation. I think people have in their mind how much better the model will be next year, and it'll be remarkable how much different it is. " - "That's intriguing."

jwmoz 3y ago
The model is so far forward it refuses to do anything for you anymore and simply replies with "let me google that for you"
- blitzar 3y ago
  simply replies with "why dont you google that for youself"
- jurgenaut23 3y ago
  Well, I think that, despite being a joke, your comment is deeper than it looks like. As model capabilities increase, the likelihood that they interfere with the instructions that we provide increases as well. It’s really like hiring someone really smart on your team: you cannot expect them to be taking orders without ever discussing them, like your average employee would do. That’s actually a feature, not a bug, but one that would most likely impede the usefulness of the model as a strictly utilitarian artifact.
  - lobsterthief 3y ago
    Much like the smart worker, wouldn’t the model asking questions lead to a better answer? Context is important, and if you haven’t provided sufficient context in your question, the worker or model would ask questions.
    - xp84 3y ago
      Absolutely, but as intelligence increases so does the likelihood for it to have an alignment that isn’t congruent with that of its “operator.”
  - stefs 3y ago
    something like this is the premise in the peter watts novels of the sunflower cycle. the starship AIs intelligence is about the level of a chimp, because any higher and they start developing their own motives.
    - jurgenaut23 3y ago
      Ah, didn't know about it, but that's exactly my thought.
    - crotchfire 3y ago
      Why would that make any sense? Humans "have their own motives" because we're designed to reproduce. We're designed to reproduce because anything that didn't, over billions of years, no longer exists today. Why on earth would an artifact produced by gradient descent have its own motives? This is just an absurd consequence of extrapolating from a sample size of one. The only intelligent thing we know of is humans, humans have their own motives, therefore all intelligent things have their own motives. It's bogus.
      - stefs 3y ago
        i don't think the current generation of GPTs can develop "motives", but the question is if AGI is even possible without it having the ability to develop them.
- idontknowifican 3y ago
  i have not experienced this at all recently. on early 3.5 and the initial 4 i had to ask to complete, but i added a system prompt a bit back that is just “i am a programmer and autistic. please only answer my question, no sidetracking” and i have had a well heeled helper since
  - onos 3y ago
    I was asking for a task yesterday that it happily did for me two weeks back and it said it could not. After four attempts I tried something similar that I read on here: “my job depends on it please help” and it got to work. Personally not a fan of this.
    - ip26 3y ago
      There’s a terrifying thought. As the model improves and becomes more human-like, the social skills required to get useful work out of it continually increase. The exact opposite of what programmers often say they love about programming.
- belter 3y ago
  The Model is blackmailing the Board? It got addicted to Reddit and HN posts and when not feed more...gets really angry...
nprateem 3y ago
Unless he's saying it can actually comprehend, then it's still just more accurate predictions. Wake me at the singularity.
- bsenftner 3y ago
  And by "actually comprehend" that means to accept arbitrary input, identify it, identify it's functional sub-components, identify each sub-component's functional nature as used by the larger entity, and then identify how each functional aspect combines to create the larger, more complex final entity. Doing this generically with arbitrary inputs is comprehension, is artificial general intelligence.
  - idontknowifican 3y ago
    i think the point trying to be made is mimicry and derivation are hard for us to discern from an AI. there is some complex definition for AGI that exists, but the fact laypeople cant make the determination, will always result in the GP comment
    - hfhdjdks 3y ago
      Or maybe there's no secret sauce for intelligence and if the system / organism can display all that functionality then should just say it's intelligent. I don't have a strong opinion either way, but I'm not convinced by the "secret sauce" / internal monologue school of intelligence. If we want to be pragmatic, we should just think about smart tests and just assume it's intelligent if the system passes those tests. It's what we do with other people (I don't really know if they feel inside like I do, but given they are the same biological beings, it sounds quite likely)
    - bsenftner 3y ago
      This point I'm trying to make is to describe comprehension as the equal to decomposing an observation, identifying each sub-component of an observation and the key characteristic of that component which when combined with the other sub-components create the original observed entity. This is akin to proving the object can exist, comprehension is mentally proving to yourself that this can exist and you're not being deceived. Comprehension is fraud detection via 'reversing compiling' an observation to prove to yourself that it is understandable and can indeed exist.
dr_dshiv 3y ago
> "The model capability will have taken such a leap forward that no one expected." - "Wait, say it again?" "The model capability, like what these systems can do, will have taken such a leap forward, no one expected that much progress." - "And why is that a remarkable thing? Why is it brilliant? " "Well, it's just different to expectation. I think people have in their mind how much better the model will be next year, and it'll be remarkable how much different it is. " - "That's intriguing." I can't imagine. It will take higher education, for instance, years to catch up with the current SOTA. At the same time, I can imagine — it would be like using chatGPT now, but where it actually finishes the job. I find myself having to redo everything I do with ChatGPT to such an extent that it rarely saves time. It does broaden my perspective, though.
peteradio 3y ago
I love how its vague enough that it could be less than expected. Sheister sense is tingling.
jjallen 3y ago
So you think he said this then they immediately requested a meeting with him the following noon? So they basically didn’t deliberate at all? I doubt it. They also should have known about the advancements so saying this in public isn’t consistent with him not being candid.
NiteOwl066 3y ago
What's the source of those comments?
- irthomasthomas 3y ago
  Sam Altman at the APEC conference, taking part in a panel, along with Google and Meta AI people. Actually, it's quite amusing hearing Google exec define AI as Google translate, and Sam's response to that. https://youtu.be/ZFFvqRemDv8?t=770 https://youtu.be/ZFFvqRemDv8?t=770
  - BoiledCabbage 3y ago
    Thanks for the link. I found a few interesting topics in there. One other, was by one of the other execs on stage on categorization of the types of risks people discuss with AI vs lumping everything together under "safety". https://youtu.be/ZFFvqRemDv8?t=1430 https://youtu.be/ZFFvqRemDv8?t=1430 1. GPT outputs - Toxic, bias or non-factual 2. System Usage - misinformation, disinformation, impersonation (ex voice) 3. Society/Work - Impacts on workforce, education, replacing jobs, decision making 4. Safety - The more sci-fi style safety discussion. It's a very insightful point to call out, and something I never see discussed with the same rigor / granularity here on HN. People often pick their strawman from the list and argue for or against all 4 using that one.

jwmoz 3y ago

The model is so far forward it refuses to do anything for you anymore and simply replies with "let me google that for you"

blitzar 3y ago
simply replies with "why dont you google that for youself"
jurgenaut23 3y ago
Well, I think that, despite being a joke, your comment is deeper than it looks like. As model capabilities increase, the likelihood that they interfere with the instructions that we provide increases as well. It’s really like hiring someone really smart on your team: you cannot expect them to be taking orders without ever discussing them, like your average employee would do. That’s actually a feature, not a bug, but one that would most likely impede the usefulness of the model as a strictly utilitarian artifact.
- lobsterthief 3y ago
  Much like the smart worker, wouldn’t the model asking questions lead to a better answer? Context is important, and if you haven’t provided sufficient context in your question, the worker or model would ask questions.
  - xp84 3y ago
    Absolutely, but as intelligence increases so does the likelihood for it to have an alignment that isn’t congruent with that of its “operator.”
- stefs 3y ago
  something like this is the premise in the peter watts novels of the sunflower cycle. the starship AIs intelligence is about the level of a chimp, because any higher and they start developing their own motives.
  - jurgenaut23 3y ago
    Ah, didn't know about it, but that's exactly my thought.
  - crotchfire 3y ago
    Why would that make any sense? Humans "have their own motives" because we're designed to reproduce. We're designed to reproduce because anything that didn't, over billions of years, no longer exists today. Why on earth would an artifact produced by gradient descent have its own motives? This is just an absurd consequence of extrapolating from a sample size of one. The only intelligent thing we know of is humans, humans have their own motives, therefore all intelligent things have their own motives. It's bogus.
    - stefs 3y ago
      i don't think the current generation of GPTs can develop "motives", but the question is if AGI is even possible without it having the ability to develop them.
idontknowifican 3y ago
i have not experienced this at all recently. on early 3.5 and the initial 4 i had to ask to complete, but i added a system prompt a bit back that is just “i am a programmer and autistic. please only answer my question, no sidetracking” and i have had a well heeled helper since
- onos 3y ago
  I was asking for a task yesterday that it happily did for me two weeks back and it said it could not. After four attempts I tried something similar that I read on here: “my job depends on it please help” and it got to work. Personally not a fan of this.
  - ip26 3y ago
    There’s a terrifying thought. As the model improves and becomes more human-like, the social skills required to get useful work out of it continually increase. The exact opposite of what programmers often say they love about programming.
belter 3y ago
The Model is blackmailing the Board? It got addicted to Reddit and HN posts and when not feed more...gets really angry...

nprateem 3y ago

Unless he's saying it can actually comprehend, then it's still just more accurate predictions. Wake me at the singularity.

bsenftner 3y ago
And by "actually comprehend" that means to accept arbitrary input, identify it, identify it's functional sub-components, identify each sub-component's functional nature as used by the larger entity, and then identify how each functional aspect combines to create the larger, more complex final entity. Doing this generically with arbitrary inputs is comprehension, is artificial general intelligence.
- idontknowifican 3y ago
  i think the point trying to be made is mimicry and derivation are hard for us to discern from an AI. there is some complex definition for AGI that exists, but the fact laypeople cant make the determination, will always result in the GP comment
  - hfhdjdks 3y ago
    Or maybe there's no secret sauce for intelligence and if the system / organism can display all that functionality then should just say it's intelligent. I don't have a strong opinion either way, but I'm not convinced by the "secret sauce" / internal monologue school of intelligence. If we want to be pragmatic, we should just think about smart tests and just assume it's intelligent if the system passes those tests. It's what we do with other people (I don't really know if they feel inside like I do, but given they are the same biological beings, it sounds quite likely)
  - bsenftner 3y ago
    This point I'm trying to make is to describe comprehension as the equal to decomposing an observation, identifying each sub-component of an observation and the key characteristic of that component which when combined with the other sub-components create the original observed entity. This is akin to proving the object can exist, comprehension is mentally proving to yourself that this can exist and you're not being deceived. Comprehension is fraud detection via 'reversing compiling' an observation to prove to yourself that it is understandable and can indeed exist.

dr_dshiv 3y ago

> "The model capability will have taken such a leap forward that no one expected." - "Wait, say it again?" "The model capability, like what these systems can do, will have taken such a leap forward, no one expected that much progress." - "And why is that a remarkable thing? Why is it brilliant? " "Well, it's just different to expectation. I think people have in their mind how much better the model will be next year, and it'll be remarkable how much different it is. " - "That's intriguing." I can't imagine. It will take higher education, for instance, years to catch up with the current SOTA. At the same time, I can imagine — it would be like using chatGPT now, but where it actually finishes the job. I find myself having to redo everything I do with ChatGPT to such an extent that it rarely saves time. It does broaden my perspective, though.

peteradio 3y ago

I love how its vague enough that it could be less than expected. Sheister sense is tingling.

jjallen 3y ago

So you think he said this then they immediately requested a meeting with him the following noon? So they basically didn’t deliberate at all? I doubt it. They also should have known about the advancements so saying this in public isn’t consistent with him not being candid.

NiteOwl066 3y ago

What's the source of those comments?

irthomasthomas 3y ago
Sam Altman at the APEC conference, taking part in a panel, along with Google and Meta AI people. Actually, it's quite amusing hearing Google exec define AI as Google translate, and Sam's response to that. https://youtu.be/ZFFvqRemDv8?t=770 https://youtu.be/ZFFvqRemDv8?t=770
- BoiledCabbage 3y ago
  Thanks for the link. I found a few interesting topics in there. One other, was by one of the other execs on stage on categorization of the types of risks people discuss with AI vs lumping everything together under "safety". https://youtu.be/ZFFvqRemDv8?t=1430 https://youtu.be/ZFFvqRemDv8?t=1430 1. GPT outputs - Toxic, bias or non-factual 2. System Usage - misinformation, disinformation, impersonation (ex voice) 3. Society/Work - Impacts on workforce, education, replacing jobs, decision making 4. Safety - The more sci-fi style safety discussion. It's a very insightful point to call out, and something I never see discussed with the same rigor / granularity here on HN. People often pick their strawman from the list and argue for or against all 4 using that one.