4 ms·

What's your definition of competence boundary for human coworker?

by mirekrusin 1mo ago

Sharlin 1mo ago
I'm not in the business of defining AGI, but "really good" obviously isn't that.
- mirekrusin 1mo ago
  You're not answering the question. When somebody says that colleague is "really good", it's a good judgement signal for me.
  - Sharlin 1mo ago
    I may miss some context because the GP’s link has a paywall. But Altman said, "Let’s say we make an AI that’s really good". What is that supposed to mean? Really good relative to what? Current models are "really good" in many ways but nowhere near AGI. Really good compared to an average human? At everything? We’re talking about AGI, so "it’s a really good programmer/coworker/whatever" is a necessary but nowhere near sufficient condition, obviously. But given the constraints of LLMs, we can cut them some slack and only demand they be human-equivalent at digital tasks rather than walking and cooking. Still, being "really good" at all that seems to me really difficult to measure. But it’s a sufficient but not necessary condition anyway, AGI just means equivalent to human, not equivalent to a really smart human. So I really do wonder what Altman meant there if anything.
    - mirekrusin 1mo ago
      "at everything"? Surely you know a friend or two who is not good at almost anything – but you wouldn't hesitate to say that he possesses general intelligence.
  - edukite 1mo ago
    Learn, adapt and is capable of admitting mistakes & fixing it without addition input. Something AI is not capable. Even today Claude was not able to dig deep and try other ways to do what it suppose to do for me. It was constantly "I gave up"
    - mirekrusin 1mo ago
      Most short-term learning/adaptation is already handled in-context. Modern context windows can hold several books worth of text - plenty for most tasks. Everybody is already using it to adapt models to their projects through skills/instructions/guides etc. ps. I often say that after glossary-skill next must have one is update-skill-skill that threats all .md files as live documents. Persistent weight adaptation also happens just not in real time - sessions are captured, analyzed, transformed into training data, fed into SFT/RL environments and later contribute to model updates. Takes a bit of time for the whole loop but you can't say it's not present. There's nothing fundamentally preventing real-time weight updates, ie. LoRA-style online adaptation would be one obvious approach. It's just generally not worth doing at scale. Updating a shared model centrally gives much better data efficiency, batching, evaluation, control etc. than continuously training a separate set of weights for every user/session. There is some work happening on narrowing that gap, for example Mistral has been pushing efficient LoRA-based customization, continuous pretraining, model adaptation etc. I also did play a bit with activation steering – it's super cool where you extract profile for some concepts (emotional in my case) and you have effectively toggles to control "brightness/contrast" those areas (enhancing or suppressing those activation regions from profile) injecting to the model those concepts (emotions in my case) – you can do it in real time and it's fun thing to play with.
    - badestrand 29d ago
      My Claude admits mistakes and then fixes them on its own all the time. Often even without my input: "(thinking..) Oh I discovered that I misjudged XYX, let me fix that.. (thinking) (executing scripts) Okay I corrected my mistake, I had accidentily ABC."
  - dimitrios1 1mo ago
    Here's my definition of AGI Would I trust to let an AI, with zero human input or oversight, to diagnose, come up with treatment plan, and ultimately operate on my l5/s1 disc that's been bugging me for the better part of my adult life? Would I take a novel drug "discovered" by AI (I mean entirely by AI, no human input, remember we are talking AGI) that promises to cure some chronic neurological disorder? In both of those cases, they are the biggest hell-no's I can emphatically say. Until I can say hell yes to that question, we aren't close. Preempting those who say "Well your doctor/drug companies are probably mostly using/going to be using AI to do that" -- not what we are talking about here, and in both cases, not AGI (and I would probably find a new doctor)
    - mirekrusin 1mo ago
      Sounds like category error to me, you wouldn't trust teen or Einstein to do it either, right?
      - dimitrios1 1mo ago
        The G in AGI stands for general, correct?
        Sharlin 29d ago
        Yes, but it means general the way humans are general. Clearly being a general intelligence shouldn’t require being any better at any individual task than the average human, or even the bottom decile of humans.
        dimitrios1 26d ago
        What is the way humans are general? How general is the average human? What constitutes average here? The goalpost moving is getting exhausting.
    - Sharlin 29d ago
      Humans are general intelligences. An average human can’t do either of those. Your bar is way too high.

What's your definition of competence boundary for human coworker?

Sharlin 1mo ago

I'm not in the business of defining AGI, but "really good" obviously isn't that.

mirekrusin 1mo ago
You're not answering the question. When somebody says that colleague is "really good", it's a good judgement signal for me.
- Sharlin 1mo ago
  I may miss some context because the GP’s link has a paywall. But Altman said, "Let’s say we make an AI that’s really good". What is that supposed to mean? Really good relative to what? Current models are "really good" in many ways but nowhere near AGI. Really good compared to an average human? At everything? We’re talking about AGI, so "it’s a really good programmer/coworker/whatever" is a necessary but nowhere near sufficient condition, obviously. But given the constraints of LLMs, we can cut them some slack and only demand they be human-equivalent at digital tasks rather than walking and cooking. Still, being "really good" at all that seems to me really difficult to measure. But it’s a sufficient but not necessary condition anyway, AGI just means equivalent to human, not equivalent to a really smart human. So I really do wonder what Altman meant there if anything.
  - mirekrusin 1mo ago
    "at everything"? Surely you know a friend or two who is not good at almost anything – but you wouldn't hesitate to say that he possesses general intelligence.
- edukite 1mo ago
  Learn, adapt and is capable of admitting mistakes & fixing it without addition input. Something AI is not capable. Even today Claude was not able to dig deep and try other ways to do what it suppose to do for me. It was constantly "I gave up"
  - mirekrusin 1mo ago
    Most short-term learning/adaptation is already handled in-context. Modern context windows can hold several books worth of text - plenty for most tasks. Everybody is already using it to adapt models to their projects through skills/instructions/guides etc. ps. I often say that after glossary-skill next must have one is update-skill-skill that threats all .md files as live documents. Persistent weight adaptation also happens just not in real time - sessions are captured, analyzed, transformed into training data, fed into SFT/RL environments and later contribute to model updates. Takes a bit of time for the whole loop but you can't say it's not present. There's nothing fundamentally preventing real-time weight updates, ie. LoRA-style online adaptation would be one obvious approach. It's just generally not worth doing at scale. Updating a shared model centrally gives much better data efficiency, batching, evaluation, control etc. than continuously training a separate set of weights for every user/session. There is some work happening on narrowing that gap, for example Mistral has been pushing efficient LoRA-based customization, continuous pretraining, model adaptation etc. I also did play a bit with activation steering – it's super cool where you extract profile for some concepts (emotional in my case) and you have effectively toggles to control "brightness/contrast" those areas (enhancing or suppressing those activation regions from profile) injecting to the model those concepts (emotions in my case) – you can do it in real time and it's fun thing to play with.
  - badestrand 29d ago
    My Claude admits mistakes and then fixes them on its own all the time. Often even without my input: "(thinking..) Oh I discovered that I misjudged XYX, let me fix that.. (thinking) (executing scripts) Okay I corrected my mistake, I had accidentily ABC."
- dimitrios1 1mo ago
  Here's my definition of AGI Would I trust to let an AI, with zero human input or oversight, to diagnose, come up with treatment plan, and ultimately operate on my l5/s1 disc that's been bugging me for the better part of my adult life? Would I take a novel drug "discovered" by AI (I mean entirely by AI, no human input, remember we are talking AGI) that promises to cure some chronic neurological disorder? In both of those cases, they are the biggest hell-no's I can emphatically say. Until I can say hell yes to that question, we aren't close. Preempting those who say "Well your doctor/drug companies are probably mostly using/going to be using AI to do that" -- not what we are talking about here, and in both cases, not AGI (and I would probably find a new doctor)
  - mirekrusin 1mo ago
    Sounds like category error to me, you wouldn't trust teen or Einstein to do it either, right?
    - dimitrios1 1mo ago
      The G in AGI stands for general, correct?
      - Sharlin 29d ago
        Yes, but it means general the way humans are general. Clearly being a general intelligence shouldn’t require being any better at any individual task than the average human, or even the bottom decile of humans.
        dimitrios1 26d ago
        What is the way humans are general? How general is the average human? What constitutes average here? The goalpost moving is getting exhausting.
  - Sharlin 29d ago
    Humans are general intelligences. An average human can’t do either of those. Your bar is way too high.