4 ms·

There was a meme going around that said the fall of Rome was an unannounced anticlimactic event where one day someone went out and the bridge wasn't ever repair

by 9x39 8mo ago

There was a meme going around that said the fall of Rome was an unannounced anticlimactic event where one day someone went out and the bridge wasn't ever repaired. Maybe AGI's arrival is when one day someone is given an AI to supervise instead of a new employee. Just a user who's followed the whole mess, not a researcher. I wonder if the scaffolding and bolt-ons like reasoning will sufficiently be an asymptote to 'true AGI'. I kept reading about the limits of transformers around GPT-4 and Opus 3 time, and then those seem basic compared to today. I gave up trying to guess when the diminishing returns will truly hit, if ever, but I do think some threshold has been passed where the frontier models are doing "white collar work as an API" and basic reasoning better than the humans in many cases, and once capital familiarizes themselves with this idea more, it's going to get interesting.

deleted 8mo ago
[deleted]
esafak 8mo ago
But it's already like that; models are better than many workers, and I'm supervising agents. I'd rather have the model than numerous juniors; esp. the kind that can't identify the model's mistakes.
- causal 8mo ago
  This is my greatest cause for alarm regarding LLM adoption. I am not yet sure AI will ever be good enough to use without experts watching them carefully; but they are certainly good enough that non-experts cannot tell the difference.
  - al_borland 8mo ago
    My dad is retired and enamored with ChatGPT. He’s been teaching classes to seniors and evangelizing the use to all his friends. Every time he calls he gives me an update on who he’s converted into a ChatGPT user. He seems disappointed with anyone who doesn’t use it for everything after he tells them about it. A couple days ago he was telling me one lady he was trying to sell on it wouldn’t use it. She took the position that if she can’t trust the answers all the time, she isn’t going to trust or use it for anything. My dad almost seemed offended by this idea, he couldn’t understand why someone wouldn’t want the benefits it could offer, even if it wasn’t perfect. I think her position was very sound. We see how much misinformation spreads online and how vulnerable people are to it. Wanting a trusted source of information is not a bad thing. Getting information more quickly is of little value if it isn’t reliable data. If I prod my dad enough about it, he will admit that ChatGPT has made some mistakes that he caught. He knew enough to question it more when it was wrong. The problem is, if he already knew the answer, why was he asking in the first place… and if it was something he wasn’t well versed on, how does he know it’s giving him good data? People are defaulting to trust, unless they catch the LLM in a lie. How many times does someone have to lie to a person before they are labeled a liar and no longer trusted at face value? For me, these LLMs have been labeled a liar and I don’t trust them. Trust takes a long time to rebuild once it’s broken. I mostly use LLMs to augment search, not replace it. If it gives me an answer, I’ll click through to the sourced reference and see what it says there, and evaluate if it’s a source with trusting. In many cases the LLM will get me to the right page, but it will jumble up the details and get them wrong, like a bad game of telephone.
    - causal 8mo ago
      Thanks for sharing that anecdote. I think everyone is susceptible to misinformation, and seniors might be especially unprepared to adapt to LLMs tricks.
    - entech 8mo ago
      How do you know that it’s a source worth trusting? I think the expectation of AI being perfect all the time is probably driven by the hype and marketing of “1 million PhDs in your pocket”. If you compare AI to an average person or a random website you’d come across google I would wager that AI is more likely to be accurate in almost every scenario. Hyper specific areas, niche domains and rapidly evolving data that is not being published - a lot less so.
- greedo 8mo ago
  The problem becomes your retirement. Sure, you've earned "expert" status, but all the junior developers won't be hired, so they'll never learn from junior mistakes. They'll blindly trust agents and not know deeper techniques.
  - charcircuit 8mo ago
    You can get experience without an actual job.
    - igor47 8mo ago
      Can I rephrase this as "you can get experience without any experience"? Certainly, there's stuff you can learn that's adjacent to doing the thing; that's what happens when juniors graduate with CS degrees. But the lack of doing the thing is what makes them juniors.
      - charcircuit 8mo ago
        >that's what happens when juniors graduate with CS degrees A CS degree is going to give you much less experience than building projects and businesses yourself.
        al_borland 8mo ago
        How much time will someone realistically dedicate to this if they need to have a separate day job? How good will they get without mentors? How much complexity will they really need to manage without the bureaucracy of an organization? Are senior software engineers of the future going to be waiting tables along side actors for the first 10+ years of their adult life, working on side projects on nights and weekends, hoping to one day jump straight to a senior position in a large company? The questions I instinctively ask myself when looking at a new problem, having worked in an enterprise environment for 20 years, are much different than what I’d be asking having just worked on personal projects. Most of the technology I’ve had access to isn’t something a solo hobbyist dev will ever touch. Most of the questions I’m asking are influenced by having that access, along with some of the personalities I’ve had to deal with. How will people get that kind of experience? There is also the big issue of people not knowing what to build. When a person gets a job, they no long need to come up with their own ideas. Or they generate ideas based on the needs of the environment they’re in. In the context of my job, I have no shortage of ideas. For solo projects, I often draw a blank. The world doesn’t need a hundred more todo apps.
- bamboozled 8mo ago
  From my experience, if you think AI is better than most workers, you're probably just generating a whole bunch of semi-working garbage, accepting that input as good enough and will likely learn the hardware your software is full of bugs and incorrect logic.
  - bamboozled 8mo ago
    hardware / hard way, auto-correct is a thing of beauty sometimes :)
beej71 8mo ago
I'd always imagined that AGI meant an AI was given other AIs to manage.
- davnicwil 8mo ago
  I don't think this is how it'll play out, and I'm generally a bit skeptical of the 'agent' paradigm per se. There doesn't seem to be a reason why AIs should act as these distinct entities that manage each other or form teams or whatever. It seems to me way more likely that everything will just be done internally in one monolithic model. The AIs just don't have the constraints that humans have in terms of time management, priority management, social order, all the rest of it that makes teams of individuals the only workable system. AI simply scales with the compute resources made available, so it seems like you'd just size those resources appropriately for a problem, maybe even on demand, and have a singluar AI entity (if it's even meaningful to think of it as such, even that's kind of an anthropomorphisation) just do the thing. No real need for any organisational structure beyond that. So I'd think maybe the opposite, seems like what agents really means is a way to use fundamentally narrow/limited AI inside our existing human organisations and workflows, directed by humans. Maybe AGI is when all that goes away because it's just obviously not necessary any more.

by 9x39 8mo ago

deleted 8mo ago

[deleted]

esafak 8mo ago

But it's already like that; models are better than many workers, and I'm supervising agents. I'd rather have the model than numerous juniors; esp. the kind that can't identify the model's mistakes.

causal 8mo ago
This is my greatest cause for alarm regarding LLM adoption. I am not yet sure AI will ever be good enough to use without experts watching them carefully; but they are certainly good enough that non-experts cannot tell the difference.
- al_borland 8mo ago
  My dad is retired and enamored with ChatGPT. He’s been teaching classes to seniors and evangelizing the use to all his friends. Every time he calls he gives me an update on who he’s converted into a ChatGPT user. He seems disappointed with anyone who doesn’t use it for everything after he tells them about it. A couple days ago he was telling me one lady he was trying to sell on it wouldn’t use it. She took the position that if she can’t trust the answers all the time, she isn’t going to trust or use it for anything. My dad almost seemed offended by this idea, he couldn’t understand why someone wouldn’t want the benefits it could offer, even if it wasn’t perfect. I think her position was very sound. We see how much misinformation spreads online and how vulnerable people are to it. Wanting a trusted source of information is not a bad thing. Getting information more quickly is of little value if it isn’t reliable data. If I prod my dad enough about it, he will admit that ChatGPT has made some mistakes that he caught. He knew enough to question it more when it was wrong. The problem is, if he already knew the answer, why was he asking in the first place… and if it was something he wasn’t well versed on, how does he know it’s giving him good data? People are defaulting to trust, unless they catch the LLM in a lie. How many times does someone have to lie to a person before they are labeled a liar and no longer trusted at face value? For me, these LLMs have been labeled a liar and I don’t trust them. Trust takes a long time to rebuild once it’s broken. I mostly use LLMs to augment search, not replace it. If it gives me an answer, I’ll click through to the sourced reference and see what it says there, and evaluate if it’s a source with trusting. In many cases the LLM will get me to the right page, but it will jumble up the details and get them wrong, like a bad game of telephone.
  - causal 8mo ago
    Thanks for sharing that anecdote. I think everyone is susceptible to misinformation, and seniors might be especially unprepared to adapt to LLMs tricks.
  - entech 8mo ago
    How do you know that it’s a source worth trusting? I think the expectation of AI being perfect all the time is probably driven by the hype and marketing of “1 million PhDs in your pocket”. If you compare AI to an average person or a random website you’d come across google I would wager that AI is more likely to be accurate in almost every scenario. Hyper specific areas, niche domains and rapidly evolving data that is not being published - a lot less so.
greedo 8mo ago
The problem becomes your retirement. Sure, you've earned "expert" status, but all the junior developers won't be hired, so they'll never learn from junior mistakes. They'll blindly trust agents and not know deeper techniques.
- charcircuit 8mo ago
  You can get experience without an actual job.
  - igor47 8mo ago
    Can I rephrase this as "you can get experience without any experience"? Certainly, there's stuff you can learn that's adjacent to doing the thing; that's what happens when juniors graduate with CS degrees. But the lack of doing the thing is what makes them juniors.
    - charcircuit 8mo ago
      >that's what happens when juniors graduate with CS degrees A CS degree is going to give you much less experience than building projects and businesses yourself.
      - al_borland 8mo ago
        How much time will someone realistically dedicate to this if they need to have a separate day job? How good will they get without mentors? How much complexity will they really need to manage without the bureaucracy of an organization? Are senior software engineers of the future going to be waiting tables along side actors for the first 10+ years of their adult life, working on side projects on nights and weekends, hoping to one day jump straight to a senior position in a large company? The questions I instinctively ask myself when looking at a new problem, having worked in an enterprise environment for 20 years, are much different than what I’d be asking having just worked on personal projects. Most of the technology I’ve had access to isn’t something a solo hobbyist dev will ever touch. Most of the questions I’m asking are influenced by having that access, along with some of the personalities I’ve had to deal with. How will people get that kind of experience? There is also the big issue of people not knowing what to build. When a person gets a job, they no long need to come up with their own ideas. Or they generate ideas based on the needs of the environment they’re in. In the context of my job, I have no shortage of ideas. For solo projects, I often draw a blank. The world doesn’t need a hundred more todo apps.
bamboozled 8mo ago
From my experience, if you think AI is better than most workers, you're probably just generating a whole bunch of semi-working garbage, accepting that input as good enough and will likely learn the hardware your software is full of bugs and incorrect logic.
- bamboozled 8mo ago
  hardware / hard way, auto-correct is a thing of beauty sometimes :)

beej71 8mo ago

I'd always imagined that AGI meant an AI was given other AIs to manage.

davnicwil 8mo ago
I don't think this is how it'll play out, and I'm generally a bit skeptical of the 'agent' paradigm per se. There doesn't seem to be a reason why AIs should act as these distinct entities that manage each other or form teams or whatever. It seems to me way more likely that everything will just be done internally in one monolithic model. The AIs just don't have the constraints that humans have in terms of time management, priority management, social order, all the rest of it that makes teams of individuals the only workable system. AI simply scales with the compute resources made available, so it seems like you'd just size those resources appropriately for a problem, maybe even on demand, and have a singluar AI entity (if it's even meaningful to think of it as such, even that's kind of an anthropomorphisation) just do the thing. No real need for any organisational structure beyond that. So I'd think maybe the opposite, seems like what agents really means is a way to use fundamentally narrow/limited AI inside our existing human organisations and workflows, directed by humans. Maybe AGI is when all that goes away because it's just obviously not necessary any more.