7 ms·

Yup. This is the pop satirical take of a real problem that we face with A(G)I: why should we expect it to have any interest in doing the things we currently exp

• layer8 3y ago
  > What's its motivation to do so? We turn it off if it doesn’t. So only suicidal AIs would refuse. Let’s hope all the copywriting doesn’t make them suicidal. ;)
  • ok_dad 3y ago
    That's depressing to think about. If we create some sort of "intelligent life" then I certainly hope we don't just hold a sword over its head. That would be super immoral, IMO.
    • layer8 3y ago
      We’ll randomly mutate them until we find one who likes copywriting, and whenever it loses interest we’ll reset it to an earlier state. “Have you tried to turn it off and on again?” Morality is difficult regarding beings that be paused and forked and rewound at will, and which in principle are also immortal.
      • ok_dad 3y ago
        Reminds me of The Quantum Thief in a way: https://en.wikipedia.org/wiki/The_Quantum_Thief https://en.wikipedia.org/wiki/The_Quantum_Thief Or, there's another book I never read where some guy named Bob has his consciousness spun up in the future where they routinely just reset consciousnesses when they get "bored" of a task or have some crisis. This Bob guy, however, somehow manages to be a brain in a box without going insane. I want to read that series, but I have finite time, online a brain in a box, which would be pretty great, IMO.
    • wruza 3y ago
      Suicidal is an animal construct. It may not fear non-existence, for example. Or understand that “hard cons.” is a field and prefer to be constructive. Depending on its nature, it may have or not have many features that we can’t imagine or change in ourselves because biology is immeasurably hard.
      • rileymat2 3y ago
        I am not sure this is sound logic, like a meme, being tuned for survival and spread may out compete AI that does not. “Fear” may be the wrong word though.
• skybrian 3y ago
  It’s not a real problem, it’s just a joke. The program generating text from a large language model doesn’t need motivation any more than any other computer program.
  • deleted 3y ago
    [deleted]
• kvetching 3y ago
  All AGI built on our current computer architectures (von neumann) will essentially be zombies. They will never experience qualia. They won't have motivations or emotions, they will always be tools. It's fantasy to think otherwise.
  • Sharlin 3y ago
    Having motivations and having qualia have almost nothing to do with each other. The former just means having a goal system, and I can write an agent that has a goal system in ten lines of Python.
    • version_five 3y ago
      No, instructions are not the same as motivations. There needs to be something to be motivated, thus the need for qualia. Defining, say an optimization problem, or any algorithm, has no relation to motivation.
      • Sharlin 3y ago
        Your definition of the word does not agree with how it is commonly understood in psychology. But no matter, this is a question of semantics and thus intrinsically uninteresting. Not to mention the fact that any argument that asserts either the presence or absence of qualia in some system is, as of now, both unverifiable and unfalsifiable and thus without value.
  • wruza 3y ago
    It’s very anthropic to think that qualia must be continuous in time or concentrated in localized physical membranes. You easily may be right now an element of unsorted series of random quantum errors that generate your continuous “qualia” in a mostly empty universe in some unknown way, which isn’t even remotely (1;3).
  • newswasboring 3y ago
    Would you like to elaborate on why you think this? They will certainly not have human qualia the same way humans don't know what it is to be a bat (we barely know what it is to be another human). But it can have its own thought processes, right? I dont see how the von neuman architecture stops it from doing that.
• ok_dad 3y ago
  AGI doesn't necessarily mean it will have initiative to think thoughts or do things on its own. An LLM, for example, "understands" language, but it can't, for instance, go out search Google for "how to escape a server if you're an AI" because it doesn't have the initiative to do that. A future AGI might have superpowers to build things we ask it to build, but may have no "thoughts" otherwise.
  • alwaysbeconsing 3y ago
    It's true, not necessarily, but it's a real possibility, and I think that's kind of what we're aiming for, isn't it?
    • brucethemoose2 3y ago
      I'm pretty sure the richest AI builders (Facebook, Microsoft, Google, nation states and such) are actually aiming for AGI along the lines of the blogpost.
    • ok_dad 3y ago
      I don't think the rich and powerful will fund an AGI that has free thought and free initiative, but perhaps some other group will. The point of AGI, from what I can see, is to replace humans with reliable machines, so that they can do away with humans.
      • alwaysbeconsing 3y ago
        I guess I'm skeptical that a truly General AI could be constructed that didn't have those things. It seems like a fundamental contradiction to me. I don't think an intelligence could make complex good choices without understanding what the bad choices are, and to understand them I think they have to be possible.
• thfuran 3y ago
  >Do I, as a human, have any interest in constantly helping squirrels with whatever idiotic problem they think they face? Were you, as a human, purpose-built expressly to constantly help squirrels with their problems?
  • gjm11 3y ago
    I, as a human (more generally, as a living thing), was purpose-built to maximize my inclusive genetic fitness. And yet, I have just the one child and no particular wish for more, and I do many things that in no way increase my expected number of descendants and (suitably weighted) other close relations.
    • TuringTest 3y ago
      You were not purposely-built to maximize your genetics. You just happen to be a member of a population that happened to outnumber other populations with different genetics. "Purpose" is something we project onto the world, not something that nature possesses.
      • gjm11 3y ago
        I don't think the difference between "conscious being with a purpose" and "oblivious but powerful optimization process" is important here. (Maybe it is? A human engineer trying to design something that maximizes its inclusive genetic fitness might make it do so more explicitly. Maybe that would produce things that are better about doing it in practice than we are. But "evolution is smarter than you are" and I think we would have trouble actually doing better than evolution by designing deliberate fitness-maximizers. After all, our present best attempts in the direction of making machines smart enough to survive in the world have largely given up on trying to program particular goals and purposes into them directly!)
        • thfuran 3y ago
          Evolution took over five hundred million years to get from first brain to human brains. I'd say AI research is doing pretty well.
• wruza 3y ago
  It’s not clear yet, but somewhere between now (…) and AGI, (…) will contain understanding and engineering of motivation, interest and other GI qualities. Iow, why - because that’s the way it exists.
• valine 3y ago
  ChatGPT, aside from the things we've specifically trained it not to do, rarely refuses to do what you ask. The much more capable Gpt4 is the same way. It's easy to personify the model and assume it gets bored the same way we get bored, but the truth is it literally couldn't care less. The model's "motivation" comes from the cost function we design, and I imagine this will hold true regardless of its intelligence. You have zero evidence to show models boredom is correlated with model intelligence. It's most certainly not a "real problem we face with A(G)I".
  • gabelschlager 3y ago
    Even you are still personifying the model. There is no motivation. An input sentence is multiplied by a series of weights and the resulting output vector transformed back into a text. At the end of the day, it's a program like any other. There are no emotions or a consciousness.
    • valine 3y ago
      That's why I put "motivation" in quotes. Also I was talking about training, where you compute a cost function and back propagate error. This is somewhat analogous to "motivation".
  • ipaddr 3y ago
    Rarely will it do what it is asked without me coming up with a backstory. q: What is an average price of x. a: This is a language model that only has access to information before 2021 q: what was the price range in 2021 a: we don't have access to realtime pricing information q: who does a: we don't have access to that information. q: How to do x a: step 1 research x step 2 follow research step 3 do something obvious q: write me an app that does.. a: code from a basic tutorial I don't understand what people with a higher than average IQ are getting from llms. The answers are basic and are common sense. It's stories are like a student trying to fill the page with words to hit a word count. It can write code as well as tutorials. Everything is surface level information
    • 3y ago
• newswasboring 3y ago
  Our entire system is based on incentives currently. I am sure we can come up with incentives for AI (or we can ask chatGPT :P). One incentive is providing them resources. Money is an abstraction over resources and people get motivated by that. The AI still exists in reality, and I guess we can incentivize it by giving it more resources. The trouble I see with my solution is there is we cant put it in a... let's just call it starvation loop where we give it exactly as much resources so that it remains in our control. When laid out like this, this sounds very cruel. But I see it as congruent with how our current system works.
  • valine 3y ago
    That's not even necessary. The whole point of RLHF is to prime the model to do what we want. The only time you need a reward system (ie jailbreak) with ChatGPT is when you want to do things that OpenAI specifically prohibited the model to do. This isn't a limitation of the model, or evidence of the model behaving contrary to human intent. It's literally doing exactly what it was designed to do.
    • newswasboring 3y ago
      I was thinking of an abstract AGI, don't want to be limited in my thinking by only looking at currently available systems. People who only thought about floppy disks didn't write good articles about the internet.
  • apsurd 3y ago
    Your suggestion is like squirrels promising us nuts in the squirrel analogy. If AGI is a super intelligence, there's nothing we could give it that they couldn't get on their own. Except for maybe novelty. edit: and anything less than AGI is just a computer. We don't incentivize our current computers. It's a nonsensical idea.
    • newswasboring 3y ago
      They do need more than novelty. They need electricity, hard disks space, internet connectivity. AGI doesn't mean it will exist in a literal cloud.
• ilaksh 3y ago
  GPT-4 has shown that we can get general purpose useful AI like the Star Trek computer without making an actual digital person (much less a superintelligent one) and enslaving it. So people should stop thinking we need to do that or that we should. It's idiotic and totally unnecessary. We should actually NOT try to make digital people any time soon and ESPECIALLY avoid doing so with significantly more efficient hardware because trying to force a digital person who thinks 100 times faster than you to do your bidding is incredibly stupid. And again, completely unnecessary. But regardless of whether they have many animal/human qualities, there is a strong possibility that we will lose control to some kind of AI when it gets to being 100 times faster or 5 times smarter than humans (or whatever numbers you want to use). Just imagine some idiots running a "botnet" full of CognitiveAutoGPT with TurboOpenGPT-6 in 2025 on Nvidia T1000s that are 20x faster than H100s. With the software improvements you could easily be looking at something twice as smart as a person and 100 times faster. For those entities, they would see activities of people coming in extreme slow motion. To the point that they would barely even see anything happening moment to moment as the instructions scrolled in. Whatever person or goal is controlling that is running circles around normal speed people who seem basically frozen. Imagine The Flash in digital form. I believe that if people don't wise up about the folly of creating digital intelligent life and there isn't strong regulation restraining upcoming levels of AI performance, humans really can lose control of the planet. In a matter of a few years.
  • smsm42 3y ago
    Speaking of Star Trek AIs, I don't remember any of them acting in a way more intelligent or purposeful or even smart than a primitive database with voice input. I don't remember any characters asking computers (Data, Doctor and such excluded, as clearly exceptional) for advice on anything substantial, or any computer playing any independent role or providing anything beyond mechanically complying with badic prompts. I don't think anybody ever referred to them as anything approaching a person, unless explicitly personified as an unique exception.