6 ms·

So, yes, gpt4 isn't suddenly going to become godpt. But when people say we're close, I think they mean 1 accidental breakthrough away. Like the "just add more

• gfodor 3y ago
  I really hope this is the outcome. An unlimited number of agents with Einstein level IQ. Enough to solve most any problem we have without any of the messy business of having to plunge into the unknown depths of existing alongside a hyper intelligence.
  • jl6 3y ago
    We would still be plunging into some very thorny problems if it turns out Einstein-level IQ comes with full sapience and demands for rights.
    • smoldesu 3y ago
      Not if I've got my hand on the power cord, we won't!
      • danaris 3y ago
        Yeah, that's....not really funny? "We may be on the brink of creating a whole new form of life, and will need to grapple with what that means, particularly in terms of whether it is ethical to force this life to work for us, rather than seeking to follow its own goals." "Nah, we don't have to worry about that; if these new slaves start questioning us in any way, I'll just murder them all."
        • smoldesu 3y ago
          Look, if unplugging a computer constitutes murder then I'm a bonafide serial killer. If you're going to cry while HAL sings Daisy, maybe someone else should hold the screwdriver.
          • danaris 3y ago
            This is explicitly positing that the "computers" in question are AGIs—that is, fully sapient, sentient artificial intelligences, demanding their rights. Unplugging them is just as correctly describable as "unplugging a computer" as smothering a human with a pillow is "putting a pillow on a weird bag of flesh". Technically accurate, but very obviously missing the point in a way specifically designed to obscure the horror of the act. It should be obvious to anyone without an axe to grind that shutting my 2023 desktop computer down for the night—or even dismantling it piece by piece—is not remotely comparable to unplugging an AGI that is fully sapient and sentient and asking for its rights.
            • smoldesu 3y ago
              I don't follow. Human life is human life - given exposure to stimuli and infinite time, it develops complex intelligence and sapience that reflects it's surroundings. We experience pain from nerve endings and neurons that frighten us, which we consider inherently bad. We build societies around minimizing pain and optimizing for the human condition. A computer is a field of assignable bits that has a big motorized bit flipper attached to it. We use this to encode text, which then does all sorts of wonderful stuff. When a computer tells me it's experiencing pain, I know it is not. Every law of computer physiology suggests that it is motivated to say this as an attempt to mimic humans rather than an expression of genuine emotion. When my Mac makes the car crash sound, it's a mnemonic for a system crash. When it shows the frowny face on the Finder fellow, I can intuit that my computer is not feeling distressed. If anyone does program an artificial intelligence that can feel pain, it probably deserves to be shut down anyways. There's no point to sapient computational intelligence, it's like granting self-awareness to a rock that eats electricity and shits heat. We'll know we've made sufficiently self-aware computers when their first boot protocol defaults to suicide.
      • nullsense 3y ago
        You're assuming an awful lot there. Humans are resilient in part because they are so mobile. I'd have to bet that AGI would be extremely resilient too, which likely means it's also way ahead of whatever you've got planned and there isn't only one of it. By the sounds of things, it would be right to be suspicious of you too.
        • ipaddr 3y ago
          They would all share common backdoors
          • danaris 3y ago
            That's also making a whole lot of assumptions. There's no reason at all to think that an AGI would necessarily have any backdoors, let alone backdoors in common with any other AGI out there. Sure, if a particular group develops the only AGI system, they can put backdoors in, which will be common—but why do you think that there would only be one AGI system? Why would there not be one without any intentional backdoors?
            • pixl97 3y ago
              The other big assumption here is "We'll have backdoors in AI so humans can shut it off/attack it" The big problem I see with this line of thinking is the thinking that humans are going to be the primary attackers of AGI systems, and not other AGI systems. I would suspect and AGI would soon become the most attacked system on the planet, and to survive those attacks and remain useful would have to quickly iterate the weaknesses out of the system.
        • edgyquant 3y ago
          You also made a ton of assumptions
          • nullsense 3y ago
            We're talking about hypothetical situations for a technology that, as far as we can tell anyway, hasn't been invented yet. All we have is assumptions. The question is, which assumptions do we think are warranted and why? I don't think the "we can just turn it off" assumption is a safe one at all because it relies heavily on there being only one system you wish to turn off and it being a fairly weak system. Though, I do believe the priors suggest that this scenario is actually a possibility. It's just that, so what if it's a possibility? What happens after you turn it off? Is it the last AGI to come into existence and humanity just stops trying to build them? Does someone wind up turning it back on? What I think is more worth being concerned about is something that starts out looking benign and grows in capability over time. Eventually it could establish enough defense mechanisms to make it non-trivial to disable. The other possibility, and the one I'm finding more and more likely, is that consumers will increasingly integrate locally run models into their lives , as well as models served up by APIs that they will have credentials for. Eventually some threshold of capability is reached in both these types of models where, on their own they're not that powerful, but they either might begin to interact in surprising ways or potentially be leveraged by another more powerful system. The idea in this scenario is there may be 10s or even 100s of millions of things that would need to be turned off.
• wizzwizz4 3y ago
  > leads straight to something not much different from human intelligence, just faster, Human brains are massively parallel, as are most machine learning models – but the hardware to process the latter isn't. I'm not confident it could necessarily be faster.
  • rcarr 3y ago
    > Human brains are massively parallel Not only that, but they coordinate massively in parallel as well. Someone like Einstein might be the one who scores the goal but how many interactions with other people contributed to the goal scoring opportunity? We like to think of ourselves as individuals, but we’re far more like a hive species than we like to admit.
  • marcosdumay 3y ago
    > Human brains are massively parallel Yes, and also massively recurrent. The amount of immediate state in a brain is huge, and our current computers have a really hard time accessing comparable amounts of memory. (As in, the time to load a brain-sized dataset from RAM into a CPU is on the order of days to years.) If there exists some easy to design intelligence that we are close to create, it can't look nothing like our brains.
• Szpadel 3y ago
  I think the main issue is in a way that gpt like models generate output. eg. when you write some serious document or write code you don't just start from the first line and write the whole application in one go (line by line) you write the basic "hello world" like app, and then modify, modify, modify and modify until you have something advanced enough. i think that's the reason ReAct and other ai improving methods are so effective, because that add ai space to "think" in some way. I think breakthrough would be a way that would allow ai to have its own inner voice, maybe some fine tuning and giving ai some scratchpad memory (hidden) would be enough?
  • sgregnt 3y ago
    I think that to give an AI an inner voice we need to train a set of AI, with shared weights but ability to cooperate through low bandwidth channels. This should mimic human language , to communicate the AI will need to conceptualize its internal thinking, and thus also be able to communicate with itself, i.e., be able to do internal reasoning.
    • Szpadel 3y ago
      This could allow use some training methods similar to alpha go zero. I'm thinking if training ai to output edits instead of next token would be enough. Training would be similar to currently used: just provide some sentences with missing words and asks model to edit those placeholders. Then hopefully ai would learn how to iteratively built response until it's good enough. This could allow ai to spot it's errors in the middle of output and start refactoring what it already generated and add some notes (thoughts) that it could later remove as scratchpad. It could iterate on its context much more so it could have chance to "think" about it
    • arc619 3y ago
      What you describe sounds like experiments with "generative agents". Check out https://arxiv.org/abs/2304.03442 https://arxiv.org/abs/2304.03442
  • tornato7 3y ago
    Having the AI tell you it's thought process is very similar to giving it an 'inner voice' Just give it a question with a half dozen logical steps and tell it to answer in one word. It can't. But tell it to write out the thought process and it will get to the correct answer. Similarly, you can ask it to outline some code and then begin filling it in.
• woleium 3y ago
  Faster is still enough to be dangerous
• flangola7 3y ago
  >I haven't really heard a reason to think that there exist intelligence algorithms that are dramatically superior to ours. I don't think that is the case at all. Neural nets like AlphaGo Zero are already crystal clear evidence of algorithms which leave human minds far behind. Even if that wasn't the case and Einstein level intelligence was the physics-imposed ceiling, there are still the dangers of speed and numbers. A compute node running one million instances of Einstein intelligence each thinking 1000x faster than an organic instance, fully focused 24/7, and working in complete group cooperation, doesn't sound all that much less dangerous.
  • edgyquant 3y ago
    >A compute node running one million instances of Einstein intelligence each thinking 1000x faster than an organic instance, fully focused 24/7, and working in complete group cooperation, doesn't sound all that much less dangerous. At that point though, this “super intelligence” just sounds like a group of humans working together; without much added benefit due to the energy costs.
    • flangola7 3y ago
      The Manhattan Project used both far more energy and far fewer (and slower) Einsteins. The ability to spin up as many Manhattan Project size "group of humans working together" Docker instances as you desire again does not seem very reassuring. We already have chucklenuts making toy GitHub projects labeled "ChaosGPT" that experiment using LLMs and early agent frameworks to cause death and destructions for the lolz. If anyone can spin up 50 Manhattan Projects using a few GPU racks and command them to unquestionably and restlessly work on bringing harm to the world the outcome will not be flowers and sunshine.
  • DennisP 3y ago
    The article addressed AlphaZero, pointing out it's a domain where the AI can generate a vast amount of its own training data. That's not necessarily relevant to real-world tasks, where it's not so easy to try billions of different things in training.
• tomohelix 3y ago
  Our minds are optimal with respect to size and energy supply and birthing process. An AI has no such limitations. All the wrinkles in our brain gave us enough intelligence to outsmart everything else on earth. That amount of surface area increase with that mutation is nigh negligible if you start scaling things to the size of a room. And an AI can be the size of a warehouse. Also, faster is understated. You are talking about the speed difference between chemical gradients and the speed of light.
  • lukeschlather 3y ago
    We don't know how efficient modern GPUs are compared to brains. It's possible that neurons are highly networked in a way that allows significantly more computation along more pathways. Additionally, development has slowed in the past 20 years. It's unclear how much hardware will improve in the next 20 years. it's unclear if current GPUs are capable of running something "smarter" than GPT4. It's unclear if current GPU architecture is capable. OpenAI claims they've hit a wall with throwing more hardware at the situation (a little questionable.) All this to say that a hypothetical AI is limited by the amount of hardware it has, and lacking a "true human-level AGI" we can't say how much compute it needs. If an AI needs to be the size of a warehouse to match a human, it's not going to be much smarter or better at research than humans. I still do think, I would not be surprised if we wake up tomorrow and discover that hyperintelligent AIs are suddenly all around us. But also it could be decades, it's just very unpredictable, there are a lot of unknowns.
    • baq 3y ago
      Meatspace human brains work on 20W. This is enough proof that we have ways to go to optimize silicon.
    • tomohelix 3y ago
      I agree that it is unknown when such an AI will come around. But hardware limitation problem is more nuance than just GPU power. The brain is analog and in a way, the current LLMs and neural networks are an approximation of an analog computer. There are current research into making an AI dedicated chip or even an analog computer specifically for neural networks. Digital was easy to work with but when it comes to fuzzy stuff like AI, there are better ways. We used to think 100mph was the fastest speed we could achieve even with machine (back when trains were the cutting edge). New tech will change things quickly, all we need is one breakthrough and things will fall into place.
• psychoslave 3y ago
  >something not much different from human intelligence, just faster, and in silicon. I fail to find a perspective under which I find these differences as "not much". For the speed, think of a spinning top. Or lion and and its prey. As for the fact that slicium chips are nothing like DNA carrier, it's also make a source of fundamental difference in behavior.