4 ms·

I’d like to remind people that these ppl have no more knowledge about AGI as anyone else on this planet since there is no knowledge yet and everything they say

• reducesuffering 3y ago
  Yes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure. Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"
  • i5heu 3y ago
    They did not build a AGI yet so they have build as many AGIs as anyone else and therefore they are also laypersons regarding the effects of AGI on humanity.
    • reducesuffering 3y ago
      What do you call something humans built (artificial) that can solve all topics of AP tests (general) with most being 5/5, better than almost all humans (intelligence)?
      • roarcher 3y ago
        I call it an LLM, not AGI. Consider this scenario: - Ask ChatGPT to produce some output meeting certain criteria - ChatGPT responds, but the output does not meet the criteria in one or more obvious ways - Point this out and ask for a corrected version - ChatGPT says "I apologize, here's the corrected version:" and spits out a nearly identical answer with the same obvious mistake. - Repeat steps 3 and 4 ad nauseam. I experience this frequently when I ask it to do anything somewhat outside the norms of its training. It cannot correct itself or even see its own mistake. It cannot even conclude that it doesn't have the ability to do what I've asked and tell me so. Whatever sort of intelligence it may have, I would categorize it as not "general" enough to fall under AGI. Being able to answer AP test questions cannot be a sufficient measure of generality if a model can do it with flying colors but still fail at far more basic tasks.
        • hyperadvanced 3y ago
          I’d say that LLM’s inability to handle DSL’s in particular is an interesting question on the topic of knowledge and language. Should a generally intelligent entity be able to quickly figure out a new dialect with little background? If I were to air-drop you into an extremely foreign nation, how long would it take you to organically decode the local language and be able to meaningfully craft expressions in it? I’m not sure if humans are “generally intelligent” but a common bar for AGI is for it to be able to beat or meet average humans at normal tasks. I’m not sure if something like writing perfect k8s yaml should be a requirement, though I do agree that LLM’s inability to do word puzzles or relatively straightforward math should disqualify them.
          • roarcher 3y ago
            I agree with your points in general, though I wasn't talking about DSL generation or any specific task. I was talking about ChatGPT's general tendency to cheerfully apologize for mistakes, explain exactly what the mistake was, and then present the same mistake while claiming that it's been corrected. You can ask ChatGPT what it means to be asked to correct a mistake, and it will give you a perfectly thorough and eloquent answer. But it is often unable to apply this concept to its own behavior. If asked, it can explain back to you exactly what correction you want it to make. You can even make it pledge to correct the mistake in exactly the manner that it just described. And then it will completely fail to do it. It reminds me of that "repeat after me" meme from Friends [1]. This makes me lean in the direction of "stochastic parrot" when I think about what LLMs are. As impressive as it is, ChatGPT demonstrably lacks a sense of self. It talks as if it understands that it's an agent in control of its behavior, but then fails to control or even recognize its own actions. [1] https://en.meming.world/images/en/c/c2/Friends-_Repeat_After_Me_origin_2.jpg https://en.meming.world/images/en/c/c2/Friends-_Repeat_After...
      • kmnc 3y ago
        An advanced calculator.
      • giantrobot 3y ago
        We don't give AP tests to humans to determine whether or not they are in fact an intelligent being. We give them to humans to show that over the course of about a decade and a half that human went from not realizing it had hands to ingesting a bunch of knowledge and had the capability to recall it answer questions. ChatGPT was trained on hundreds of terabytes of information. I would hope that with all that effort it could answer AP test questions. That doesn't make it intelligent.
        • fragmede 3y ago
          what's your definition of intelligence that you're using here. there are various takes on what it means, so I'd like to know your take on it. That it's able to spit out essays makes it something, intelligence is too loosely defined to describe it for everybody, so we need to develop new words and meanings and new language for what it does. It doesn't think, but it does dot products to tensors to get create its output.
          • serf 3y ago
            intelligence in animal species, roughly, is how well the animal deals with novel situations. Corvid tool use is a common example of animal intelligence. It is interesting to consider AIs that way given that almost every AI will fail catastrophically when confronted with an area of expertise that is outside of their training corpus, although I guess Corvid tool use isn't exactly out of the animals' range of expertise.
      • otabdeveloper4 3y ago
        Google search can solve AP tests too. (And before that library indexes did the same.) This isn't some new technology.
        • ben_w 3y ago
          Google (and library indexes) can solve exactly and only the questions in their databases. All AI, even narrow ones, necessarily have the ability to interpret novel questions in some fashion — how novel and how well they interpret being the "G" and the "I" in AGI.
      • IncreasePosts 3y ago
        Fairly boring - "The answer is known and I want to see if you know the answer" is good for judging students abilities, but no real world problem exists like that. I will be much more hopeful for AGI when AI starts producing new and interesting knowledge, rather than regurgitating already known facts.
    • mitthrowaway2 3y ago
      Is your position that the only people who will ever be qualified to have an opinion that AGI is a threat that might destroy humanity, are the people who have already successfully built this thing? If that is the case, by what means might any credible warning be provided before a possibly-humanity-destroying AGI gets made? That's rather like saying "the only credible way to tell if there's a gas leak is to strike a match and see if we explode or not". I would suggest lowering the bar a little.
      • i5heu 3y ago
        No, my position is that you can not extrapolate on something that does not exist and that’s “category” ever never existed. We don’t know any significant other intelligent then humans, not even aliens, so we have just no trace of direction that is rooted in reality rather then fantasy. This is especially obvious since opinion differ so much about the effects of AGI on humanity and since you cannot prove them right or wrong, every opinion is equally realistic as each other. Like I could say that every AGI that becomes self aware will kill itself instantly because of boringness, and you cannot prove me wrong. We just don’t know yet.
        • mitthrowaway2 3y ago
          It still seems to me that you're saying that we won't possibly be able to declare AGI an existential threat to humanity until after it has already been built. At that point, we can presumably settle the question by seeing if humanity goes extinct. This poses something of a paradox to those of us who prefer existential threats to humanity not get built in the first place.
          • i5heu 3y ago
            We will not know if something is an existential threat until we can extrapolate on at least some data that is relevant. Feeling uncomfortable with this, will not change this sad reality.
            • mitthrowaway2 3y ago
    • ben_w 3y ago
      The hour before Thomas Newcomen invented the first practical fuel-burning engine in 1712, the most informed person in the world to do so was an ironmonger by the name Thomas Newcomen. The hour before the Kitty Hawk flight, the two most informed people in the world on the topic of heavier than air flight were a pair of bicycle retailers and manufacturers named Wilbur and Orville. These people sure don't spot everything, but they're still way closer than the average human.
      • xamuel 3y ago
        >The hour before the Kitty Hawk flight, the two most informed people in the world on the topic of heavier than air flight were a pair of bicycle retailers and manufacturers named Wilbur and Orville. Actually, this is quite false. There were many people working on flight with massive credentials and massive funding. That the Wright brothers beat them is part of what makes the story of the Wright brothers so interesting.
        • ben_w 3y ago
          That the other teams were better funded and yet failed, says to me the Wright Brothers were better informed. That society didn't know this at the time is indeed relevant to AI development, and the analogy would be that the next big thing might come out of some minor research team that most people dismiss instead of the biggest and most well known groups… a bit like how Google's LaMDA was making headlines before ChatGPT surprised everyone. It's certainly still possible today that some random individual has a crucial insight that gives them an edge over the big names, yet even those big names know far more than me. Which is rather annoying, because I'd be interested in taking the opportunity of having been laid off to switch from iOS to AI. (If anyone reading this is hiring people in Berlin, contact details on my website. Plus side of hiring me: I'm relatively cheap in that field, and enthusiastic about the possibilities).
        • IncreasePosts 3y ago
          Now, imagine if the people who were beaten by the Wright brothers had set up a regulatory system 20 years before their first flight, which instituted a bunch of rules and requirements that made it impossible for the Wright brothers (or anyone) to actually achieve flight.
  • halflings 3y ago
    Your comment (+ username) reads like what I would have written once upon a time when I was fully in the EA bubble. Truly no offense meant, as I was deeply into the EA movement myself, and still consider myself one (in the original "donate money effectively" sense), but the movement has now morphed into a death cult obsessed by things like: * OMG we're all going to die any time now (repeated every year since circa 2018) * What is your pDoom? What are your timelines? (aka: what is your totally made up number that makes you feel like you're doing something rational/scientific) I'm deep in the weeds w/ LLMs, e.g. I probably finetune an average of 1 model a day, and working with bleeding edge models... and AI safety just sounds so silly. Wanting to take drastic measures today to prevent an upcoming apocalypse makes as much sense as taking the same drastic measures when gradient descent was invented.
    • reducesuffering 3y ago
      My username was created before I knew anything about EA or adjacent. I'm not in any EA movement, though I am sympathetic. I've spent 100x the time on HN, with people mostly in denial, than I do in EA or adjacent forums, nor have I met any of them. It's sadly twisted how mentioning that -- the majority of leaders doing the cutting edge research on AGI think it has a significant chance that it kills humanity -- is considered being part of a "cult" movement. Your analogy is the same as early Intel engineers completely unaware that those chips would bring on the ramifications of social media. "In the weeds" and yet unable to foresee the trajectory and wider impact. Same with the physics that led to nuclear weapons. > Wanting to take drastic measures today to prevent an upcoming apocalypse makes as much sense as taking the same drastic measures when [nuclear fission] was invented.
      • halflings 3y ago
        > Your analogy is the same as early Intel engineers completely unaware that those chips would bring on the ramifications of social media Exactly! As they should be. (for both Intel engineers developing chips, and physicists developing nuclear research) There were a billion more potential dangers from those technologies that never materialized, and never will. I'm glad we didn't stop them in their track because a poll of 10 leaders in the field thought they were too dangerous and progress should stop. (note that no one is against regulating dangerous uses of AI, e.g. autonomous weapons, chemical warfare; the problem is regulating AI research and development in the first place)
• classified 3y ago
  They are just banking on their putative status as influencers to grab more influence.
• ecoquant 3y ago
  [flagged]