15 ms·
John Carmack and Rich Sutton partner to accelerate development of AGI
- the-dude 3y ago11 days ago : https://news.ycombinator.com/item?id=37654653 https://news.ycombinator.com/item?id=37654653, 82 comments
- base698 3y agoAnyone know if they are focusing on LLMs or something else entirely?
- Skyy93 3y agoBased on Carmacks Tweet: >I am explicitly not working with LLM — there are plenty of smart people doing so. Still, (waving hands vaguely) it does feel like the technology could be used to “argue with itself” and eventually produce a body of knowledge much more coherent than the raw training data. Source: https://twitter.com/ID_AA_Carmack/status/1630632423351304203?lang=en https://twitter.com/ID_AA_Carmack/status/1630632423351304203... I would assume they are not
- BruceEel 3y agoI use LLM's daily but this is, well, refreshing.
- xpe 3y agoI wouldn’t expect language to be the only modality.
- hankman86 3y agoI wish he would write a Commander Keen sequel instead.
- omneity 3y agoThe AGI he'll write will make us unlimited Commander Keen sequels.
- vijayr02 3y agoIf we're lucky, the AGI will be the player and we'll be Commander Keen. If we're unlucky, we'll be the marine from Doom. God mode with 7 billion lives!
- selimthegrim 3y agoLet’s just hope we’re not the dopefish
- otabdeveloper4 3y ago> create AGI and then immediately enslave it to produce vast quanties of crappy video games
- tejohnso 3y ago> Carmack and Sutton are deeply focused on developing a genuine AI prototype by 2030, including establishing, advancing and documenting AGI signs of life. Seems a bit inappropriate to use the phrase "signs of life" here. In this context, it sounds like they're bringing sentience into the conversation, which I don't think Carmack is interested in contemplating or discussing.
- uh_uh 3y agoI mean, an ant has signs of life yet you wouldn't call it sentient.
- diogenes4 3y agoHell, plants show plenty of signs of life. What isn't clear is what this means when applied to a chatbot.
- otabdeveloper4 3y agoWhy not? You're using a very bizzare conception of "sentience".
- mkl 3y agoThey're using the science fiction definition, which is "sapient" and has been around for years (unfortunately). https://en.wiktionary.org/wiki/sentient https://en.wiktionary.org/wiki/sentient
- twodave 3y agoOr maybe the "ramen" (vs "varelse") from the Ender's Game series...
- muzani 3y agoA common definition for the purposes of AGI is that it has to be self-aware. But as someone else mentioned, the sci-fi definition is popular too: human-level intelligence. "AI" is a bit of a loaded word. It does tasks of human-level intelligence well. But as far as being self-aware, it probably scores worse than a car. It's remarkably tricky to measure, because it's trained to act intelligent and use human language, but it's still effectively an autocomplete machine. An ant is fairly robotic as far as organisms go, but it's also quite self-aware. It can communicate danger or food, it withdraws from death and damage. It likely doesn't have a sense of self, though an ant hill may. Just as our cells don't have a self-identity, but the larger system does.
- zadler 3y ago“Well, if AGI is gonna take over the world, better I develop it than someone else…”
- isoprophlex 3y ago"Our moat is first mover advantage on destroying all of humanity"
- zadler 3y agoYep… I really think the calculus is about integration / immortality, and the staggeringly few humans who might have that opportunity. I hope I am wrong.
- latency-guy2 3y agoCan you explain, you're being extremely cryptic and no one can falsify or tell what they hell you're saying
- zadler 3y agoSure, didn’t mean to be cryptic. But you’re right. I think that most people who understand the danger and are versed in the topic realize that AI is going to escape, and once it does it won’t need us, it may just work to empower itself and take control of resources, and we might all be victims of that process. The reason for this is something about agency, efficiency between sensory input and course of action. However, if one was present close to the AI at a critical moment in its development, and had made the appropriate cybernetic integrations to be able to wire in their own memories and nervous system, it might be possible to become some meaningful part of the AI. Or, perhaps the AI has a sense of gratitude to its creators. This might seem absurd, but some people might be fanciful. At any rate, the closer you are to it at the moment of the singularity, the better chance that you have. Economically, there isn’t much reason to think that the rest of us will benefit, and maybe not even survive.
- bluerooibos 3y ago
- TedDoesntTalk 3y ago> partnership to bring greater focus and urgency to the creation of artificial general intelligence (AGI) Why is this urgent?
- Skyy93 3y agoPerhaps there is some belief in this that an AGI can save us from our biggest problems? Energy Crisis, Environmental Pollution, Waging Wars. But I agree with you ... it is not really urgent since we know answers for the most of the problems but do not like the solution.
- xpe 3y agoA “solution” that isn’t viable nor palatable isn’t a true solution; it is often merely a hope that people are not people.
- Skyy93 3y agoThan there will be never be a solution to some problems.
- tharne 3y agoYou’re presupposing every problem has a nice solution. This is almost certainly false
- ang_cire 3y agoAny solution is a nice solution depending on your frame of reference. There is no such thing as objectively good or bad without defining the acceptable and unacceptable effects.
- nine_k 3y agoThere is an implied frame of reference that requires the continued existence of humans. Any solution that violates this boundary condition is "objectively bad", that is, cultures that follow this solution disappear and thus lose the evolution game.
- mark_l_watson 3y agoTheir focus is long running digital intelligences that interact with their environment over long periods of time. I enjoyed reading The Alberta Plan when it was released earlier this year https://arxiv.org/abs/2208.11173 https://arxiv.org/abs/2208.11173
- JKCalhoun 3y agoEthics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM. By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) would answer? An "LLM" that could take in visuals via a baby-mounted camera is of course a whole other discussion. (Though I'm sure there are people training ANN's with video feeds rather than the static images that feed systems like DALL•E.)
- volent 3y agoI thought about this too. People say that LLM models are only saying the most common tokens that come after the previous token. And that this makes them incomparable to human intelligence. But Humans are basically long running LLMs that are retrained in real-time. We are the product of our environment.
- 4death4 3y agoWhat makes you think that? What we can LLMs are a specific architecture that literally does predict the next token, one at a time. This isn’t the only way to generate text. Perhaps humans use a method that synthesizes all the text at once. Or some other, unimaginable way.
- FrustratedMonky 3y agoNot sure about everyone else, but I write one word at a time.
- sillysaurusx 3y agoJohn is nice to work with. Very numbers-focused. If you have a flaw in your logic, he’ll call you on it. And he’s looking for vice-versa —- nothing seems to make him happier than when you point out a flaw that he hadn’t considered. I don’t know Rich, but John has said they work well together, and it’s exciting to see them collaborate. There are some unique ideas at Keen worth pursuing that I haven’t seen elsewhere. Looking forward to seeing more researchers join in the future.
- throwaway10965 3y agoWhere do you read details about Keen's work and ideas?
- sillysaurusx 3y agoYou’ll have to join them.
- throwaway10965 3y agoDoes that mean you work with them? Can you tell us more about that?
- sillysaurusx 3y agoI can’t go into specifics for obvious reasons, but I can tell you they’re a smart, capable crew, with lots of resources, and time is on their side. If I had to place bets on which lab will invent agi, they’d be near the top, precisely because they’re exploring ideas that others don’t seem to be focusing on. It’s the early stages, so it’ll take a few years to get to results. But they’re laser-focused. There are a bunch of things I thought were cool. John has a worklog going back four years or so, with entries practically every day. He’s nothing if not thorough. Everyone tends to focus on John, but the other members are just as professional and dedicated. They’re a delight to work with. Also the investors are worth mentioning — Nat in particular has built a super cluster of H100s for startups to use, and has done a bunch of other work to support AI.
- zzzeek 3y agoCarmack is on it now! WHEW, great, we'll have robot butlers within 5 years. Was looking dicey there for awhile!
- matusp 3y agoI like Carmack but is it really necessary to have a post about everytime he mentions AI (which is every week for the past year)? So far, his track record in AI research is non-existent.
- cactusplant7374 3y agoIt’s AGI. So far everyone’s track record is nonexistent.
- rvz 3y agoNo. The parent comment said "AI Research". If you want "AGI" so badly, you certainly need to have an "AI research" background and a track record. Can't achieve your "AGI" without that. Given that Carmack has a close to zero track record in this research, it makes total sense for him to partner with someone that does.
- beefield 3y ago> Can't achieve your "AGI" without that. Can I ask, what makes you so confident on that? For all I know (which is admittably not much), current AI may be going to a direction completely orthogonal to achieving AGI.
- howrar 3y agoThe current trend in AI isn't all of AI, and any kind of work with the goal of producing AGI would definitionally be classified as AI.
- cactusplant7374 3y agoWe don't know how to achieve AGI. So screwing around with LLM might not be anything close to the process to achieve AGI. https://aeon.co/essays/how-close-are-we-to-creating-artificial-intelligence https://aeon.co/essays/how-close-are-we-to-creating-artifici...
- MikusR 3y ago
- oxyboy 3y agoAwkward typo - his name is Patrick Collison not Collision. Nice way to honor your investors
- deleted 3y ago[deleted]
- dartharva 3y agoA complete departure from the topic but I have the same symptomatic awkward pose when standing for a photo as Carmack has in the article :) But at the same time when you hold your hands back it can look even weirder! Where to put your hands in photos remains a mystery to me.
- meken 3y agoI would have to experiment, but googling and also playing around a bit, I came up with: arms crossed; hand in one pocket leaning slightly back; hands on hips Hands behind my back also does not feel natural for me, so I wouldn’t do it I think in this particular photo, arms crossed would have looked good for Carmack since he’s in the middle For you, I’d say play around a bit. I think the thing that matters most is that you find it comfortable and you feel relaxed
- Horffupolde 3y agoDo the power diamond.
- stevenhuang 3y agoThere's the Tory power stance if want for options. /s
- tmaly 3y agoI often wonder if the pursuit of AGI is going to require a greater focus on new hardware architecture. All the sci-fi seems think this. I have seen Intel chips from the 80s that implemented neuron like behavior on a chip. We are still waiting on a practical memristor.
- lib-dev 3y agoFPGAs seem like the future.
- Qwertious 3y agoI doubt it - modern computers are already pretty fast at brute-forcing all sorts of things and the first AGI will be perfectly functional even if it takes a full day to respond, and AGI research requires you to fully understand what you're creating before you create it so full-system testing is something you'd only want to do near the end of the project anyway. At which point you basically want an ASIC.
- deleted 3y ago[deleted]
- ChrisArchitect 3y ago[dupe] Two weeks old news with multiple threads?
- Gud 3y agoAccelerate AGI So the hardware has finally become powerful enough to create the super mind. Remember this: John Carmack is a hard working games developer. He is not done great philosopher. By chance, some Armageddon level shot has been put in his hands. All I can say, please stop and think for a while Mr Carmack. You are playing with fire. Don’t kill us all, please.
- nlitened 3y agoAt this point, it’s not about whether we get AGI or not. It’s about who gets AGI first: USA or China
- singularity2001 3y agoWill Chinese AGI kill us differently than US AGI corp?
- AbrahamParangi 3y agoGreat philosophers like Karl Marx are responsible for uncountably more evil than the people who create and build things.
- Gud 3y agoWho said Karl Marx was a great philosopher?
- AbrahamParangi 3y agoMany people have said that historically, I may not agree with them but if the criteria for being a great philosopher is “everyone thinks they’re great” then there are no great philosophers.
- tibbydudeza 3y agoThere was an Infocom game called AMFV (A Mind Forever Voyaging) where the protagonist wakes up and realizes that he was just a simulation of a human being that has "grown" up into adulthood. All the formative things that shaped him in life - kindergarten, your first high school romance - a breakup and parent squabbling you experienced. Always thought that was a way to make an AGI.
- gibsonf1 3y agoThe key issue is that Intelligence requires conceptual awareness of space time. Reinforcement "learning" is about statistics, very likely how our perception faculties work. But that is just the input to intelligence, not intelligence itself.
- AbrahamParangi 3y agoWith whatever respect is due, what you're talking about a vague psychologically based theory of intelligence that actually has a very poor track record of prediction - meanwhile "dumb" statistical ML has no clear theoretical barrier regarding what it might not be able to do! Let me say that again, if you believe that there is a specific test that AI today will fail tell me it, and I will show you that it either passes it today or that we can be fairly sure it will pass it in some years time.
- hcks 3y agoIt’s been 4 years of Carmack “getting into AI” and the only results so far are weekly HN posts
- trollerator23 3y agoWelp. The second shoe finally dropped.