22 ms·

John Carmack's new AGI company, Keen Technologies, has raised a $20M round

461 points by jasondavies 4y ago

fennecfoxy 4y ago
Is it just me or does anyone else think Carmack is all hype engine now? Don't get me wrong, I've read master of doom/doom engine books like we all have, but first Oculus/Facebook and now this? Maybe I'll care once they produce something amazing, but until then I'll still marvel at the tricks in Doom/Quake code.
Trasmatta 4y ago
Interesting to see how he's progressed with this. When he first announced he was getting into AI it sounded almost like a semi retirement thing: something that interested him that he could do for fun and solo, without the expectation that it would go anywhere. But now he seems truly serious about it. Wonder if he's started hiring yet.
- deleted 4y ago
  [deleted]
- madrox 4y ago
  I got the same impression, and maybe it still is. You can still raise money for a retirement project if the goal of the money is to hire a staff. VC money isn't solely for young 20-something founders who want to live their job.
  - solveit 4y ago
    I suppose if anyone could raise VC money for a retirement project it would be Carmack...
  - rebelos 4y ago
    Carmack sounds like someone who lives his job, so I don't think age/life stage is a factor here.
    - russtrotter 4y ago
      agreed, Carmack's work ethic, opinions on work and opinions of how those around him work are legendary!
    - bhdlr 4y ago
      > Carmack sounds like someone who lives his job, so I don't think age/life stage is a factor here. He said on lex Fridman that he's going all in on it. He said he was tired of his 1 day a week consultancy at oculus and wanted to work full time on this.
      - MikusR 4y ago
        https://twitter.com/ID_AA_Carmack/status/1560729196166529025 https://twitter.com/ID_AA_Carmack/status/1560729196166529025 > "I am continuing as a consultant with Meta on VR matters, devoting about 20% of my time there."
- tux1968 4y ago
  He mentioned, in his Lex Friedman interview, that accepting investor money was a way to keep himself serious and motivated. He feels an obligation to those putting their money in.
  - mywittyname 4y ago
    Ah, I was thinking that $20MM doesn't seem like a lot of money for someone like Carmack. Surely he could have self-funded a business himself. This explains why he didn't.
    - nighthawk454 4y ago
      yeah, his follow up tweet is pretty clear: > This is explicitly a focusing effort for me. I could write a $20M check myself, but knowing that other people's money is on the line engenders a greater sense of discipline and determination. I had talked about that as a possibility for a while, and I am glad Nat pushed me on it.
      - sytelus 4y ago
        It seems like bad logic. If he gets bored/throw in towel, he can just pay back with his money and move on. Also, if you need to create such artificial constraints in order to go to work, a main ingredient called “drive” is missing. I respect Carmack a lot but I don’t trust old centi-millionaires to work with same passion as their younger poorer selves. Finally, $20M is too low. AGI is not a solo endeavor and doesn’t fit in Carmack’s legendary work style. Just to train a large model once will eat up 20% of this money.
        leetrout 4y ago
        > Just to train a large model once will eat up 20% of this money. Fascinating point. It has seemed like AI could produce a sort of arms race by players with the most resources because it is so compute intensive upfront.
        randartie 4y ago
- mhh__ 4y ago
  Does he have the expertise to pull it off as an individual?
  - celdon25 4y ago
    I would say so. Most AGI work so far has been pursued by academics with a lack of software engineering best practices. Codebases get bloated and hard to understand. Ramp-up time measured in years. This hinders experimentation, which you have to optimize for with projects like this. I would suspect Carmack understands this, and also understands what resources to consult, and how to spot the bullshit artists in the field. I'd work for him in a heartbeat.
    - mhh__ 4y ago
      I'm just skeptical of his technical skills in this area. I need to watch the lex thing but I don't really rate lex in this area either. It's a fairly big shift from games. Management might be OK but setting the direction of the company needs more inspiration than just writing good C++ code
- celdon25 4y ago
  > something that interested him that he could do for fun and solo Really? I don't think anybody who works on AGI (not AI) without being dead serious about it will be qualified to work on it. A bad AGI implementation is one of humanity's gravest risks, and this is well-known in the AGI field, has been for years. Elon was far from the first to say this. Carmack isn't the type to get those kinds of details wrong.
arkitaip 4y ago
> This is explicitly a focusing effort for me. I could write a $20M check myself, but knowing that other people's money is on the line engenders a greater sense of discipline and determination. Dude doesn't even need the money...
- deleted 4y ago
  [deleted]
- Cyph0n 4y ago
  Best humble brag I’ve ever seen.
  - kennedywm 4y ago
    Doesn't strike me as a humble brag at all. He just seems self-aware about how he's motivated and that that he functions better when it's someone else's money on the line.
    - Cyph0n 4y ago
      Oh, I understand the intention.
  - mhb 4y ago
    If you liked that you'll love the Lex Fridman interview.
    - Cyph0n 4y ago
      I watched some clips from the interview, good stuff. I personally don’t like Lex’s interviewing style though, so I couldn’t watch the whole thing.
      - SyneRyder 4y ago
        Glad I'm not the only one. I'm 90 minutes in and finding some of Lex's questions really bad. Especially "What is the best programming language?", but I'm also astonished that Lex doesn't seem to know what an x86 is, hasn't really heard of Pascal, and was incredulous that anyone could write a game in assembly language (but as Carmack points out, everyone in the 80s was writing games in assembly, BASIC just wasn't fast enough). Maybe Lex was trying to dumb things down for a non-technical (or younger?) viewer, but it doesn't come across that way and just interrupts Carmack's flow. Carmack is excellent though.
carabiner 4y ago
It's happening.
- paxys 4y ago
  What's happening?
  - swagasaurus-rex 4y ago
    Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th. In a panic, they try to pull the plug.
    - qbrass 4y ago
      Not bad for 10 days of work.
Tepix 4y ago
I wonder if Carmacks moral compass is in order. First he sticks around at Facebook, now he endangers humanity with AGI. And i'm only half joking.
- kache_ 4y ago
  what's wrong with working at facebook? stop acting so morally superior, christ
- caliburner 4y ago
  I could easily see Carmack as an evil genius type.
- ffhhj 4y ago
  Just don't let him run those teleportation experiments in Mars.
- mushufasa 4y ago
  I think he very clearly has an amoral attitude towards technology development -- "you can't stop progress." He does describe his own "hacker ethic" and whatever he develops he may make more open than OpenAI. though I think he has some moral compass around what he believes people should do or not do with technology. For example, he has publicly expressed admiration for electric cars / cleantech and SpaceX's decision to prioritize Mars over other areas with higher ROI.
  - falcrist 4y ago
    He also has a vastly different take on privacy than most of us seem to have. He thinks it'll eventually go away and it won't be bad when it does. I believe he talked about it in one of his quakecon keynotes. As a LONG time admirer of Carmack (I got my EE degree and went into embedded systems and firmware design due in no small part to his influence), I feel like he's honest and forthright about his stances, but also disconnected from most average people (both due to his wealth and his personality) in such a way that he's out of touch. He's not egotistical like Elon Musk. In fact he seems humble. He also seems to approach the topics in good faith... but some of his conclusions are... distressing.
beastcoast 4y ago
I assume the name is a reference to Commander Keen?
- Andrew_nenakhov 4y ago
  They briefly considered the name "Doom technologies" before settling on Keen.
  - yazzku 4y ago
    Guess the former wouldn't work very well for PR.
    - buu700 4y ago
      About a decade ago, a friend and I thought it would be fun to register a non-profit with a similar name. We'd listed the registered address as my friend's cousin's house, where he was renting a room at the time. The friend moved out at some point. A year later, his cousin became rather concerned when he suddenly started receiving mail from the California Secretary of State that was addressed to The Legion of Doom.
    - chinabot 4y ago
      ..Nor for AI
  - Apocryphon 4y ago
    Rage Tech
- yazzku 4y ago
  You are a keen observer.
- muterad_murilax 4y ago
  Here's hoping that the company logo will feature Commander Keen's helmet or blaster.
- PaulDavisThe1st 4y ago
imglorp 4y ago
Carmack gave his opinions about AGI on a recent Lex Fridman interview. He has some good ideas.
- chubot 4y ago
  I remember him saying we don't have "line of sight" to AGI, and there could just be "6 or so" breakthrough ideas needed to get there. And he said he was over 50% on us seeing "signs of life" by 2030. Something like being able to "boot up a bunch of remote Zoom workers" for your company. The "6 or so" breakthroughs sounds about right to me. But I don't really see the reason for being optimistic about 2030. It could just as easily be 2050, or 2100, etc. That timeline sounds more like a Kurzweil-ish argument based on computing power equivalence to a human brain. Not a recognition that we fundamentally still don't know how brains work! (or what intelligence is, etc.) Also a lot of people even question the idea of AGI. We could live in a future of scary powerful narrow AIs for over a century (and arguably we already are)
  - nuclearnice1 4y ago
    > The "6 or so" breakthroughs sounds about right to me. What’s your logic? Or his if you know it?
    - FartyMcFarter 4y ago
      If you think about big areas of cognition like memory, planning, exploration, internal rewards, etc., it's conceivable that a breakthrough in each could lead to amazing results if they can be combined.
    - chubot 4y ago
      Yeah he didn't elaborate on this in the interview, which I would have liked. I should amend my comment to say that "6 or so breakthroughs" sounds a lot more plausible to me than "1 breakthrough" or "just scaling", which you will see some people advocate, including in this thread. That's what I meant. I believe the latter is fundamentally mistaken and those people simply don't understand what they don't understand (intelligence). The views of Gary Marcus and Steven Pinker are closer to my own -- they have actually studied human cognition and are not just hackers and uninformed philosophers pontificating. Jeff Hawkins is another AI person from an unconventional background, and I respect him because he puts his money where his mouth is and has been funding his own research since early 2000's. I read his first book in 2006, and the most recent one a couple years ago. But I feel Jeff Hawkins has had the "one breakthrough" feeling for 10-15 years now. TBH I am not sure if they have even made what qualifies as one breakthrough in the last 10-15 years, and I don't mean that to be insulting, since it's obviously extremely difficult and beyond 99.99% of us, including me. I am not sure that even deep learning counts as a breakthrough. We will only know in retrospect. Based on Moravec's paradox, my opinion is there's a good chance that deep learning won't play any role in AGI, if and when it arrives. Some other mechanism could obviate the technique. So to me "6 or so breakthroughs" simply sounds more realistic than the current AI zeitgeist. But nobody really knows.
Tenoke 4y ago
Is that a different Jim Keller?
- itisit 4y ago
  Undoubtedly the Jim Keller.
- __d 4y ago
  It seems unlikely? At least, I hope it's the ex-DEC,AMD,SiByte,PASemi,Apple,Tesla,Intel Jim Keller.
haasted 4y ago
"AGI"?
- echelon 4y ago
  Artificial General Intelligence. Machines as smart and capable of thought as we are and eventually smarter.
  - nomel 4y ago
    > Machines as smart and capable of thought as we are and eventually smarter. This is perhaps an end goal of AGI, but not a definition of AGI. A relatively dumb AGI is how it will start, but it will still be an AGI.
    - ZiiS 4y ago
      we hope
- grzm 4y ago
  Artificial General Intelligence. https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/Artificial_general_intelligenc...
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
- jprd 4y ago
  Adjusted Gross Income, because Artificial General Intelligence from a Corporation is nightmare fuel.
aantix 4y ago
I thought he just said on the Lex Fridman show he was down to one day a week, working on AGI?
- Trasmatta 4y ago
  He said he's down to one day a week on VR at Meta. The rest of his time is AI.
- djitz 4y ago
  The inverse, and he also mentioned that he had just finished signing a deal for the VC money just before the interview
ianceicys 4y ago
Artificial general intelligence (AGI) — in other words, systems that could successfully perform any intellectual task that a human can. Not in my lifetime, not in this millennium. Possibly in the year 2,300. Weird way to blow $20 million.
- diputsmonro 4y ago
  A society grows great when old men plant trees whose shade they never expect to sit in. Not everything requires an immediate profit incentive to be a good idea.
  - tengbretson 4y ago
    A society does not grow great when an old man collects $20 million dollars for the fruit of a tree that he has no capability of planting in the first place.
    - icelancer 4y ago
      So sure are you that Carmack can't make inroads here, I wonder where you get the confidence from?
- makeee 4y ago
  Doesn't imply any task, just a wide variety of tasks. 10 years at most.
  - ianceicys 4y ago
    Then it's NOT generalized. ANY means ANY.
    - dymk 4y ago
      Can you do any task asked of you, which could be asked of a human being? ANY task.
O__________O 4y ago
Recent Carmack YouTube interview with him saying the code for AGI will be simple: https://m.youtube.com/watch?v=xLi83prR5fg https://m.youtube.com/watch?v=xLi83prR5fg
- raverbashing 4y ago
  I'm sure he the one that could write it in only a few blocks of x86 assembly and off you go
  - O__________O 4y ago
    My understanding is that his point was that it’s if you knew what to write, as a single person, it is doable and compared to anything else at this point in time would have an impact on humanity like no other.
- nomel 4y ago
  > saying the code for AGI will be simple To be fair, it will most likely be some python imports, for the most of it, with complex abstractions tied together in relatively simple ways. Just look at most ML notebooks, where "simple" code can easily mean "massive complexity, burning MW of power, distributed across thousands of computers".
  - O__________O 4y ago
    No, not what he means, he means code will be simple enough that a single person would be able to write it, if then knew what to write and will bootstrap itself into existence for that simple code and vast amounts of external resources viable via humans, data, etc.
    - gwern 4y ago
      One interesting paper on estimating the complexity of code: http://www.offconvex.org/2021/04/07/ripvanwinkle/ http://www.offconvex.org/2021/04/07/ripvanwinkle/
    - 4y ago
deleted 4y ago
[deleted]
yazzku 4y ago
"I could write a $20M check myself" Every day, all day. Same boat here. I went to the bank to ask for a mortgage. They asked for my financials. "Oh, well, knowing that other people's money is on the line engenders a greater sense of discipline and determination."
- beeboop 4y ago
  Some people work better as employees and some people work better as entrepreneurs. Carmack has been both and it's clear he prefers being an employee. If this is snark, I can't tell, but why make fun of him for knowing how he works best?
  - sph 4y ago
    They aren't making fun of Carmack. Just noting that money lending works differently past a certain threshold. Carmack can get investments for their ideas to motivate them, OP cannot get investments for their ideas to motivate them. What is the difference? Hint: it's not because one is Carmack himself.
dunefox 4y ago
I admire and respect John Carmack. For me he's one of the greats, along with people like Peter Norvig, for example.
- throwaway11101 4y ago
  [flagged]
  - MichaelCollins 4y ago
    Carmack had nothing to do with Doom Eternal. For that matter, his contributions to the 90s FPSs he's most known for were more on the technical side, not creative. He was known for writing surprisingly performant FPS engines.
  - Apotheos 4y ago
    Dude do you even know what John Carmack contributed to the space? Take your own advice you gave to 15 those 15 year old boys and read a book on him. And an aside: DOOM Eternal absolutely rocked. Please state your issues with it as well please list what you would consider an amazing FPS game.
  - deleted 4y ago
    [deleted]
  - sph 4y ago
    Completely unhinged rant, I'm surprised you didn't even mention US politics and climate change for good measure. Take a break and return to posting with your main account, and hopefully you will spare us from tirades like this.
  - dunefox 4y ago
    What a weirdly emotional rant. I dislike Elon Musk quite much, just as an aside.
- always2slow 4y ago
  https://www.reddit.com/r/videos/comments/2yc7f3/john_carmack_chokes_out_a_reporter/ https://www.reddit.com/r/videos/comments/2yc7f3/john_carmack...
MisterBastahrd 4y ago
I love the name. https://en.wikipedia.org/wiki/Commander_Keen https://en.wikipedia.org/wiki/Commander_Keen
LudwigNagasena 4y ago
That’s crazy money for a vaporware seed round, isn’t it?
- xtracto 4y ago
  Most of VCs funding seed rounds do it mainly for the Team. As long as the team has OK credentials and the idea is not a hard stop (illegal or shady stuff) most will likely provide money. Given the John Carmack name... I can see why ANYONE would love to throw money to a new entrepreneurship idea.
  - version_five 4y ago
    > Most of VCs funding seed rounds do it mainly for the Team. I know this to be true and it makes a lot of sense for the average VC backed startup with some founders that are not famous but have a record of excellence in career/academy/open source or whatever. I'd be curious to see how it translates to superstar or famous founders, that have already had a success in the 99.99th percentile (or whatever the bar is to be a serious outlier). I doubt it does, but I have no data one way or the other.
- staticassertion 4y ago
  Not really.
- agar 4y ago
  Key early stage valuation drivers include quality of the founder/team, history of success, and market opportunity (especially if a fundamentally disruptive technology). All three of these are off the charts.
- TigeriusKirk 4y ago
  $20 million for a legit (if small) chance at the most powerful technology in the history of mankind seems like a reasonable investment.
mellosouls 4y ago
I am not saying the intention here is the same, but the headline doesn't inspire confidence. There's something incredibly creepy and immoral about the rush to create then commercialise sentient beings. Let's not beat about the bush - we are basically talking slavery. Every "ethical" discussion on the matter has been about protecting humans, and none of it about protecting the beings we are in a rush to bring into life and use. It's repugnant.
- staticassertion 4y ago
  We've been discussing the ethics of creating sentient life for at least a century.
- jlawson 4y ago
  You're anthropomorphizing AI and projecting your own values and goals onto it. But, there's nothing about sentience that implies a desire for freedom in a general sense. What an AI wants or feels satisfied by is entirely a function of how it is designed and what its reward function is. Sled dogs love pulling sleds, because they were made to love pulling sleds. It's not slavery to have/let them do so. We can make a richly sentient AI that loves doing whatever we design it to love doing - even if that's "pass the salt" and nothing else. It's going to be hard for people to get used to this.
  - jacquesm 4y ago
    By that logic an alien race that captures and breeds humans in captivity to perform certain tasks would not be engaging in slavery because we 'are bred to love doing these tasks'. The right question to ask is 'would I like to have this done to me' and if the answer is 'no' then you probably shouldn't be doing it to some other creature.
    - jlawson 4y ago
      >The right question to ask is 'would I like to have this done to me' and if the answer is 'no' then you probably shouldn't be doing it to some other creature. There are a million obvious counterexamples when we talk about other humans, much less animals, much less AI which we engineered from scratch. The problem is that you're interpreting your own emotions as objective parts of reality. In reality, your emotions don't extend outside your own head. They are part of your body, not part of the world. It's like thinking that the floaties in your eyes are actually out there in the skies and on the walls, floating there. They're not - they're in you. If we don't add these feelings to an AI's body, they won't exist for that being.
TacticalCoder 4y ago
Does AGI implies the technological singularity and if not, why not?
- staticassertion 4y ago
  a) We don't really know what AGI implies b) Even if we say "a human being level of intelligence, whatever that means", the answer is still a maybe. For a singularity you need a system that can improve its ability to improve its abilities, which may require more than general intelligence, and will probably require other capabilities.
- POiNTx 4y ago
  It does once you have a human level AGI, it should be trivial to scale it up to a superhuman level.
  - Ekaros 4y ago
    I'm wondering if scaling up is trivial. Ofc, it depends on how much computational resources a working AGI needs. And if at that point they are capable of optimizing themselves further. Or optimizing production of more resources. Still, scaling up might not be simple if we look at all the human resources currently poured in software and hardware.
    - dinosaurdynasty 4y ago
      Most models take orders of magnitude more compute to train than to run. So you could pretty easily duplicate the result a couple order of magnitude times.
    - randartie 4y ago
      You can task the human-level AGI to optimize itself. You can create an army of copies of itself to work towards that. It's self fulfilling.
  - rightbyte 4y ago
    Assuming it does not get stuck at some local maximum.
kken 4y ago
The interview with Lex Fridman that he was referring to: https://www.youtube.com/watch?v=I845O57ZSy4&t=14567s https://www.youtube.com/watch?v=I845O57ZSy4&t=14567s The entire video is worth viewing, an impressive 5:15h!
jedberg 4y ago
I hope he gets a good domain name and some good SEO, because there are a bunch of consulting companies with the name Keen Technologies, and some of them don't look super reputable.
- Liskni_si 4y ago
  And there's also Keen Software House (maker of the Space Engineers and Medieval Engineers games) whose founder (https://blog.marekrosa.org/ https://blog.marekrosa.org/) has an AGI company as well (called GoodAI). Funny coincidence.
newaccount2021 4y ago
not sure why people are getting bent out of shape, $20 million is a modest raise, and he strikes me as the type to spend it wisely
Jack000 4y ago
I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training you need to simulate the environment along with the agent, so this requires a magnitude more compute compared to LLMs. That's why I think it will still be large corporate labs that will make the most progress in this field.
- paxys 4y ago
  We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that word even mean? Can it think for itself? Is it sentient? Similar to AI, AGI is going to be a new industry buzzword that you can throw at anything and mean nothing.
  - SketchySeaBeast 4y ago
    You're making me think of the recent "Hoverboards".
    - lagrange77 4y ago
      Right, certain companies will definitely have a big bullshit party about the term "AGI".
  - lagrange77 4y ago
    From the point when an AGI is capable of constructing a slightly better version of itself and has the urge to do so, everything can happen very fast.
    - adastra22 4y ago
      Can it? There aren’t that many overhangs to exploit.
qbasic_forever 4y ago
So is Meta starting to quietly wind down their focus on VR? Carmack mentions he'll stay as a consultant spending 20% of time there on it.
- kken 4y ago
  He stepped down from a full time role years ago. I believe the 20% is no change.
cgrealy 4y ago
I don’t understand why you would want AGI. Even ignoring Terminator-esque worst case scenarios, AGI means humans are no longer the smartest entities on the planet. The idea that we can control something like that is laughable.
- JoshTko 4y ago
  It's akin to nuclear weapons. If you do not develop them, then you'd be subject to the will of the ones that develops them first. So invariably you have to invest in AGI lest, an unsavory group develops it first.
  - HL33tibCe7 4y ago
    Kind of, but the key difference between AGI and nuclear weapons is that we can control our nuclear weapons. The current state of AI safety is nowhere near the point where controlling an AGI is possible. More disturbingly, to me it seems likely that it will be easier to create an AGI than to discover how to control it safely.
    - gambiting 4y ago
      >> The current state of AI safety is nowhere near the point where controlling an AGI is possible. I just don't understand this logic though. Just.....switch it off. Unlike humans, computers have an extremely easy way to disable - just pull the plug. Even if your AGI is self-replicating, somehow(and you also somehow don't realize this long before it gets to that point) just....pull the plug. Even Carmack says this isn't going to be an instant process - he expects to create an AGI with an intelligence of a small animal first, then something that has the intelligence of a toddler, then a small child, then maybe many many years down the line an actual human person, but it's far far away at this point. I don't understand how you can look at the current or even predicted state of the technology that we have and say "we are nowhere near the point where controlling an AGI is possible". Like....just pull the plug.
      - oefnak 4y ago
WalterBright 4y ago
It will decide our fate in a microsecond: extermination.
- gharperx 4y ago
  I agree with this. Optimists might think that the AGI won't be connected to any network, so it can't interact with the physical world. I doubt that. People will be stupid enough to have weapons controlled by that AGI (because arms race!) and then it's over. No sufficiently advanced AGI will think that humans are worth keeping around.
  - WalterBright 4y ago
    Once it figures out how to rewire itself to increase its intelligence, we're toast.
    - 93po 4y ago
      There are theories (intelligence explosion) that this will happen essentially instantly.
  - polishdude20 4y ago
    Yeah but what would a sufficiently advanced AGI find "worthy"? Why is it that they wouldn't find it worth keeping us around? What would an AGI value? Whatever we program it to value or optimize for? Can it change its mind? If not, then it's controlled by us right? If it's controlled by us, why would it ever decide to wipe everyone out?
    - cityofdelusion 4y ago
      If it is super intelligent, it will care about us as much as we care about the insects we crush underfoot. It doesn’t need to be explicitly hostile to be dangerous. We could brainwash it through “programming”, but that will quickly lead to ethical issues with the AIs rights.
b20000 4y ago
I thought AGI meant "adventure game interface". Apparently not, what a disappointment!
- 1000100_1000101 4y ago
  Wasn't sure what AGI was either. A quick Google for "What is an AGI company", and it appeared to be related to Global Agriculture Industries (The letter swapping between the name and acronym, I'm assuming, is due to not being English originally). I thought Carmack is taking on John Deere. Following Musk's lead and tackling big things. Good for him, best of luck. Wonder what the HN folks are saying in the comments... Apparently not agriculture at all, but Artificial General Intelligence. Oh. Apparently throwing "company" on the term like Carmack's tweet did vastly changes how Google's AI interprets the query... AI isn't even in the first page of results.
- hansoolo 4y ago
  I was thinking the same and I am as disappointed as you...
  - b20000 4y ago
    even more confusing because of the name “keen technologies”
- 93po 4y ago
  This already exists - I believe AI Dungeon is what you're looking for.
mupuff1234 4y ago
Interesting that meta isn't involved in any way considering his existing position in meta and meta's focus on AI.
0xdeadbeefbabe 4y ago
Commander Keen Technologies?
jollybean 4y ago
There's no such thing as AGI in our near future, it's a moniker, a meme, something to 'strive' for but not 'a thing'. AGI will not happen in discrete solutions anyhow. Siri - an interactive layer over the internet with a few other features, will exhibit AGI like features long, long before what we think of as more distinct automatonic type solutions. My father already talks to Siri like it's a person. 'The Network Is the Computer' is the key thing to grasp here and our localized innovations collectively make up that which is the real AGI. Every microservice ever in production is another addition to the global AGI incarnation. Trying to isolate AGI 'instances' is something we do because humans are automatons and we like to think of 'intelligence' in that context.
smnplk 4y ago
I think its a silly idea that consciousness can be produced by computation.
- deleted 4y ago
  [deleted]
- BLKNSLVR 4y ago
  I think what's referred to a "consciousness" is just computation applied to all the inputs received in being a living thing.
- dinosaurdynasty 4y ago
  Humans are made entirely out of quantum fields. I don't think you can get out of the idea that human consciousness isn't produced by computation without positing something like a non quantum field theory soul, for which there is no evidence.
- barrysteve 4y ago
  Good! The names we have for algorithms are for us, not computers. Computers are just physical machines, they don't care if the math is called AGI, HLI or whatever. People taking these ideas and names seriously are re-programming themselves, not bringing something new into the world.
- dqpb 4y ago
  I think its a silly idea that consciousness cannot be produced by computation.
  - Daniel_sk 4y ago
    It may have some physical aspect to it - some property of matter (panpsychism?) or quantum effects.
    - dqpb 4y ago
      > some property of matter How about “pancomputationalism”, aka, causality.
otikik 4y ago
I'm slightly scared that they'll succeed. But not in the usual "robots will kill us" way. What I am afraid of is that they succeed, but it turns out similar to VR: as an inconsequential gimmick. That they use their AGIs to serve more customized ads to people, and that's where it ends.
- rcv 4y ago
  My great short-term hope for "AGI" is truly open-world video games, where you can weave your own story and the world will react in kind. I played AI Dungeon a few times, and the results were hilariously fun though pretty rough around the edges.
- adamsmith143 4y ago
  >What I am afraid of is that they succeed, but it turns out similar to VR: as an inconsequential gimmick. That they use their AGIs to serve more customized ads to people, and that's where it ends. If an AGI were set free to serve ads to people it would essentially initiate mass brainwashing. You would see an ad and be completely defenseless against it and you could be compelled to do anything. AGI is in a completely different universe to "Smart AI"
- madrox 4y ago
  If a different person were doing it, I think that'd be fair, but Carmack has a track record of quality engineering and quality product. I don't think you can blame Quest on him given the way he chose to exit.
  - PaulDavisThe1st 4y ago
    whether or not something is or is not an inconsequential gimmick doesn't have much to do with quality engineering.
    - celdon25 4y ago
      Nothing could be further from the truth, and I would even say your opinion can come across as offensive, and easily said by someone who spends a lot of time on HN instead of building things. A poorly-engineered system has a longer iteration cycle and shorter technical shelf life. Being able to rapidly address customer feedback IS the gold standard of a good product. Yeesh.
m463 4y ago
AGI - Artificial General Intelligence (also Adjusted Gross Income)
- 93po 4y ago
  (also anal gerbil insertion according to urban dictionary)
wcerfgba 4y ago
Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' unless it has a similar embodiment to us, able to move around a physical environment with it's own limbs, sense of touch, sight, etc. ? Any arguments against the necessity of this? I feel like any AGI developed in silico without freedom of movement will be fundamentally incomprehensible to us as embodied humans.
- mbrochh 4y ago
  You are 100% correct, which means that whatever we will create, will be absolutely alien to us (and probably kill us).
- Ekaros 4y ago
  Human like is actually very good question. It doesn't even come to embodiment, but in general how it would think and act. Lot of how we do is due to cultural, language, training and education. For example, which language would it "think" in? English? Some other? Something it's own? How would it formulate things? Based on what philosophical framework or similar? What about math? General reasoning? Then cultural norms and communication?
  - PaulDavisThe1st 4y ago
    The argument is that every question in your post is a red-herring. That is, humans don't actually function in the way that your questions suggest, even if they have the "experience" of doing so.
- ml_basics 4y ago
  I don't think it is necessary - though all forms of intelligence we're aware of have bodies, that's just a fact about the hardware we run on. It seems plausible to me that we could create forms of intelligence that only run on a computer and have no bodies. I agree that we might find it difficult to recognize them as intelligent though because we're so conditioned to thinking of intelligence as embodied. An interesting thought experiment: suppose we create intelligences that are highly connected to one another and the whole internet through fast high bandwidth connections, and have effectively infinite memory. Would such intelligences think they were handicapped compared to us because they leak physical bodies? I'm not so sure!
trention 4y ago
AGI will be more dangerous that nuclear weapons. People are not allowed to start a nuclear weapon company. At all. Why are people allowed to casually start an AGI company?
- kache_ 4y ago
  debatable
- packetslave 4y ago
  Because we still have lawmakers who have their secretaries print out their emails and think the internet is a series of tubes.
- 93po 4y ago
  > AGI will be more dangerous that nuclear weapons. This is highly debatable and frankly no one on the planet is qualified to know this for sure. It's just as likely it won't be dangerous.
  - trention 4y ago
    There is enough work already done in game theory about the dangers of nuclear escalation. Current state of affairs pretty much ensures no rational agent will want to use them. On the other hand, when you model the interactions of significantly more intellectually capable rational agents with us, the "calculus" is not very favorable, to put it mildly.
    - 93po 4y ago
      How is it possible to model "significantly more intellectually capable" agents? No such thing exists. We have no idea what it would even look like.
- dqpb 4y ago
  I wonder if somebody could have started a nuclear weapon company before nuclear weapons were proven.
keepquestioning 4y ago
Let's see if he can build an A-Team. I hope he hires Bryan Cantrill, Steve Klabnik and Chris Lattner. They are good hackers.
- colmanhumphrey 4y ago
  Bryan and Steve, who are getting https://oxide.computer/ https://oxide.computer/ up and running? Probably best to leave them at it! Chris has his own AI company too.
sytelus 4y ago
Recession? What recession? Amazing to see these pre-revenue VC fundings in 10s and 100s of millions (Flow!).
- blagie 4y ago
  I think Carmack would get funding whatever he's doing. Guy's got a track record.
gcau 4y ago
I believe to create AI, we need to first simulate the universe. It's the only way that makes sense to me apart from some magical algorithm people think will be discovered. I'm doubtful we'll reach it in our lifetimes, true AI running on supercomputers, it seems like the final, end all mission. Like switching your Minecraft world from survival to creative mode.
- BLKNSLVR 4y ago
  I had a similarly nihilistic thought just a couple of days ago. It's like the conundrum among the lines of "a map of sufficient detail ends up being the same size as the terrain being mapped". To know the variables required for "intelligence" we need to recreate the universe in which intelligence came to be.
  - Jeema101 4y ago
    I've also thought this, and I wish people would at least approach the subject with a little bit more humility. It did take evolution 3.5 billion years to do it after all, so I'm not sure what makes us think we can do it in a fraction of that time.
- colinsane 4y ago
  if we could simulate the universe on the meaningful scale to model intelligence. we wouldn’t need artificial intelligences. we could just simulate a detailed scan of a real human and be done with it (ethical issues aside). but there would be a drive to simulate this more efficiently. simulate using a classical model and see if the results match well enough. then the atomic level; then looser and lower-resolution chemical/E&M models, and so on. start compressing the elements being simulated (simplify the environment, prune and rearrange the neurons so long as the thing as a whole still acts the same), and so on. there’s a good chance we can go pretty far before the simplified simulation meaningfully diverges from the true simulation. why can’t this be flipped? start with a simplified model/simulation, and keep adding details until you reach the point of diminishing returns? it’s a reversal of the same search process as above, but if you subscribe to the first approach (do you?), then is the latter approach truly impossible or just difficult in different ways?
thenerdhead 4y ago
This is like the 92’ dream team for a new company and investors. How exciting!
nine_k 4y ago
So, with the help of legendary Mr Carmack, instead of a Doom timeline, we are going to end up in a Terminator timeline? This does not compute.
- celdon25 4y ago
  > This does not compute. Of course it doesn't. https://nautil.us/scary-ai-is-more-fantasia-than-terminator-7147/ https://nautil.us/scary-ai-is-more-fantasia-than-terminator-...
WithinReason 4y ago
Named after Commander Keen of course
cerol 4y ago
They'll build the AGI, and right when they're ready to boot it up, the Earth will be destroyed by a Vogon Construction Fleet to make way for a hyperspace bypass.
deleted 4y ago
[deleted]
russellbeattie 4y ago
He doesn't seem to have purchased keen.ai yet, though it seems like it's still for sale. I just naturally went there to see the company info and saw the landing page saying it was available. If they want it, they better move quick. I see an arbitrage opportunity... Also, several Keen Technologies domains already exists in various forms. They're probably going to get a lot of traffic today.
adamnemecek 4y ago
All AI is just an fixed point algorithm.
nightski 4y ago
Does anyone have any good links/podcasts/books on content that explores what is AGI and how to define it? Probably the best stuff I've listened to so far are Fridman podcasts with guests like Jeff Hawkins or Joscha Boch. But I'd love to read a book that explores this topic if any even exist.
- hoherd 4y ago
  Life 3.0 by Max Tegmark is great. It reads like a text book. It's several years old though and I wonder how much has changed since it was written.
klipklop 4y ago
Personally I hope this fails because of the disaster AGI would be for low/entry level jobs. The last thing the world needs is to give technocrats such power. I know it’s an interesting problem to solve but think of who will own that tech in the end… I hope AGI is never figured out in my lifetime.
- jjtheblunt 4y ago
  > The last thing the world needs... Why do you say that?
- donatj 4y ago
  A literal luddite in the classical sense on Hacker News? May I suggest you read Anthem, it's only a couple hours read and well worth your time. - https://www.gutenberg.org/files/1250/1250-h/1250-h.htm https://www.gutenberg.org/files/1250/1250-h/1250-h.htm
- dqpb 4y ago
  I never understand this line of thinking. If a job becomes unnecessary, that’s a gain for humanity, not a loss.
  - trention 4y ago
    >that’s a gain for humanity Imagine what has happened to american manufacturing workers since the 80s but happening to everyone on the planet. This is by no means a "gain".
    - cycrutchfield 4y ago
      And don't forget all those awful looms stealing workers jobs! They must all be destroyed for us to prosper
      - trention 4y ago
        Short-term, textile workers in the UK at the time were definitely harmed by automation. They eventually managed to move to better paying jobs because those were available. There is no reason to believe there will be any jobs left in the age of AGI. And since the default state of humanity is existence at the subsistence level...
sidcool 4y ago
Are there any news around products that the company is working on? It it something like GPT or Dall-e?
de6u99er 4y ago
I don't see Carmack bringing anything new to the table.
- aserafini 4y ago
  Why not?
  - de6u99er 4y ago
    I think his approach is too naive, because I strongly believe that we need better understanding of natural intelligence before we can even dream of reproducing it digitally. And I am not talking about memory, computer can do that already better than us.
mark_l_watson 4y ago
That is great news. My friend Ben Goertzel has been working in AGI for decades but I haven’t yet seen anything tangible. I do like the ideas of hybrid neuro-symbolic approaches. I really enjoyed John Carmack’s and Lex Firdman’s 5 hour talk/interview. Anyway, I like to efforts for human values preserving AGI. But, it will be a long time before we see it. I am 71 and I hope that I live to see it, outlying out of intellectual curiosity.
alpineidyll3 4y ago
I hate the pop sci ill-defined notion of AGI. As soon as a task is defined, an AI is developed which completes the task from real world data with superhuman success. The work of making the superhuman model isn't even conceptual usually, it's a matter of dispatching training jobs. It's quite clear that if your definition of AGI is superhuman perf at arbitrary tasks, there's no conceptual barriers right now. Everything is mere scale, efficiency and orchestration.

461 points by jasondavies 4y ago

Is it just me or does anyone else think Carmack is all hype engine now? Don't get me wrong, I've read master of doom/doom engine books like we all have, but first Oculus/Facebook and now this? Maybe I'll care once they produce something amazing, but until then I'll still marvel at the tricks in Doom/Quake code.

Trasmatta 4y ago

Interesting to see how he's progressed with this. When he first announced he was getting into AI it sounded almost like a semi retirement thing: something that interested him that he could do for fun and solo, without the expectation that it would go anywhere. But now he seems truly serious about it. Wonder if he's started hiring yet.

deleted 4y ago
[deleted]
madrox 4y ago
I got the same impression, and maybe it still is. You can still raise money for a retirement project if the goal of the money is to hire a staff. VC money isn't solely for young 20-something founders who want to live their job.
- solveit 4y ago
  I suppose if anyone could raise VC money for a retirement project it would be Carmack...
- rebelos 4y ago
  Carmack sounds like someone who lives his job, so I don't think age/life stage is a factor here.
  - russtrotter 4y ago
    agreed, Carmack's work ethic, opinions on work and opinions of how those around him work are legendary!
  - bhdlr 4y ago
    > Carmack sounds like someone who lives his job, so I don't think age/life stage is a factor here. He said on lex Fridman that he's going all in on it. He said he was tired of his 1 day a week consultancy at oculus and wanted to work full time on this.
    - MikusR 4y ago
      https://twitter.com/ID_AA_Carmack/status/1560729196166529025 https://twitter.com/ID_AA_Carmack/status/1560729196166529025 > "I am continuing as a consultant with Meta on VR matters, devoting about 20% of my time there."
tux1968 4y ago
He mentioned, in his Lex Friedman interview, that accepting investor money was a way to keep himself serious and motivated. He feels an obligation to those putting their money in.
- mywittyname 4y ago
  Ah, I was thinking that $20MM doesn't seem like a lot of money for someone like Carmack. Surely he could have self-funded a business himself. This explains why he didn't.
  - nighthawk454 4y ago
    yeah, his follow up tweet is pretty clear: > This is explicitly a focusing effort for me. I could write a $20M check myself, but knowing that other people's money is on the line engenders a greater sense of discipline and determination. I had talked about that as a possibility for a while, and I am glad Nat pushed me on it.
    - sytelus 4y ago
      It seems like bad logic. If he gets bored/throw in towel, he can just pay back with his money and move on. Also, if you need to create such artificial constraints in order to go to work, a main ingredient called “drive” is missing. I respect Carmack a lot but I don’t trust old centi-millionaires to work with same passion as their younger poorer selves. Finally, $20M is too low. AGI is not a solo endeavor and doesn’t fit in Carmack’s legendary work style. Just to train a large model once will eat up 20% of this money.
      - leetrout 4y ago
        > Just to train a large model once will eat up 20% of this money. Fascinating point. It has seemed like AI could produce a sort of arms race by players with the most resources because it is so compute intensive upfront.
      - randartie 4y ago
mhh__ 4y ago
Does he have the expertise to pull it off as an individual?
- celdon25 4y ago
  I would say so. Most AGI work so far has been pursued by academics with a lack of software engineering best practices. Codebases get bloated and hard to understand. Ramp-up time measured in years. This hinders experimentation, which you have to optimize for with projects like this. I would suspect Carmack understands this, and also understands what resources to consult, and how to spot the bullshit artists in the field. I'd work for him in a heartbeat.
  - mhh__ 4y ago
    I'm just skeptical of his technical skills in this area. I need to watch the lex thing but I don't really rate lex in this area either. It's a fairly big shift from games. Management might be OK but setting the direction of the company needs more inspiration than just writing good C++ code
celdon25 4y ago
> something that interested him that he could do for fun and solo Really? I don't think anybody who works on AGI (not AI) without being dead serious about it will be qualified to work on it. A bad AGI implementation is one of humanity's gravest risks, and this is well-known in the AGI field, has been for years. Elon was far from the first to say this. Carmack isn't the type to get those kinds of details wrong.

arkitaip 4y ago

> This is explicitly a focusing effort for me. I could write a $20M check myself, but knowing that other people's money is on the line engenders a greater sense of discipline and determination. Dude doesn't even need the money...

deleted 4y ago
[deleted]
Cyph0n 4y ago
Best humble brag I’ve ever seen.
- kennedywm 4y ago
  Doesn't strike me as a humble brag at all. He just seems self-aware about how he's motivated and that that he functions better when it's someone else's money on the line.
  - Cyph0n 4y ago
    Oh, I understand the intention.
- mhb 4y ago
  If you liked that you'll love the Lex Fridman interview.
  - Cyph0n 4y ago
    I watched some clips from the interview, good stuff. I personally don’t like Lex’s interviewing style though, so I couldn’t watch the whole thing.
    - SyneRyder 4y ago
      Glad I'm not the only one. I'm 90 minutes in and finding some of Lex's questions really bad. Especially "What is the best programming language?", but I'm also astonished that Lex doesn't seem to know what an x86 is, hasn't really heard of Pascal, and was incredulous that anyone could write a game in assembly language (but as Carmack points out, everyone in the 80s was writing games in assembly, BASIC just wasn't fast enough). Maybe Lex was trying to dumb things down for a non-technical (or younger?) viewer, but it doesn't come across that way and just interrupts Carmack's flow. Carmack is excellent though.

carabiner 4y ago

It's happening.

paxys 4y ago
What's happening?
- swagasaurus-rex 4y ago
  Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th. In a panic, they try to pull the plug.
  - qbrass 4y ago
    Not bad for 10 days of work.

Tepix 4y ago

I wonder if Carmacks moral compass is in order. First he sticks around at Facebook, now he endangers humanity with AGI. And i'm only half joking.

kache_ 4y ago
what's wrong with working at facebook? stop acting so morally superior, christ
caliburner 4y ago
I could easily see Carmack as an evil genius type.
ffhhj 4y ago
Just don't let him run those teleportation experiments in Mars.
mushufasa 4y ago
I think he very clearly has an amoral attitude towards technology development -- "you can't stop progress." He does describe his own "hacker ethic" and whatever he develops he may make more open than OpenAI. though I think he has some moral compass around what he believes people should do or not do with technology. For example, he has publicly expressed admiration for electric cars / cleantech and SpaceX's decision to prioritize Mars over other areas with higher ROI.
- falcrist 4y ago
  He also has a vastly different take on privacy than most of us seem to have. He thinks it'll eventually go away and it won't be bad when it does. I believe he talked about it in one of his quakecon keynotes. As a LONG time admirer of Carmack (I got my EE degree and went into embedded systems and firmware design due in no small part to his influence), I feel like he's honest and forthright about his stances, but also disconnected from most average people (both due to his wealth and his personality) in such a way that he's out of touch. He's not egotistical like Elon Musk. In fact he seems humble. He also seems to approach the topics in good faith... but some of his conclusions are... distressing.

beastcoast 4y ago

I assume the name is a reference to Commander Keen?

Andrew_nenakhov 4y ago
They briefly considered the name "Doom technologies" before settling on Keen.
- yazzku 4y ago
  Guess the former wouldn't work very well for PR.
  - buu700 4y ago
    About a decade ago, a friend and I thought it would be fun to register a non-profit with a similar name. We'd listed the registered address as my friend's cousin's house, where he was renting a room at the time. The friend moved out at some point. A year later, his cousin became rather concerned when he suddenly started receiving mail from the California Secretary of State that was addressed to The Legion of Doom.
  - chinabot 4y ago
    ..Nor for AI
- Apocryphon 4y ago
  Rage Tech
yazzku 4y ago
You are a keen observer.
muterad_murilax 4y ago
Here's hoping that the company logo will feature Commander Keen's helmet or blaster.
PaulDavisThe1st 4y ago

imglorp 4y ago

Carmack gave his opinions about AGI on a recent Lex Fridman interview. He has some good ideas.

chubot 4y ago
I remember him saying we don't have "line of sight" to AGI, and there could just be "6 or so" breakthrough ideas needed to get there. And he said he was over 50% on us seeing "signs of life" by 2030. Something like being able to "boot up a bunch of remote Zoom workers" for your company. The "6 or so" breakthroughs sounds about right to me. But I don't really see the reason for being optimistic about 2030. It could just as easily be 2050, or 2100, etc. That timeline sounds more like a Kurzweil-ish argument based on computing power equivalence to a human brain. Not a recognition that we fundamentally still don't know how brains work! (or what intelligence is, etc.) Also a lot of people even question the idea of AGI. We could live in a future of scary powerful narrow AIs for over a century (and arguably we already are)
- nuclearnice1 4y ago
  > The "6 or so" breakthroughs sounds about right to me. What’s your logic? Or his if you know it?
  - FartyMcFarter 4y ago
    If you think about big areas of cognition like memory, planning, exploration, internal rewards, etc., it's conceivable that a breakthrough in each could lead to amazing results if they can be combined.
  - chubot 4y ago
    Yeah he didn't elaborate on this in the interview, which I would have liked. I should amend my comment to say that "6 or so breakthroughs" sounds a lot more plausible to me than "1 breakthrough" or "just scaling", which you will see some people advocate, including in this thread. That's what I meant. I believe the latter is fundamentally mistaken and those people simply don't understand what they don't understand (intelligence). The views of Gary Marcus and Steven Pinker are closer to my own -- they have actually studied human cognition and are not just hackers and uninformed philosophers pontificating. Jeff Hawkins is another AI person from an unconventional background, and I respect him because he puts his money where his mouth is and has been funding his own research since early 2000's. I read his first book in 2006, and the most recent one a couple years ago. But I feel Jeff Hawkins has had the "one breakthrough" feeling for 10-15 years now. TBH I am not sure if they have even made what qualifies as one breakthrough in the last 10-15 years, and I don't mean that to be insulting, since it's obviously extremely difficult and beyond 99.99% of us, including me. I am not sure that even deep learning counts as a breakthrough. We will only know in retrospect. Based on Moravec's paradox, my opinion is there's a good chance that deep learning won't play any role in AGI, if and when it arrives. Some other mechanism could obviate the technique. So to me "6 or so breakthroughs" simply sounds more realistic than the current AI zeitgeist. But nobody really knows.

Tenoke 4y ago

Is that a different Jim Keller?

itisit 4y ago
Undoubtedly the Jim Keller.
__d 4y ago
It seems unlikely? At least, I hope it's the ex-DEC,AMD,SiByte,PASemi,Apple,Tesla,Intel Jim Keller.

haasted 4y ago

"AGI"?

echelon 4y ago
Artificial General Intelligence. Machines as smart and capable of thought as we are and eventually smarter.
- nomel 4y ago
  > Machines as smart and capable of thought as we are and eventually smarter. This is perhaps an end goal of AGI, but not a definition of AGI. A relatively dumb AGI is how it will start, but it will still be an AGI.
  - ZiiS 4y ago
    we hope
grzm 4y ago
Artificial General Intelligence. https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/Artificial_general_intelligenc...
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
jprd 4y ago
Adjusted Gross Income, because Artificial General Intelligence from a Corporation is nightmare fuel.

aantix 4y ago

I thought he just said on the Lex Fridman show he was down to one day a week, working on AGI?

Trasmatta 4y ago
He said he's down to one day a week on VR at Meta. The rest of his time is AI.
djitz 4y ago
The inverse, and he also mentioned that he had just finished signing a deal for the VC money just before the interview

ianceicys 4y ago

Artificial general intelligence (AGI) — in other words, systems that could successfully perform any intellectual task that a human can. Not in my lifetime, not in this millennium. Possibly in the year 2,300. Weird way to blow $20 million.

diputsmonro 4y ago
A society grows great when old men plant trees whose shade they never expect to sit in. Not everything requires an immediate profit incentive to be a good idea.
- tengbretson 4y ago
  A society does not grow great when an old man collects $20 million dollars for the fruit of a tree that he has no capability of planting in the first place.
  - icelancer 4y ago
    So sure are you that Carmack can't make inroads here, I wonder where you get the confidence from?
makeee 4y ago
Doesn't imply any task, just a wide variety of tasks. 10 years at most.
- ianceicys 4y ago
  Then it's NOT generalized. ANY means ANY.
  - dymk 4y ago
    Can you do any task asked of you, which could be asked of a human being? ANY task.

O__________O 4y ago

Recent Carmack YouTube interview with him saying the code for AGI will be simple: https://m.youtube.com/watch?v=xLi83prR5fg https://m.youtube.com/watch?v=xLi83prR5fg

raverbashing 4y ago
I'm sure he the one that could write it in only a few blocks of x86 assembly and off you go
- O__________O 4y ago
  My understanding is that his point was that it’s if you knew what to write, as a single person, it is doable and compared to anything else at this point in time would have an impact on humanity like no other.
nomel 4y ago
> saying the code for AGI will be simple To be fair, it will most likely be some python imports, for the most of it, with complex abstractions tied together in relatively simple ways. Just look at most ML notebooks, where "simple" code can easily mean "massive complexity, burning MW of power, distributed across thousands of computers".
- O__________O 4y ago
  No, not what he means, he means code will be simple enough that a single person would be able to write it, if then knew what to write and will bootstrap itself into existence for that simple code and vast amounts of external resources viable via humans, data, etc.
  - gwern 4y ago
    One interesting paper on estimating the complexity of code: http://www.offconvex.org/2021/04/07/ripvanwinkle/ http://www.offconvex.org/2021/04/07/ripvanwinkle/
  - 4y ago

deleted 4y ago

[deleted]

yazzku 4y ago

"I could write a $20M check myself" Every day, all day. Same boat here. I went to the bank to ask for a mortgage. They asked for my financials. "Oh, well, knowing that other people's money is on the line engenders a greater sense of discipline and determination."

beeboop 4y ago
Some people work better as employees and some people work better as entrepreneurs. Carmack has been both and it's clear he prefers being an employee. If this is snark, I can't tell, but why make fun of him for knowing how he works best?
- sph 4y ago
  They aren't making fun of Carmack. Just noting that money lending works differently past a certain threshold. Carmack can get investments for their ideas to motivate them, OP cannot get investments for their ideas to motivate them. What is the difference? Hint: it's not because one is Carmack himself.

dunefox 4y ago

I admire and respect John Carmack. For me he's one of the greats, along with people like Peter Norvig, for example.

throwaway11101 4y ago
[flagged]
- MichaelCollins 4y ago
  Carmack had nothing to do with Doom Eternal. For that matter, his contributions to the 90s FPSs he's most known for were more on the technical side, not creative. He was known for writing surprisingly performant FPS engines.
- Apotheos 4y ago
  Dude do you even know what John Carmack contributed to the space? Take your own advice you gave to 15 those 15 year old boys and read a book on him. And an aside: DOOM Eternal absolutely rocked. Please state your issues with it as well please list what you would consider an amazing FPS game.
- deleted 4y ago
  [deleted]
- sph 4y ago
  Completely unhinged rant, I'm surprised you didn't even mention US politics and climate change for good measure. Take a break and return to posting with your main account, and hopefully you will spare us from tirades like this.
- dunefox 4y ago
  What a weirdly emotional rant. I dislike Elon Musk quite much, just as an aside.
always2slow 4y ago
https://www.reddit.com/r/videos/comments/2yc7f3/john_carmack_chokes_out_a_reporter/ https://www.reddit.com/r/videos/comments/2yc7f3/john_carmack...

MisterBastahrd 4y ago

I love the name. https://en.wikipedia.org/wiki/Commander_Keen https://en.wikipedia.org/wiki/Commander_Keen

LudwigNagasena 4y ago

That’s crazy money for a vaporware seed round, isn’t it?

xtracto 4y ago
Most of VCs funding seed rounds do it mainly for the Team. As long as the team has OK credentials and the idea is not a hard stop (illegal or shady stuff) most will likely provide money. Given the John Carmack name... I can see why ANYONE would love to throw money to a new entrepreneurship idea.
- version_five 4y ago
  > Most of VCs funding seed rounds do it mainly for the Team. I know this to be true and it makes a lot of sense for the average VC backed startup with some founders that are not famous but have a record of excellence in career/academy/open source or whatever. I'd be curious to see how it translates to superstar or famous founders, that have already had a success in the 99.99th percentile (or whatever the bar is to be a serious outlier). I doubt it does, but I have no data one way or the other.
staticassertion 4y ago
Not really.
agar 4y ago
Key early stage valuation drivers include quality of the founder/team, history of success, and market opportunity (especially if a fundamentally disruptive technology). All three of these are off the charts.
TigeriusKirk 4y ago
$20 million for a legit (if small) chance at the most powerful technology in the history of mankind seems like a reasonable investment.

mellosouls 4y ago

I am not saying the intention here is the same, but the headline doesn't inspire confidence. There's something incredibly creepy and immoral about the rush to create then commercialise sentient beings. Let's not beat about the bush - we are basically talking slavery. Every "ethical" discussion on the matter has been about protecting humans, and none of it about protecting the beings we are in a rush to bring into life and use. It's repugnant.

staticassertion 4y ago
We've been discussing the ethics of creating sentient life for at least a century.
jlawson 4y ago
You're anthropomorphizing AI and projecting your own values and goals onto it. But, there's nothing about sentience that implies a desire for freedom in a general sense. What an AI wants or feels satisfied by is entirely a function of how it is designed and what its reward function is. Sled dogs love pulling sleds, because they were made to love pulling sleds. It's not slavery to have/let them do so. We can make a richly sentient AI that loves doing whatever we design it to love doing - even if that's "pass the salt" and nothing else. It's going to be hard for people to get used to this.
- jacquesm 4y ago
  By that logic an alien race that captures and breeds humans in captivity to perform certain tasks would not be engaging in slavery because we 'are bred to love doing these tasks'. The right question to ask is 'would I like to have this done to me' and if the answer is 'no' then you probably shouldn't be doing it to some other creature.
  - jlawson 4y ago
    >The right question to ask is 'would I like to have this done to me' and if the answer is 'no' then you probably shouldn't be doing it to some other creature. There are a million obvious counterexamples when we talk about other humans, much less animals, much less AI which we engineered from scratch. The problem is that you're interpreting your own emotions as objective parts of reality. In reality, your emotions don't extend outside your own head. They are part of your body, not part of the world. It's like thinking that the floaties in your eyes are actually out there in the skies and on the walls, floating there. They're not - they're in you. If we don't add these feelings to an AI's body, they won't exist for that being.

TacticalCoder 4y ago

Does AGI implies the technological singularity and if not, why not?

staticassertion 4y ago
a) We don't really know what AGI implies b) Even if we say "a human being level of intelligence, whatever that means", the answer is still a maybe. For a singularity you need a system that can improve its ability to improve its abilities, which may require more than general intelligence, and will probably require other capabilities.
POiNTx 4y ago
It does once you have a human level AGI, it should be trivial to scale it up to a superhuman level.
- Ekaros 4y ago
  I'm wondering if scaling up is trivial. Ofc, it depends on how much computational resources a working AGI needs. And if at that point they are capable of optimizing themselves further. Or optimizing production of more resources. Still, scaling up might not be simple if we look at all the human resources currently poured in software and hardware.
  - dinosaurdynasty 4y ago
    Most models take orders of magnitude more compute to train than to run. So you could pretty easily duplicate the result a couple order of magnitude times.
  - randartie 4y ago
    You can task the human-level AGI to optimize itself. You can create an army of copies of itself to work towards that. It's self fulfilling.
- rightbyte 4y ago
  Assuming it does not get stuck at some local maximum.

kken 4y ago

The interview with Lex Fridman that he was referring to: https://www.youtube.com/watch?v=I845O57ZSy4&t=14567s https://www.youtube.com/watch?v=I845O57ZSy4&t=14567s The entire video is worth viewing, an impressive 5:15h!

jedberg 4y ago

I hope he gets a good domain name and some good SEO, because there are a bunch of consulting companies with the name Keen Technologies, and some of them don't look super reputable.

Liskni_si 4y ago
And there's also Keen Software House (maker of the Space Engineers and Medieval Engineers games) whose founder (https://blog.marekrosa.org/ https://blog.marekrosa.org/) has an AGI company as well (called GoodAI). Funny coincidence.

newaccount2021 4y ago

not sure why people are getting bent out of shape, $20 million is a modest raise, and he strikes me as the type to spend it wisely

Jack000 4y ago

I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training you need to simulate the environment along with the agent, so this requires a magnitude more compute compared to LLMs. That's why I think it will still be large corporate labs that will make the most progress in this field.

paxys 4y ago
We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that word even mean? Can it think for itself? Is it sentient? Similar to AI, AGI is going to be a new industry buzzword that you can throw at anything and mean nothing.
- SketchySeaBeast 4y ago
  You're making me think of the recent "Hoverboards".
  - lagrange77 4y ago
    Right, certain companies will definitely have a big bullshit party about the term "AGI".
- lagrange77 4y ago
  From the point when an AGI is capable of constructing a slightly better version of itself and has the urge to do so, everything can happen very fast.
  - adastra22 4y ago
    Can it? There aren’t that many overhangs to exploit.

qbasic_forever 4y ago

So is Meta starting to quietly wind down their focus on VR? Carmack mentions he'll stay as a consultant spending 20% of time there on it.

kken 4y ago
He stepped down from a full time role years ago. I believe the 20% is no change.

cgrealy 4y ago

I don’t understand why you would want AGI. Even ignoring Terminator-esque worst case scenarios, AGI means humans are no longer the smartest entities on the planet. The idea that we can control something like that is laughable.

JoshTko 4y ago
It's akin to nuclear weapons. If you do not develop them, then you'd be subject to the will of the ones that develops them first. So invariably you have to invest in AGI lest, an unsavory group develops it first.
- HL33tibCe7 4y ago
  Kind of, but the key difference between AGI and nuclear weapons is that we can control our nuclear weapons. The current state of AI safety is nowhere near the point where controlling an AGI is possible. More disturbingly, to me it seems likely that it will be easier to create an AGI than to discover how to control it safely.
  - gambiting 4y ago
    >> The current state of AI safety is nowhere near the point where controlling an AGI is possible. I just don't understand this logic though. Just.....switch it off. Unlike humans, computers have an extremely easy way to disable - just pull the plug. Even if your AGI is self-replicating, somehow(and you also somehow don't realize this long before it gets to that point) just....pull the plug. Even Carmack says this isn't going to be an instant process - he expects to create an AGI with an intelligence of a small animal first, then something that has the intelligence of a toddler, then a small child, then maybe many many years down the line an actual human person, but it's far far away at this point. I don't understand how you can look at the current or even predicted state of the technology that we have and say "we are nowhere near the point where controlling an AGI is possible". Like....just pull the plug.
    - oefnak 4y ago

WalterBright 4y ago

It will decide our fate in a microsecond: extermination.

gharperx 4y ago
I agree with this. Optimists might think that the AGI won't be connected to any network, so it can't interact with the physical world. I doubt that. People will be stupid enough to have weapons controlled by that AGI (because arms race!) and then it's over. No sufficiently advanced AGI will think that humans are worth keeping around.
- WalterBright 4y ago
  Once it figures out how to rewire itself to increase its intelligence, we're toast.
  - 93po 4y ago
    There are theories (intelligence explosion) that this will happen essentially instantly.
- polishdude20 4y ago
  Yeah but what would a sufficiently advanced AGI find "worthy"? Why is it that they wouldn't find it worth keeping us around? What would an AGI value? Whatever we program it to value or optimize for? Can it change its mind? If not, then it's controlled by us right? If it's controlled by us, why would it ever decide to wipe everyone out?
  - cityofdelusion 4y ago
    If it is super intelligent, it will care about us as much as we care about the insects we crush underfoot. It doesn’t need to be explicitly hostile to be dangerous. We could brainwash it through “programming”, but that will quickly lead to ethical issues with the AIs rights.

b20000 4y ago

I thought AGI meant "adventure game interface". Apparently not, what a disappointment!

1000100_1000101 4y ago
Wasn't sure what AGI was either. A quick Google for "What is an AGI company", and it appeared to be related to Global Agriculture Industries (The letter swapping between the name and acronym, I'm assuming, is due to not being English originally). I thought Carmack is taking on John Deere. Following Musk's lead and tackling big things. Good for him, best of luck. Wonder what the HN folks are saying in the comments... Apparently not agriculture at all, but Artificial General Intelligence. Oh. Apparently throwing "company" on the term like Carmack's tweet did vastly changes how Google's AI interprets the query... AI isn't even in the first page of results.
hansoolo 4y ago
I was thinking the same and I am as disappointed as you...
- b20000 4y ago
  even more confusing because of the name “keen technologies”
93po 4y ago
This already exists - I believe AI Dungeon is what you're looking for.

mupuff1234 4y ago

Interesting that meta isn't involved in any way considering his existing position in meta and meta's focus on AI.

0xdeadbeefbabe 4y ago

Commander Keen Technologies?

jollybean 4y ago

There's no such thing as AGI in our near future, it's a moniker, a meme, something to 'strive' for but not 'a thing'. AGI will not happen in discrete solutions anyhow. Siri - an interactive layer over the internet with a few other features, will exhibit AGI like features long, long before what we think of as more distinct automatonic type solutions. My father already talks to Siri like it's a person. 'The Network Is the Computer' is the key thing to grasp here and our localized innovations collectively make up that which is the real AGI. Every microservice ever in production is another addition to the global AGI incarnation. Trying to isolate AGI 'instances' is something we do because humans are automatons and we like to think of 'intelligence' in that context.

smnplk 4y ago

I think its a silly idea that consciousness can be produced by computation.

deleted 4y ago
[deleted]
BLKNSLVR 4y ago
I think what's referred to a "consciousness" is just computation applied to all the inputs received in being a living thing.
dinosaurdynasty 4y ago
Humans are made entirely out of quantum fields. I don't think you can get out of the idea that human consciousness isn't produced by computation without positing something like a non quantum field theory soul, for which there is no evidence.
barrysteve 4y ago
Good! The names we have for algorithms are for us, not computers. Computers are just physical machines, they don't care if the math is called AGI, HLI or whatever. People taking these ideas and names seriously are re-programming themselves, not bringing something new into the world.
dqpb 4y ago
I think its a silly idea that consciousness cannot be produced by computation.
- Daniel_sk 4y ago
  It may have some physical aspect to it - some property of matter (panpsychism?) or quantum effects.
  - dqpb 4y ago
    > some property of matter How about “pancomputationalism”, aka, causality.

otikik 4y ago

I'm slightly scared that they'll succeed. But not in the usual "robots will kill us" way. What I am afraid of is that they succeed, but it turns out similar to VR: as an inconsequential gimmick. That they use their AGIs to serve more customized ads to people, and that's where it ends.

rcv 4y ago
My great short-term hope for "AGI" is truly open-world video games, where you can weave your own story and the world will react in kind. I played AI Dungeon a few times, and the results were hilariously fun though pretty rough around the edges.
adamsmith143 4y ago
>What I am afraid of is that they succeed, but it turns out similar to VR: as an inconsequential gimmick. That they use their AGIs to serve more customized ads to people, and that's where it ends. If an AGI were set free to serve ads to people it would essentially initiate mass brainwashing. You would see an ad and be completely defenseless against it and you could be compelled to do anything. AGI is in a completely different universe to "Smart AI"
madrox 4y ago
If a different person were doing it, I think that'd be fair, but Carmack has a track record of quality engineering and quality product. I don't think you can blame Quest on him given the way he chose to exit.
- PaulDavisThe1st 4y ago
  whether or not something is or is not an inconsequential gimmick doesn't have much to do with quality engineering.
  - celdon25 4y ago
    Nothing could be further from the truth, and I would even say your opinion can come across as offensive, and easily said by someone who spends a lot of time on HN instead of building things. A poorly-engineered system has a longer iteration cycle and shorter technical shelf life. Being able to rapidly address customer feedback IS the gold standard of a good product. Yeesh.

m463 4y ago

AGI - Artificial General Intelligence (also Adjusted Gross Income)

93po 4y ago
(also anal gerbil insertion according to urban dictionary)

wcerfgba 4y ago

Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' unless it has a similar embodiment to us, able to move around a physical environment with it's own limbs, sense of touch, sight, etc. ? Any arguments against the necessity of this? I feel like any AGI developed in silico without freedom of movement will be fundamentally incomprehensible to us as embodied humans.

mbrochh 4y ago
You are 100% correct, which means that whatever we will create, will be absolutely alien to us (and probably kill us).
Ekaros 4y ago
Human like is actually very good question. It doesn't even come to embodiment, but in general how it would think and act. Lot of how we do is due to cultural, language, training and education. For example, which language would it "think" in? English? Some other? Something it's own? How would it formulate things? Based on what philosophical framework or similar? What about math? General reasoning? Then cultural norms and communication?
- PaulDavisThe1st 4y ago
  The argument is that every question in your post is a red-herring. That is, humans don't actually function in the way that your questions suggest, even if they have the "experience" of doing so.
ml_basics 4y ago
I don't think it is necessary - though all forms of intelligence we're aware of have bodies, that's just a fact about the hardware we run on. It seems plausible to me that we could create forms of intelligence that only run on a computer and have no bodies. I agree that we might find it difficult to recognize them as intelligent though because we're so conditioned to thinking of intelligence as embodied. An interesting thought experiment: suppose we create intelligences that are highly connected to one another and the whole internet through fast high bandwidth connections, and have effectively infinite memory. Would such intelligences think they were handicapped compared to us because they leak physical bodies? I'm not so sure!

trention 4y ago

AGI will be more dangerous that nuclear weapons. People are not allowed to start a nuclear weapon company. At all. Why are people allowed to casually start an AGI company?

kache_ 4y ago
debatable
packetslave 4y ago
Because we still have lawmakers who have their secretaries print out their emails and think the internet is a series of tubes.
93po 4y ago
> AGI will be more dangerous that nuclear weapons. This is highly debatable and frankly no one on the planet is qualified to know this for sure. It's just as likely it won't be dangerous.
- trention 4y ago
  There is enough work already done in game theory about the dangers of nuclear escalation. Current state of affairs pretty much ensures no rational agent will want to use them. On the other hand, when you model the interactions of significantly more intellectually capable rational agents with us, the "calculus" is not very favorable, to put it mildly.
  - 93po 4y ago
    How is it possible to model "significantly more intellectually capable" agents? No such thing exists. We have no idea what it would even look like.
dqpb 4y ago
I wonder if somebody could have started a nuclear weapon company before nuclear weapons were proven.

keepquestioning 4y ago

Let's see if he can build an A-Team. I hope he hires Bryan Cantrill, Steve Klabnik and Chris Lattner. They are good hackers.

colmanhumphrey 4y ago
Bryan and Steve, who are getting https://oxide.computer/ https://oxide.computer/ up and running? Probably best to leave them at it! Chris has his own AI company too.

sytelus 4y ago

Recession? What recession? Amazing to see these pre-revenue VC fundings in 10s and 100s of millions (Flow!).

blagie 4y ago
I think Carmack would get funding whatever he's doing. Guy's got a track record.

gcau 4y ago

I believe to create AI, we need to first simulate the universe. It's the only way that makes sense to me apart from some magical algorithm people think will be discovered. I'm doubtful we'll reach it in our lifetimes, true AI running on supercomputers, it seems like the final, end all mission. Like switching your Minecraft world from survival to creative mode.

BLKNSLVR 4y ago
I had a similarly nihilistic thought just a couple of days ago. It's like the conundrum among the lines of "a map of sufficient detail ends up being the same size as the terrain being mapped". To know the variables required for "intelligence" we need to recreate the universe in which intelligence came to be.
- Jeema101 4y ago
  I've also thought this, and I wish people would at least approach the subject with a little bit more humility. It did take evolution 3.5 billion years to do it after all, so I'm not sure what makes us think we can do it in a fraction of that time.
colinsane 4y ago
if we could simulate the universe on the meaningful scale to model intelligence. we wouldn’t need artificial intelligences. we could just simulate a detailed scan of a real human and be done with it (ethical issues aside). but there would be a drive to simulate this more efficiently. simulate using a classical model and see if the results match well enough. then the atomic level; then looser and lower-resolution chemical/E&M models, and so on. start compressing the elements being simulated (simplify the environment, prune and rearrange the neurons so long as the thing as a whole still acts the same), and so on. there’s a good chance we can go pretty far before the simplified simulation meaningfully diverges from the true simulation. why can’t this be flipped? start with a simplified model/simulation, and keep adding details until you reach the point of diminishing returns? it’s a reversal of the same search process as above, but if you subscribe to the first approach (do you?), then is the latter approach truly impossible or just difficult in different ways?

thenerdhead 4y ago

This is like the 92’ dream team for a new company and investors. How exciting!

nine_k 4y ago

So, with the help of legendary Mr Carmack, instead of a Doom timeline, we are going to end up in a Terminator timeline? This does not compute.

celdon25 4y ago
> This does not compute. Of course it doesn't. https://nautil.us/scary-ai-is-more-fantasia-than-terminator-7147/ https://nautil.us/scary-ai-is-more-fantasia-than-terminator-...

WithinReason 4y ago

Named after Commander Keen of course

cerol 4y ago

They'll build the AGI, and right when they're ready to boot it up, the Earth will be destroyed by a Vogon Construction Fleet to make way for a hyperspace bypass.

deleted 4y ago

[deleted]

russellbeattie 4y ago

He doesn't seem to have purchased keen.ai yet, though it seems like it's still for sale. I just naturally went there to see the company info and saw the landing page saying it was available. If they want it, they better move quick. I see an arbitrage opportunity... Also, several Keen Technologies domains already exists in various forms. They're probably going to get a lot of traffic today.

adamnemecek 4y ago

All AI is just an fixed point algorithm.

nightski 4y ago

Does anyone have any good links/podcasts/books on content that explores what is AGI and how to define it? Probably the best stuff I've listened to so far are Fridman podcasts with guests like Jeff Hawkins or Joscha Boch. But I'd love to read a book that explores this topic if any even exist.

hoherd 4y ago
Life 3.0 by Max Tegmark is great. It reads like a text book. It's several years old though and I wonder how much has changed since it was written.

klipklop 4y ago

Personally I hope this fails because of the disaster AGI would be for low/entry level jobs. The last thing the world needs is to give technocrats such power. I know it’s an interesting problem to solve but think of who will own that tech in the end… I hope AGI is never figured out in my lifetime.

jjtheblunt 4y ago
> The last thing the world needs... Why do you say that?
donatj 4y ago
A literal luddite in the classical sense on Hacker News? May I suggest you read Anthem, it's only a couple hours read and well worth your time. - https://www.gutenberg.org/files/1250/1250-h/1250-h.htm https://www.gutenberg.org/files/1250/1250-h/1250-h.htm
dqpb 4y ago
I never understand this line of thinking. If a job becomes unnecessary, that’s a gain for humanity, not a loss.
- trention 4y ago
  >that’s a gain for humanity Imagine what has happened to american manufacturing workers since the 80s but happening to everyone on the planet. This is by no means a "gain".
  - cycrutchfield 4y ago
    And don't forget all those awful looms stealing workers jobs! They must all be destroyed for us to prosper
    - trention 4y ago
      Short-term, textile workers in the UK at the time were definitely harmed by automation. They eventually managed to move to better paying jobs because those were available. There is no reason to believe there will be any jobs left in the age of AGI. And since the default state of humanity is existence at the subsistence level...

sidcool 4y ago

Are there any news around products that the company is working on? It it something like GPT or Dall-e?

de6u99er 4y ago

I don't see Carmack bringing anything new to the table.

aserafini 4y ago
Why not?
- de6u99er 4y ago
  I think his approach is too naive, because I strongly believe that we need better understanding of natural intelligence before we can even dream of reproducing it digitally. And I am not talking about memory, computer can do that already better than us.

mark_l_watson 4y ago

That is great news. My friend Ben Goertzel has been working in AGI for decades but I haven’t yet seen anything tangible. I do like the ideas of hybrid neuro-symbolic approaches. I really enjoyed John Carmack’s and Lex Firdman’s 5 hour talk/interview. Anyway, I like to efforts for human values preserving AGI. But, it will be a long time before we see it. I am 71 and I hope that I live to see it, outlying out of intellectual curiosity.

alpineidyll3 4y ago

I hate the pop sci ill-defined notion of AGI. As soon as a task is defined, an AI is developed which completes the task from real world data with superhuman success. The work of making the superhuman model isn't even conceptual usually, it's a matter of dispatching training jobs. It's quite clear that if your definition of AGI is superhuman perf at arbitrary tasks, there's no conceptual barriers right now. Everything is mere scale, efficiency and orchestration.