39 ms·
John Carmack: I’m going to work on artificial general intelligence
- bitwize 7y agoBecause of course he is. Because John Carmack.
- hans1729 7y agoHeh, from the comments: >Congratulations on the new project, and may your hubris not doom us all.
- kotrunga 7y agoThat's the very comment that stood out the most to me as well...
- dcwca 7y agoGood Doom pun
- shurcooL 7y agoI wanted to find some discussion on this topic. I’m not pessimistic, but I am curious to consider what it may mean for humanity if/when AGI happens.
- patentatt 7y agoMy pet theory is that anything we regard as truly intelligent has to have a sense of self, an identity if you will. In addition, it has to be motivated by something. All living things, and all the intelligent living things, have a survival and reproduction instinct on some level which is an animating force driving higher level thinking and actions. Basically, any AGI has to have something approaching a ‘soul’ to be ‘intelligent.’ And if that emerges, combined with exponential iterative evolution that isn’t limited by biology ... well all the sci-fi tropes seem plausible.
- ses1984 7y agoWhat a boss.
- mc3 7y agoAnd now, without a boss!
- laxatives 7y agoIs he doing this under the Facebook umbrella? Or departing? Or more-or-less retiring from regular obligations entirely?
- k__ 7y agoHe said that he will work as a consulting partime CTO for Oculus and do GAI from home, sounded like non-Facebool work to me.
- Someone1234 7y agoI think semi-retiring. He's going to be "consulting [on Oculus]" while: > I am going to be going about it “Victorian Gentleman Scientist” style, pursuing my inquiries from home, and drafting my son into the work. Which to me reads like part time work on Oculus, part time work on this AGI project. If it is with Facebook it isn't at all clear from the post (plus I'd assume it would be accompanied by marketing copy in that situation).
- hans1729 7y ago>Starting this week, I’m moving to a "Consulting CTO” position with Oculus. I will still have a voice in the development work, but it will only be consuming a modest slice of my time. As for what I am going to be doing with the rest of my time: [...] For the time being at least, I am going to be going about it “Victorian Gentleman Scientist” style, pursuing my inquiries from home, and drafting my son into the work.
- tcbawo 7y agoHaving listened to his appearance on the Joe Rogan Experience, I can't help but hear this message in my head with his distinct voice. But also, I think his brutal honesty and relentless work ethic will suit this problem domain.
- drefanzor 7y agoIf anyone can initiate the singularity, it's John Carmack.
- deleted 7y ago[deleted]
- eggy 7y agoGreat, AGI from the guy who brought DOOM to many. Do we ever learn?
- chrisco255 7y agoDOOM 4: AGI confirmed.
- The_rationalist 7y agoI hope he'll go the hybrid symbolic and neural network way (causal and statistical), instead of just statistical. AGI needs a type system... I hope I'll achieve AGI before him but it's nice to know there's some real competition! (because, reader, there are almost 0 researchers seriously trying to achieve AGI in a not totally bullshit way. Only opencog and Cyc comes to mind).
- cr0sh 7y agoYou forgot "he who shall not be named"... /ok, maybe his project falls under "total bullshit"...
- catalogia 7y agoWho are you referring to? I feel out of the loop here.
- p1esk 7y agoSchmidhuber
- cr0sh 7y agoThose of us who've been on the net long enough, and have at least dabbled in AI/ML circles, know of him. He claims to have invented a program that is a mind, originally written in Forth, translated by others to many other languages, etc. He has published the code of this program in a form of "open source", so you can easily find it if you dig enough. He's widely considered to be a crank. That said, the line between genius and madness can be mighty thin, and what side he lands on is anyone's guess, but most put him well over the line into madness, for whatever its worth. My own opinion? Well - looking at his work from purely the modern understanding and research into ML/AI - that is, deep learning and such - his work would be considered pointless, probably worse than Eliza as to its contributions to the field. But as someone who has read a lot of other works (for for and against) the idea of AGI, artificial consciousness, theory of mind, etc - his work at a certain level has echos with some of that work. Still probably a dead end, but at the same time, there's some interesting concepts within his code and theories (he's self-published a book on it, too - you can find it on Amazon - he also has it for free on his github and it can be found elsewhere). I guess I still put him in crank territory, but not in the abusive crank arena, more in the "doing his own thing, but being a bit evangelical about it" - relatively harmless. His work is not as amazing as TempleOS, imho, but there's a similar mind behind it (though comparing it with that operating system is maybe an unfair, possibly orthogonal, comparison). I won't say or reveal more (but I've written enough for you to figure it out) - he tends to monitor tons of forums and if he thinks he's being "summoned", he'll spam the forum with his writings and theories. It got him "perma-banned" from more than one newsgroup back in the day...
- deleted 7y ago[deleted]
- option 7y agoWe need more people like him in AI. Super happy!
- phillco 7y agoThis is a very artfully written statement. It avoids the most more devastating headline of “John Carmack Stepping Down”. (Yes, going part-time isn’t the same as leaving entirely, but still).
- _bxg1 7y agoMore like "John Carmack has well-balanced life priorities and has decided to do a really cool moonshot project with his son instead of making even more money"
- wpdev_63 7y agoCrap! When I thought VR was hitting its stride with the oculus quest. For anybody who hasn't played an untethered VR experience, I highly recommend it. It makes a world of difference with games like Echo combat and Beatsaber. Tons of fun. It reminds me of the first time I played wii bowling.
- nothis 7y agoThe real headline is that VR is finally confirmed to have hit a major roadblock. If John Carmack gets frustrated with its progress, there's something up. I've long thought that the issue with VR is a conceptual one, not a technical one and maybe that frustration comes from there. "Running forward" is an unsolved problem in room scale VR. For a seated experience, you're basically back to a neat display gimmick + accurate hand tracking. Any real solutions need, on the one side, real-world physical constructions (think running threadmills) that soon hit holodeck-level limitations and, on the other, software that actually benefits from the real technology VR brings to interactive media: super accurate hand- and head-tracking. The first gets impractical/impossible soon, the second limits development to a few niche genres: Shooting ranges, cockpit sims, dance/party games and some vague "experiences" where the actual tech is pretty much ignored and you just say "but it feels so immersive!" (honestly, it does work for horror games!). It's basically motion controls 2.0. The only place I could see the technology shine is, oddly enough, AR. It has way less mainstream hype to it but it makes much more sense because you actually benefit from the tracking of your real-world movement: You're still a part of it! The holo-lens demos that pop up on youtube might seem clumsy, but I can totally see a use case for replacing physical monitors with arbitrarily sized and positioned displays you can virtually move in any office space. There's rumors of Apple working with Valve on AR tech. If there's any technology that could follow the smart phone, AR is my bet. I'm honestly surprised Carmack didn't move in that direction rather than deciding to become a general AI guru.
- xwdv 7y agoImagine inventing artificial general intelligences and then there’s some public outcry for products powered by natural, organic intelligence instead.
- 0xdeadbeefbabe 7y agoThis.
- ganitarashid 7y agoI hope he doesn’t trigger a Phobos Anomalie
- vecplane 7y agoI don't understand how he could contribute to the field of AGI research from home, by himself, and maybe with his son. It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress. Wouldn't it make more sense for him to join a cutting-edge team, like DeepMind or OpenAI?
- johnsimer 7y agoMy view is that you want many people working independent from each other towards the same goal, and that everyone working in one group could hinder creativity/lead to groupthink
- dekhn 7y agoEverything I've ever read about Carmack suggests he'll do his best on his own at home. Much of this work can be done on reasonable hardware, and he's always been really good at getting a lot out of reasonable hardware. Further, if he needs enormous compute resources, he can get it at any of several cloud providers.
- the_watcher 7y agoEven at Oculus, he worked "from Dallas", but spent a huge amount of time working at home.
- K0SM0S 7y ago> if he needs enormous compute resources, he can get it at any of several cloud providers This is exactly the experience of most teams I've spoken with, be they students or businesses, for all the pre-production phase. You simply can't and shouldn't spend on costly AI infrastructure before you've nailed your solution; in fact any kind of infra not just AI. What you do is rent some cloud to power quickly through your tests — better have 10x worth of big Nvidia GPUs over 2 weeks than buy 1 or 2 max yourself and wait 5-10x more time — not even factoring that setting up clusters of GPU and running such workflows consistently over days, weeks requires pretty deep sysadmin/hardware knowledge and experience; it took me two years to really master that non-problem part on my home server (but now it's a skill I have so that was worth it, but certainly set my research and learning back by as much time). Besides, there's a time when the familiarity, safety and general comfort of home simply can't be beat. Notwithstanding pool tables and free soda, lol.
- 0xdeadbeefbabe 7y agoFrom the guy who brought us Commander Keen. Applying intelligence to artificial intelligence has happened before https://en.wikipedia.org/wiki/Shakey_the_robot https://en.wikipedia.org/wiki/Shakey_the_robot. One of the contributors to Shakey was Alfred Brain.
- rhacker 7y agoI think we just got 50% closer to making AGI happen.
- ekianjo 7y agoIf your odds were very low to begin with 50 % more wont make much of a difference.
- bronz 7y agoi say this with respect and humility, but i am very surprised at the naivete with which John addressed the subject of AGI. he is so casual about it -- not only the idea of working on it but also the idea of it existing at all. he seems oblivious to the gravity of that discovery. it is not just "very valuable," it will be earth shattering and probably wipe out humanity. and its his side-project. and his son will help out. John is the perfect representation of what is wrong with peoples attitude toward AGI. aloof and naive.
- asadlionpk 7y agoI think the real naive are the people who are doing it as a day job.
- popup21 7y agoOh those are some spicy peppers!
- randomidiot123 7y agoHe might as well casually work on Faster Than Light travel, or a Grand Unified Theory.
- randomidiot666 7y agoHe might as well casually work on Faster Than Light travel, or a Grand Unified Theory.
- whamlastxmas 7y agoWhat is your source for saying AGI will probably wipe out humanity? How could we ever even attempt to guess at the motivations of something we can barely comprehend and doesn't even exist yet?
- goatlover 7y agoThe main concern is not that it's like Skynet and wishes us harm, but that it does things harmful to us because the means to accomplish the AI's goals are at odds with human values, which was unanticipated by the human creators, since the AGI is coming up with its own solutions. And the AGI doesn't have the same values as humans, so it doesn't care if its solutions are harmful.
- deleted 7y ago[deleted]
- yaseer 7y agoJohn Carmack seems like a guy that would've made fundamental contributions to science, had he been born a century before. Computer science now occupies the place physics once did, in its impact on moving the world forward. Best of luck to him, I look forward to seeing what he produces!
- JohnJamesRambo 7y agoYou guys really get high on your own supply don't you? It's a toss-up these days whether something from computer science or a programmer will either enhance or erode the human experience and quality of life.
- whamlastxmas 7y agoI think you're on the wrong site if you're wanting to criticize software development as a career and suggest it does more harm than good
- cambalache 7y agoYes, join the echo chamber or get out.
- whamlastxmas 7y agoIt's more: offer substantiative evidence of your counter-culture claims that vilify most of your reading audience, or take your baseless negativity elsewhere
- jodrellblank 7y agoSo which are you going to do?
- catalogia 7y agoI can't think of a better site for him to make such arguments. What's the point of such criticism if the subjects of the criticism never read it?
- pyentropy 7y agoProgress in AI is due to data and computational power advances. I wonder what kind of advances are needed for AGI. 1. Biological brains are non-differentiable spiking networks much more complicated than backpropagated ANNs. 2. Ion channels may or may not be affected by quantum effects. 3. The search space is huge (but organisms aren't optimal and natural selection is probably local search) 4. If it took ~3.8b years to get from cells to humans, how do we fast-forward: * brain mapping (replicating the biological "architecture") * gene editing on animal models to build tissues and/or brains that can be interfaced (and if such interface could exist how do we prevent someone from trying to use human slaves as computers? Using which tissues for computation is torture?) * simulation with computational models outside of ECT (quantum computers or some new physics phenomenon) Note: those 3.8b years are from a cell to human. We haven't built anything remotely similar to a cell. And I'm not claiming that an AGI system will need cells or spiking nets, most likely a lot of those are redundant. But the entropy and complexity of biological systems is huge and even rodents can outperform state of the art models at general tasks. IMHO, the quickest path to AGI would be to focus on climate change and making academia more appealing.
- m0zg 7y agoYou forgot to mention, crucially, that neurons in close proximity affect each other, which is just one of the things that makes modeling of more than a few neurons in time domain a complete non-starter. It all results in enormous systems of PDEs which we don't know how to solve yet at all. You could say that we do not have the right mathematical apparatus to model any such thing.
- McTossOut 7y agoPure physical modeling are likely a bad representation for the phenomena resulting in intelligence, especially granted we've simulated it with much simpler discrete structures. PDEs may even be disastrously bad, like trying to describe a line in space with a table of points, instead of the degrees of freedom. I would imagine that a PDE may cover diffuse behaviors governing say, how learning happens mechanically, but it is almost certainly a language/representational barrier, the relationship between the structure of the animal mind, learning, and seemingly simple phenomena, like afterimages. The molecules are arbitrary and the timescale doesn't matter.
- skokage 7y ago>Runner up for next project was cost effective nuclear fission reactors, which wouldn’t have been as suitable for that style of work. I can't tell if he was serious about that comment or not... Considering he builds rockets with free time, it could go either way.
- the_watcher 7y agoI took it as "not even I would experiment with nuclear fission reactors from my house".
- Voloskaya 7y ago> I can't tell if he was serious about that comment or not... Considering he builds rockets with free time, it could go either way. Nuclear fission shocks you but not AGI?
- nafizh 7y agoThis is so vague as in borderline sarcasm. Okk, it’s AGI but really, what is it? Reinforcement learning? Combining symbolic AI with modern advances? Deep learning theory? AGI is a vacuous term. And no one knows which path would lead to something similar to human intelligence. That itself is a matter of research.
- carlosdp 7y agoThe things you all listed are possible pathways to the goal. All he said is he wants to start working toward the goal. This guy is serious business, he's never been one to spew BS. He's a real deal, no bullshit computer scientist with leagues of novel accomplishments under his belt. I wouldn't bet against him when he sets his mind to something.
- stupidcar 7y agoWhich is presumably the research he intends to do over the next few months? E.g. reading around the subject to determine where best to focus his efforts.
- h5eath5eahj 7y agoHopefully short term includes going on Lex Fridman's podcast
- deleted 7y ago[deleted]
- proc0 7y agoI think his expertise in video game engines and rendering has a lot to contribute here.
- ausjke 7y agohe is 49 this year, considered as one of the most genius programmer on earth. There are professors still doing real work at 90+ year old(yes, the UT professor goodenough for Nobel prize), John has a long way ahead, best luck!
- chasd00 7y agoi've heard he was pretty good, i didn't realize he was this widely respected and admired. I remember him for that fast inverse square root hack but that's about it.
- zhynn 7y agoHe's already a legend, and he's not even 50.
- galangalalgol 7y agoI think he got that from someone at SGI.
- galangalalgol 7y agoOr was the second person to independently come up with it.
- streetcat1 7y agoJust out of curiosity, how do you define a genius programmer? (vs regular programmer).
- hunterjrj 7y agoHow about “wrote the Doom engine, then the Quake engine”. Good start?
- tirewarehouse 7y agoHe ushered in modern gaming, and everything he did in that domain was way ahead of the curve.
- thrower123 7y agoI was wondering why his tweeting about graphics and VR went to 0. We've gone back into the part of the cycle where VR is an odd curiousity again, haven't we?
- randomidiot123 7y agoUnfortunately VR has turned out to be an intensely nauseating puke fest for many of us.
- randomidiot666 7y agoUnfortunately VR turned out to be an intensely nauseating puke fest for a lot of people.
- thrower123 7y agoSame as the last go round. In ten years, a new generation will think that this time, they know how to do it right. Maybe the hardware will even. have caught up.
- K0SM0S 7y agoI'm still convinced we'll have "good enough" barebones yet spatially aware AR first. It's just less intensive on the front end side, which is the limiting factor now as I understand it. VR is a whole other thing, and I think the "uncanny valley" stretches quite far; ie you need quality really close to what we see in movies to pass the acceptable threshold beyond a few hours of novelty.
- randomidiot666 7y agoVR sickness still applies, even if you have perfectly realistic rendering, infinite frame rate, infinite resolution, perfect tracking, and zero latency. It's caused by a discrepancy between the visual and vestibular systems. That problem cannot be solved by higher quality rendering.
- carlosdp 7y agoThis doesn't surprise me at all. He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient). (edit: I'm not claiming he's a field expert in a week guys, just that he can probably learn the basics pretty fast, especially given ML tech shares many base maths with graphics) As recent as his last Oculus Connect keynote, he extolled his frustration with having to do the sort of "managing up" of constantly having to convince others of a technical path he sees as critical. He's clearly the type that is happiest when he's deep in a technical problem rather than bureaucracy, and he likes moving fast. On top of that, he likes sharing with the community with talks and such, and ever since going under the FB umbrella, he's had to clear everything he says in public with Facebook PR, which clearly annoyed him. He's hungry for a new hard challenge. VR isn't really it right now since it's more hardware-bound by the need for hard-core optical research than software right now. With the Quest, he (in my opinion) solidified VR's path to mobile standalones. It's time to try his hand at another magic trick while he's on his game. John's the very definition of a world-class, tried and true engineer/scientist. He's shown time and time again the ability to dive into a field and become an expert very quickly (he went from making video games to literally building space rockets for a good bit before inventing the modern VR field with Palmer). If there's anyone I'd trust to both be able to dive into AGI quickly and do it the right(tm) way, it's John Carmack.
- mochomocha 7y ago> He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient). You must be joking, right? I'm as much of a Carmack fan as anyone here, but overstating the skills of one personal hero does no good to anyone.
- grumpy8 7y agoTo be fair, a whole uninterrupted week of highly focussed work can get you pretty far (considering that you have the necessary background, which Carmack has, i.e. related to linear algebra, stats, programming, etc.)
- ekianjo 7y agoOther way to read this: big corporations are slowing down on VR. The market has not taken off as rapidly as they expected so we will see more moves away from heavy investment in VR.
- baddox 7y agoCarmack never struck me as the type to chase the money from fad tech to fad tech. On the contrary, he seems keen to chase his own interests, and by now I suspect he is financially secure enough to do so.
- ekianjo 7y agoI did not mention Carmack in my comment. Corporations. Facebook probably decided that having Carmack working on a minuscule market that is barely growing (look at headset sales per year, it's virtually nothing and there is no "acceleration" in sight either) was a waste of his talent.
- cma 7y agoHe’s not working on AGI with Facebook.
- oarabbus_ 7y agohttps://qz.com/1739575/strong-oculus-quest-sales-boost-facebooks-non-advertising-revenue/ https://qz.com/1739575/strong-oculus-quest-sales-boost-faceb...
- ekianjo 7y ago180 000 units of VR headsets is pure pocket money for Facebook. And that does not even register in any kind of hardware sales charts. PC sales are more than 300 millions per year, and smartphones sales are more than 1.5 billions units per year. VR headsets don't sell at all.
- 7y ago
- eaenki 7y agoIt would be cool if he specified what he was going to work on. Current narrow AI is all about data and computer power but I don’t see AGI coming out of more data / more power anytime soon.
- mahesh_rm 7y agoThis post feels a little bit like something that'll be upvoted to top spot many years from now, for one reason or another. :-)
- nikkwong 7y agoI'm sure he knows his stuff, and is venturing into this knowing that there's somewhat of a "hard problem" to be solved here; serious proponents of AI like Kai-Fung Lee have stated that AGI may never be possible. The fact that he's making a public statement like this leads me to believe he may already have some novel solutions on how to tackle the problem. We won't be expecting to be seeing the same old parlor tricks coming out of John Carmack. He is already on the forefront of this stuff after all. That's exhilarating but also terrifying. Our still-barbarian level human systems are still nowhere near ready to deal with the socioeconomic problems that may arise with AGI.
- Voloskaya 7y ago> leads me to believe he may already have some novel solutions on how to tackle the problem. > That's exhilarating but also terrifying. Our still-barbarian level human systems are still nowhere near ready to deal with the socioeconomic problems that may arise with AGI. I think you are reading way too much into his statement. It's extremly unlikely that he just magically figured out a way to tackle the problem (just knowing where to get started would be massive).
- drcode 7y agoCarmack is a pragmatist- I'd doubt he'd be doing this if he didn't feel like he had some initial promising directions in mind, though I agree he's probably very early in the process still.
- Voloskaya 7y agoOr maybe when he is talking about making a dent on AGI, he is talking about a very long term objective (say 20-30 years) and he still has to figure out everything.
- umvi 7y agoThere's no such thing as AGI in my opinion. There is no way to create a "conscious" machine. We might be able to come up with some reasonably impressive imitations, but nothing that is conscious or actually thinking like a human.
- ralusek 7y agoYour opinion is very likely incorrect.
- umvi 7y agoWant to put money on it? I bet $1000 we will not have a conscious computer within the next 50 years.
- Nuzzerino 7y agoConscious and intelligent have different meanings though?
- retsibsi 7y agoHow would you define 'conscious' in an empirically testable way? I'm not being a smartarse here, I'm sincerely curious -- because it seems that you're confident that non-biological consciousness is impossible, and also that p-zombies (beings that give every outward appearance of consciousness without actually possessing it) are impossible. So you probably have some specific beliefs about the necessary limits of non-conscious 'intelligence', and I'm interested in what they are and how you've arrived at them.
- 77pt77 7y agoSo what goes beyond physics in the human brain? Because we can simulate physics.
- wahern 7y agoIf you can explain precisely why we cannot build a conscious machine, you could become one of the most revered, and potentially wealthiest, researchers in the world, as I presume such a proof would necessarily introduce unknown and very useful science. I personally don't believe modern machine learning is remotely close to AI, except perhaps the very lowest rung of the ladder of self-serving AI definitions. I base that belief on what seem to be the unknowns, reinforced by predictable failures[1]. But I have very little reason to believe it's impossible. Not even the possible necessity of quantum effects would seem to preclude it. Heck, we've already begun harnessing quantum effects in materials science, computing, biology, and other areas. Unless you mean that whatever we could eventually come up with would be more biological than machine or that only a human could think like a human, but that seems more like word play, the kind of game AI believers play. (That said, that poses an interesting question: which is more likely to be achieved first--a designed-from-scratch, DNA-based cellular intelligence, or something not based on DNA or otherwise mimicking existing organic life? If at all, of course. Also presuming such a distinction isn't in fact hopelessly quaint and naive.) [1] I'm not a naysayer. While I never believed self-driving cars were around the corner (not even 5 or 10 years out; you can Google my HN comments from years ago), I have no doubt the science has been useful and can and will and is put to great, largely unseen use, as is typical of most science.
- ilaksh 7y agoAfter maybe 10-15 years of being an AGI enthusiast who never dared to pursue it "seriously", something has prompted me in the last few months to decide to make my next side project into a "real" AGI effort. I think maybe its just the number of people who are talking publicly about working on it that is making me want to "work on it" "seriously"? I mean, when I get my current side project "out the door" to some degree, I plan to spend at least a few months where the weekend (or sometimes nights) project that I actually admit to working on is "AGI research". Previously I have occasionally spent a few hours here or there mainly passively trying to learn about some deep learning or AGI thing by skimming papers or watching videos. But the plan now is to actually work on active learning projects/experiments for several hours every weekend. For at least two or three months (or longer if I don't give up before then). Theoretically at least some of it could be of practical use, although I am thinking that I may avoid trying to become a deep learning expert because it seems like people have that covered and it might take me five years. Lol. So I am trying to think of GPU programming approaches that are new. Which most likely will turn out to be a waste of time but will certainly be interesting for me.
- Carzuckermack 7y agoIt's as if Carmack is simply paid to shill for Zuckerberg, like some pet under glass. Every time Carmack opens his fucking mouth, and it's the facebook domain in the URL bar, there's no accident in that. Carmack transmits this nerd signal, knowingly. Kind of disgusting.
- binarymax 7y agoDoes anyone have a non-facebook version of this story?
- curiousgal 7y agoStarting this week, I’m moving to a "Consulting CTO” position with Oculus. I will still have a voice in the development work, but it will only be consuming a modest slice of my time. As for what I am going to be doing with the rest of my time: When I think back over everything I have done across games, aerospace, and VR, I have always felt that I had at least a vague “line of sight” to the solutions, even if they were unconventional or unproven. I have sometimes wondered how I would fare with a problem where the solution really isn’t in sight. I decided that I should give it a try before I get too old. I’m going to work on artificial general intelligence (AGI). I think it is possible, enormously valuable, and that I have a non-negligible chance of making a difference there, so by a Pascal’s Mugging sort of logic, I should be working on it. For the time being at least, I am going to be going about it “Victorian Gentleman Scientist” style, pursuing my inquiries from home, and drafting my son into the work. Runner up for next project was cost effective nuclear fission reactors, which wouldn’t have been as suitable for that style of work.
- hedvig 7y agoFusion John, we need you in fusion research. It's only 10 years away
- madacoo 7y agoIs there a non-Facebook source for this information?
- whywhywhywhy 7y agoSad to see him leave VR but honestly I feel shipping the Quest was a huge achievement and is a product that sums up what his original vision for VR was and the only VR device so far with any mainstream potential
- ralusek 7y agoI think any monumental leap forward is going to come from changing the way networks are motivated. Simple back propagation attempting to reduce the error from a fixed objective have proven to be useful, but doesn't actually resemble how more generalizable intelligence works. I think getting the feedback loop integrated with something that behaves more like dopamine/serotonin/pain feedback is going to be the likely direction we'd need to go. Basically, the network needs to be able to form new objectives and recognize when it's meeting or failing at those objectives, rather than just optimizing its network to be less and less bad at predicting specific outputs.
- deleted 7y ago[deleted]
- reilly3000 7y agoArguably he got started on this years ago. The story goes that Quake 3's AI was so sophisticated, that when left alone for 4 years on a bot vs bot server, the units learned pacifism as a self-preservation strategy. https://www.forbes.com/sites/erikkain/2013/07/02/quake-iii-arena-bots-reportedly-stop-fighting-after-4-year-match/#6d05c58a6233 https://www.forbes.com/sites/erikkain/2013/07/02/quake-iii-a...
- moogly 7y agoMr. Elusive (Jean-Paul van Waveren) was the guy who wrote the bot code for Quake 3: Arena, not Carmack.
- hombre_fatal 7y agoSeems like this would be easy to actually test instead of rely on a single anecdote. And the simplest explanation is a bug though I don't believe the story at all. There are probably multi-year-uptime bot servers in the wild right now. Reminds me of when I was new to Quake 3 and found an amazing server: it was always full of players and full of non-stop action. I played with these people all the time after school. Nobody said anything, they were 100% business which was cool. I would often congratulate them on nice kills or commentate on my victories. Everyone was about the same skill level. Eventually I realized I was playing on a server that simply filled empty slots with bots. I was the only human player.
- deviantfero 7y agoThis reads like a horror story prompt of some kind, eerie!
- Qu3tzal 7y ago"learned pacifism as a self-preservation strategy" this is an interpretation, not what the bot intended to do, in other words, you're reading into it
- xeeeeeeeeeeenu 7y agoThat story is fake, its source is 4chan. Carmack himself debunked it[1] [1] - https://twitter.com/id_aa_carmack/status/352192259418103809 https://twitter.com/id_aa_carmack/status/352192259418103809
- 2OEH8eoCRo0 7y agoI'm ready for this post to disappear, strange things start happening in cyberspace, and Carmack abruptly cans the project and doesn't want to talk about it.
- zelly 7y agoElaborate
- narrator 7y agoHumans optimize for activating the opioid receptor. These receptors are distributed all over the brain and tied into all sorts of subtle neural networks. That's why opioid addicts don't do much when they're high. As far as the entire structure of the brain is concerned, an opioid addict's brain is done optimizing and the fitness function is pegged at 1. I think an AGI will end up being like an AI that plays the Sims except we're the Sims and it's optimizing for our happiness probably by remotely monitoring our opioid receptor activation and some parameters of general health.
- sneak 7y agoSo, opiate synthesis and delivery?
- narrator 7y agoI imagine the AGI system of the future would not make humans happy via that method in that its decision would be counteracted by including physical health indicators in the fitness function.
- stevenwoo 7y agoIsn't this backwards a bit or did I get it wrong - not sure if I am phrasing this right. But to simplify from a laymen's perspective a reading of The Selfish Gene - all creatures will work to continue their genetic line unless misdirected as you describe. So if in some way continuing their genetic offspring is something that activates their opioid receptor, that's what we'll do, creating and raising offspring or as grandparents/cousins/aunts/uncles, helping raise relatives. This can be applied generally to all life on earth.
- throwawayhhakdl 7y agoSuccessful propagation strategies propagate, and therefore become prevalent. Life has no goal; it doesn’t try to propagate. It’s just the result of an evolved process that happens to correlate strongly with propagation. Most processes in organisms have an arbitrary first order objective like getting those happy chemicals. The fact that these first order objectives have a second order effect of survival and reproduction is a wholly unsurprising coincidence.
- soulofmischief 7y agoI'm happy. Carmack has done a lot of good with Oculus, but it hurt seeing my hero working for Facebook. I understand the need to remain in a consulting position, but at least now he's no longer under the corporate leash. If anyone can hack AI, it's Carmack, and so when I read this headline I had a moment of fright thinking this meant Carmack was working on AI for Facebook.
- Voloskaya 7y ago> If anyone can hack AI, it's Carmack Pretty sure you don't get to AGI with some hacks.
- soulofmischief 7y agoDon't be trite; You're well aware that hack in this context means to understand/make progress.
- Voloskaya 7y agoNo, I actually understand it as solving it with unexpected tricks.
- cma 7y agoLike relu?
- soulofmischief 7y agoWell, now you can understand it differently.
- Nimitz14 7y agoHm. I don't feel like additional engineering expertise is what's missing to achieve AGI, there's still a lot more science to do I think, and I'm not sure how good Carmack is at that.
- tus88 7y agoWhat this title should have been: John Carmack gives up on VR.
- Keyframe 7y agoFollowing up on “John Carmack gives up on aerospace” and hopefully not followed up by “John Carmack gives up on AI”. I like the guy as much as most, but so far it seems like he has been wandering around. He’s had much success in the early days of 3D video games and that’s about it. A guy of his calibre would probably make a good impact if he joined one of the expert teams like DeepMind. No matter how smart he is, AI today is a completely different ballgame then what he was part of so far. I hope I’m wrong, but I don’t see him making any sort of breakthrough on his own, with his son. Maybe he wants to spend more time with his family, which is great, or he drank koolaid about his own legend. Odds are against him, heavily so. Good luck, in any case.
- mnd999 7y agoOr John Carmack (pretty much) gives up on Facebook
- jacquesm 7y agoAGI is not an engineering problem but a research problem. John Carmack is good at putting stuff together but how good he is at coming up with novel concepts for an open research problem remains to be seen. Even the rocketry example that is hailed here as a success mostly wasn't. That doesn't make me happy, it would have been far nicer if Armadillo had succeeded, more competition in that space is better. But for all the work done it was more of an advanced hobby project along the lines of those guys in the Nordics than something that moved the needle scientifically.
- drcode 7y agoCarmack is someone who has proven to be an almost unequalled productivity machine when working on medium-difficulty problems... Now, for the first time, we'll see if his approach to problem solving can also work on a truly difficult problem. I agree it's very much an open question.
- ineedasername 7y agoIs that really true though? It seems more like he's good at a medium difficulty problems in a narrow subdomain of software development, which is saying something a bit different. I might even say he's good at hard problems within that subdomain. How transferable those skills are is the most salient point. Assuming peak genius level intellect (which, I don't know, maybe?) It would still take something like 4 or 5 years to reach expert level knowledge in such a complex domain.
- randcraw 7y agoAgreed. Deep learning has revolutionized AI and anyone hoping to contribute to AGI is going to have to master DL first, and probably a lot more AI like a variety of probabilistic methods. That's a challenging learning curve that's not much different from earning a PhD. And then, to stand out in AGI, you're going to have to integrate a dozen kinds of cutting edge components, none of which are anywhere ready for prime time. At this moment in time, I think any attempt at implementing AGI is going to be half-baked at best. For now, a Siri / Alexa that can do more than answer single questions will be challenging enough.
- sabujp 7y agothere's no "software" alone that can create an AGI. It's going to take a mix of really powerful hardware, high speed interconnects, and software coming together to do something anywhere close to what sentient (human) beings can do. Life isn't (yet) star trek and carmack isn't noonian soong. Current approx say the human brain is ~10^15 flops, there are only a few of these supercomputers in this regime, with only Summit reaching into the exaflop regime. Carmack would need full access to all of any of the petaflop machines. What he'll probably come up with instead is probably some new good algorithms for making things look like AGI.
- taurath 7y agoPeople familiar with the ethics around general AI - should we be worried?
- daenz 7y agoRoko's basilisk[0] will remember this. 0. https://rationalwiki.org/wiki/Roko's_basilisk https://rationalwiki.org/wiki/Roko's_basilisk
- mactyler 7y agoCarmack might just be a basilisker after all!
- antisemiotic 7y agoI wonder what would happen if Carmack teamed up with Yudkowsky and his cult; Carmack's practicality with the imagination of people who non-facetiously talk about acausal deals with mulitiversal AIs could be a match made in heaven.
- deleted 7y ago[deleted]
- nwsm 7y agoWhat a joke. Carmack is going to sit at home and solve what teams of scientists can't do in decades. I'm complaining less about Carmack wanting to spend his time doing this and more about the comments here acting like he is some 10000x research scientist.
- kyle-rb 7y agoNo one expects him to emerge with a fully formed AGI. By through experimenting he might contribute some new incremental but still useful improvements.
- klyrs 7y agoThis feels somewhat obligatory... http://ars.userfriendly.org/cartoons/?id=20001018 http://ars.userfriendly.org/cartoons/?id=20001018
- drcode 7y agoThere are few AI researchers (maybe around 5 or so) that could credibly claim technical accomplishments of any sort in the same ballpark as Carmack's. People with this level of track record should not be underestimated, there aren't many of them out there... They matter.
- Barrin92 7y agoCarmack has had a lot of commercial success, which is great and I value his creativity in the space but working on a research question that is about ten paradigm shifts away is a different task than putting in hard labour to build games.
- taurath 7y agoAnd maybe the output is he helps bring about one of those ten paradigm shifts, which would be a wonderful success. He didn’t say “solve” it, he said work on it.
- robotresearcher 7y ago
- mark_l_watson 7y agoGood for him. For AGI development, I wonder if the behavior of people playing multi player VR games on the Oculus platform could be used as training data.
- Ono-Sendai 7y agoCool, I hope he makes blog posts or videos or something, so we can follow along at home.
- nwsm 7y agoYann LeCun's [0] comment on the post: Welcome to the club, John. A word of warning though: There is no such thing as AGI. Reaching human-level AI is a good goal. But human intelligence is very, very specialized. [0] https://en.wikipedia.org/wiki/Yann_LeCun?fbclid=IwAR2e9mzCqS48-OGTH65yiVwO27rhjGC065GxudLetpOOAAvuIN6kcGC5EUM https://en.wikipedia.org/wiki/Yann_LeCun?fbclid=IwAR2e9mzCqS...
- drcode 7y agoLeCun is more accomplished and smarter than I will ever be, but his thoughts on the term 'AGI' just seem like dumb pedantry regarding word definitions to me.
- lonelappde 7y agoPerhaps you should examine your seemings then.
- username90 7y agoSounds like it is more like an wishful thinking from a person who spent his life mastering deep nets than a nitpick. Like "Sure deep nets can never be an AGI, but Humans are not AGI's so we can reach human level performance with deep nets, it is not a dead end! IT IS NOT A DEAD END!". I think the saying "Science Advances One Funeral at a Time" applies here.
- michannne 7y agoSounds exactly like what I would say if my life's work revolved around the idea that AI boils down to how many neurons we can simulate
- objektif 7y agoFYI I also just decided that I am going to work on fusion.
- lazyjones 7y agoLet's hope Boston Dynamics doesn't hire him. ;-)
- zhoujianfu 7y agoI’ll take this opportunity to just put out some thoughts I have about intelligence, wisdom, AI, and AGI. In general, building/switching contexts in your head takes intelligence (and the more intelligent you are, the better/faster you are at it), whereas already having a context in your head is wisdom. I think of the current state of AI as us being able to teach computers a few very specific contexts, i.e. imparting wisdom to them. An AGI would be actually creating intelligence. And they are not the same thing at all. In fact, some might say your conscience/soul is just this brain context switcher/creator in action. An AGI would have consciousness.
- codesushi42 7y agoThis would be great if only he was not at Facebook.
- piinbinary 7y agoWhat's a useful definition of what AGI is? If you have a computer that does some clever AI stuff, what criteria do you look for to decide that it has a general intelligence?
- lonelappde 7y agoAGI means it can solve any problem posed to it and interface with any sensory and motor peripheral.
- drdeca 7y agonot every problem. Not expected to e.g. solve the halting problem. If it can solve every problem that a human can solve, then it would be AGI. If it can solve every problem that a somewhat unintelligent human can solve, I think that would still count as AGI.
- dingo_bat 7y agoAn average human would realize it if you pose the halting problem to him. I'd expect the same from an AGI. To recognize such a problem and refuse to continue to devote time solving it.
- olalonde 7y agoIt's generally used to mean human level intelligence (e.g. an AI that can learn any task a human can). An often cited criteria is passing the Turing test.
- username90 7y agoThe Turing test is too easy. My test would be when the AI can apply to a random remote job, get it and then receive salary for a few years, quit, apply to a new job with updated CV, get a raise etc without anyone noticing it is an AGI.
- ijiiijji1 7y agoWell, the holy grail is general AI that can also write software and develop hardware... then it can develop itself.
- chj 7y agoBold undertaking even for someone as distinguished as John Carmack. I can't help thinking that this is too broad a goal and requires more theoretical work than engineering. Maybe making some self-replicating robots that can mate and give offsprings would be less ambitious (perhaps I am underestimating as well).
- jobigoud 7y agoHe just gave the general domain of the project, it would be weird to start this kind of semi-retirement on a very specific domain. I see this as the broad topic of interest that he will be investigating, but any actual research will be done into narrow fields, wherever some interesting idea hasn't been fully explored yet.
- boomboomsubban 7y agoI have no idea what he is capable of adding to the field, but I'm quite happy to see a long time proponent of free software enter the field.
- mentat 7y agoHe invented modern graphics as a practical problem by himself as the sole researcher. Given the tools at the time that may have been a harder problem.
- justin66 7y ago> He invented modern graphics as a practical problem by himself as the sole researcher. No, he didn't, and that is not a claim that he would ever make himself.
- pixelpoet 7y agoAgreed, and if anyone could make that claim it'd be Eric Veach (who then went on to develop Google Adwords).
- sterlind 7y agoYes, a practical problem. The math behind computer graphics (i.e. optics) had been around hundreds of years. The trick was using numerical analysis to optimize and approximate on limited hardware. We don't have the laws of AGI like we had the laws of optics (Asimov notwithstanding.) Tons of research effort was poured into the wrong avenues in vision (hand-tuned HoG, transforms, optical flow analysis) and ML (support vector machines, computational learning theory) until a chain of breakthroughs hit on the right mathematical approach for vision and supervised learning more generally. We have some mathematical approaches to try with AGI (e.g. policy optimization/max-Q in reinforcement learning), but they equations are plagued with fundamental issues (e.g. reward sparcity, easily-gamed artificial objectives.) Carmack optimized some very difficult equations when he worked on graphics, but in AGI we still don't have the right equations to optimize.
- bobsil1 7y agoA dog brain can't design an artificial dog brain, and a human brain probably can't design a human-level AGI. It will likely be machine-evolved on cheap, massively parallel hardware, with the key problem being speeding up evolutionary search.
- cf 7y agoI expect John Carmack to follow a trajectory of someone like David Ha who with little previous background started to write very creative and thought provoking papers (https://scholar.google.com/citations?user=J1j92GsxVUMC&hl=en https://scholar.google.com/citations?user=J1j92GsxVUMC&hl=en) It won't be AGI by most definitions but I bet it'll be pretty cool and I'm happy to have that.
- trentnix 7y agoEverything John Carmack touches he makes better. AGI will be no different.
- fizixer 7y agoI would love to do the same, except I'm not at a point where I'm financially independent.
- sriku 7y agoI'm wondering whether this will set off a wave of folks working on AGI given that today the tools are way more accessible.
- perseusprime11 7y agoFor somebody like John Cormack, I read this as 'I am retiring'
- leesec 7y agoWow, there are a lot of people in this thread arguing whether or not John Carmack has the right skills to help AGI, or about the specifics of his knowledge. Do you all realize you're arguing about nothing? Good for him for doing something he seems excited about. Maybe we should all stop gossiping and go do something we're excited about too.
- libraryatnight 7y agoMy thoughts exactly. This is a man with a long and brilliant career. Let him work on what he wants. This comment thread is ridiculous.
- voxl 7y agoWhat is actually being argued is the cult worship of a particular engineer. I am against cult worship, so find the fact that Carmack is entering this space simply not news worthy. Yet here it is. Do I wish him the best of luck and hope he cracks the problem? Of course, all the same I would wish that of an upstart PhD student. Yet, the announcements of a brilliant PhD student attending a university to work on AGI is somehow not on hacker news. This is cult worship of the personality Carmack has amassed, perhaps completely accidentally. When Carmack actually achieves something interesting let us discuss it then, not the mere announcement that he will try, as if that means anything. Read: it doesn't.
- prvc 7y ago>For the time being at least, I am going to be going about it “Victorian Gentleman Scientist” style, pursuing my inquiries from home Who will fund the necessary computing resources? If not FB, then he will surely be joining or starting a different org
- haihaibye 7y agoIn Victorian times, a "gentleman" was someone who had so much money they didn't need to work. He's been a major shareholder in 2 companies that have been acquired (Id and Occulus)
- Animats 7y agoThis is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. They have to operate in a world, interact with others, survive, and accomplish goals. Simulator technology is now good enough that you can do quite realistic worlds. Imagine The Sims, with a lot more internal smarts and real physics, as a base for work. Robotics has the same issues, but you spend all your time fussing with the mechanical machinery. Carmack is a game developer; he can easily connect whatever he's doing to some kind of game engine. (Back in the 1990s, I was headed in that direction, got stuck because physics engines were no good, made some progress on physics engines, and sold off that technology. Never got back to the AI part. I'd been headed in a direction we now think is a dead end, anyway. I was trying to use adaptive model-based control as a form of machine learning. You observe a black box's inputs and outputs and try to predict the black box. The internal model has delays, multipliers, integrators, and such. All of these have tuning parameters. You try to guess at the internal model, tune it, see what it gets wrong, try some permutations of the model, keep the winners, dump the losers, repeat. It turns out that the road to machine learning is a huge number of dumb nodes, not a small number of complicated ones. Oh well.)
- nikki93 7y agoYeah I believe in this game / simulated world NPC idea too. To get the kind of complexity we want we either need sensors in the real world or interfacing in a virtual world that humans bring complexity to (probably both -- the humans are part of the sensing technology to start). Things like AlphaZero etc. got good cuz they had a simulatable model of the world (just a chess board + next state function in their case). We need increasingly complex and intetesting forms of that. In some sense you can think of interfacing w/ the online world + trying to win attention to yourself as the kind of general game that is being played.
- Animats 7y agoI've long taken the position that intelligence is mostly about getting through the next 10-30 seconds of life without screwing up. Not falling down, not running into stuff, not getting hurt, not breaking things, making some progress on the current task. Common sense. Most of animal brains, and a large fraction of the human brain, is devoted to managing that. On top of that is some kind of coarse planner giving goals to the lower level systems. This area is under-studied. The logicians spent decades on the high level planner part. The machine learning people are mostly at the lower and middle vision level - object recognition, not "what will happen next". There's a big hole in the middle. It's embarrassing how bad robot manipulation is. Manipulation in unstructured situations barely works better than it did 50 years ago. Nobody even seems to be talking about "common sense" any more. "Common sense" can be though of as the ability to predict the consequences of your actions. AI is not very good at this yet, which makes it dangerous. Back when Rod Brooks did his artificial insects, he was talking about jumping to human level AI, with something called "Cog".[1] I asked him "You built a good artificial insect. Why not go for a next step, a good artificial mouse?" He said "Because I don't want to go down in history as the man who created the world's best artificial mouse". Cog was a flop, and Brooks goes down in history as the inventor of the mass market robot vacuum cleaner. Oh well. [1] http://people.csail.mit.edu/brooks/papers/CMAA-group.pdf http://people.csail.mit.edu/brooks/papers/CMAA-group.pdf
- dreamcompiler 7y agoIt's nice that he's doing this and I wish him well. But thousands of very smart people have been working on AGI since 1958 and many of them initially thought they could crack the problem in a couple of years. They made a huge amount of progress, but the more progress we make toward AGI, the farther away the goal seems to recede. AGI is very much a research problem. It's not going to be solved with a clever hack.
- xamuel 7y agoI've been dabbling in AGI and it seems like the field has a lot of low-hanging fruit. I'll bet Carmack can offer some significant contributions. I'll take an opportunity to plug a paper I recently published on comparing relative intelligence. The punchline will illuminate the low-hangingness of the fruit in this field. Suppose X and Y are AGIs and you want to know which is more intelligent. For any interactive reward-giving environment E, you could place X into E and see how much reward X gets; likewise for Y. If X gets more reward, you can consider that as evidence of X being more intelligent. But there are many environments, and X might do better in some, Y in others. How can you combine those pieces of evidence into a final judgment? The epiphany I had (obvious in hindsight) is that the above situation is actually an election in disguise. The voters are interactive reward-giving environments, voting (via their rewards) in an intelligence contest between different AGIs. This allows us to import centuries of research on voting and elections! In particular, by using theorems about elections published in the 1970s, I was able to provide an elegant notion of relative intelligence. The notion I provided is elegant enough that some theorems can even be proved with it, for example, formalizations of the idea that "higher-intelligence team-members make higher-intelligence teams". Which emphasizes the low-fruit-hanginess of the field: as obvious as that idea seems, apparently no-one was able to prove it with previous formal intelligence measures, probably because those previous intelligence measures were too complicated to reason about! Here's the paper: https://philpapers.org/archive/ALEIVU.pdf https://philpapers.org/archive/ALEIVU.pdf
- mycall 7y agoHow did you rule out other attributes in the voting, such as charisma, political environment, society wellness, etc?
- xamuel 7y agoThe "voters" are not humans, but environments. A simple example of an environment would be a room with two buttons, one of which rewards you when you press it, and one of which punishes you when you press it. Independently of anything like charisma, society wellness, etc. If X figures this out quicker and pushes the "reward" button more often than Y, then we consider this simple environment to "vote" for X in the intelligence contest. (See the paper for the formal definitions.)
- yters 7y agoHe should first work on figuring out if AGI is possible. Why assume the human mind is computable?
- jobigoud 7y agoThat's exactly what a scientist would say. Engineers invent things because nobody told them they were impossible.
- yters 7y agoOr they go on wild goose chases. We call those people cranks, and there are plenty of them, such as all the people inventing perpetual motion machines (which arguably could be AGI). Also, if you address the fundamental question first, Carmack may unlock something even more powerful than AI. As it is, he's just following the crowd.
- 29athrowaway 7y agoThe more people working on this, the better. I don't expect him to crack the problem but contributions are welcome. John is smart, has a reputation, has resources, is well connected and has time. I hope he suceeds.
- unityByFreedom 7y agoAnother one bites the dust. You only need to do a few problems on Kaggle to know AGI is a long, long way away, if achievable at all. AGI proponents tend to claim that we know everything about physics and biology, and that replicating it is feasible. This is science fiction. There are much more pressing concerns in the AI space. Godspeed Carmack.
- bane 7y agoI'm surprised nobody has made the connection with the announcement of Horizon. Carmack joined Oculus in order to build the next generation of 3d Virtual worlds. It's done, other than some polish, which means he's not needed anymore. However Horizon turns out (I'm bearish on it tbh), Carmack has had his shot to build the digital future, it's now turned out how it has, and there's not much flexibility left for him to maneuver, it's time to move on. I think AGI is going to turn out like his shot at Rocketry, big and complex enough that he'll find his niche and contribute, but not make any significant breakthroughs.
- tibbydudeza 7y agoAlways liked him ... humble dude .. will he give us the fast inverse square root for AGI ???. But I suspect that "intelligence" is not so structured and reducible as we would like .. it somehow just works.
- iamleppert 7y agoIt must be hard to be someone like him. Always chasing the original high that made him famous, the enormous pressure to succeed must be nauseating. Instead of retiring and relaxing and looking back on an impactful and lucky career, it says something about how powerful the original emotions were that led him to his current point. He will do anything to get back to that state, that place in time, even sacrifice what are supposed to be the good years of his life stuck behind a screen.
- neonate 7y agoHow do you know he isn't just doing what interests him? Not everyone wants to retire, especially highly creative people.
- dfischer 7y agoHis interview on JRE was pretty enlightening. He definitely has a polymath background to navigate towards the right direction. I believe he’ll be working closely with Musk and Neuralink.
- unityByFreedom 7y agoOne could imagine him beginning to work in gaming AI, but he says he wants to work on AGI, not simply gaming AI. I'm personally not convinced that it is encouraging when someone bright sets their sights on AGI, particularly someone who appears to have never competed on Kaggle. It screams hubris.
- bitexploder 7y agoI am not convinced current AI is the approach to an AGI. I think it is at least feasible someone outside of this sphere has a reasonable shot at it. It feels like many AI researchers get caught up in refining existing techniques that amount to fancy statistics algorithms and data crunching, but not AGI. Current AI techniques may be synthesized or used in part in some AGI but it’s clear there is a revolutionary step to be made. Kaggle is almost just an optimization fest, and not really advancing towards AGI.
- TeamSlytherin 7y agoThe announcement today has the potential to cut the number of years estimated (to reach AGI) in half, and I'm sure VC funds are DM'ing him non-stop right now. But we don't know which path Carmack will take, and as you rightly point out, current trends in AI is mostly ML/xNN's with a goal of turning data into profit (heavy focus on products/markets). Even those talking about AGI are fractured into different groups. Oddly, many in ML are talking about future abilities that are mostly defined by AGI research (even if they see AGI as a distraction. From his post, I don't think "product development" is a goal. Not clear what challenges or milestones he will set for himself (just having a better testing suite has become an AGI issue of late, so maybe he will contribute to that first).
- unityByFreedom 7y ago> I am not convinced current AI is the approach to an AGI This is a non-sequitor. I didn't argue that current AI theory, or Kaggle, will lead us to AGI. > Kaggle is almost just an optimization fest Public machine learning competitions have produced a lot of innovative learning techniques. If Kaggle is so easy, and AGI so hard, it would follow that anyone tackling AGI would have some experience applying machine learning competitively in some public space. It doesn't necessarily need to be Kaggle. Kaggle just happens to be good at hosting such public competitions, and in fact has surfaced several state-of-the-art implementations. The difference between prize money (~top 5) and no-prize-money (5-10) may be an "optimization fest", but without the competition, the solutions presented would have entered the public sphere at a much slower rate. EDIT: Please be kind and explain your downvotes.
- kache_ 7y agoI can see where he's coming from. He's in the unique position where he can work on whatever interests him, he has the mathematical background to absorb all this knowledge. I think a big part of this is that he's worried about his age catching up to him and disabling his ability to make any meaningful contribution. One thing to note is that he's going to loop his son into his research. The creation of synthetic intelligence will be a result of multiple distinct breakthroughs. The more people with unlimited resources and high creativity working on this problem, the more likely those breakthroughs will be made.
- zerr 7y agoSeems like a nice excuse for an early retirement and spending more time at home with a family :)
- prando 7y agoI am reminded of his programming retreat :). https://www.facebook.com/permalink.php?story_fbid=2110408722526967&id=100006735798590 https://www.facebook.com/permalink.php?story_fbid=2110408722...
- avl999 7y agoHalf of you folks here are like teenage girls in highschool gossiping with each other about Kristen dumping Drew and instead trying to date Paul.
- tehjoker 7y agoIt's nice being rich.
- maxpert 7y agoWhy do I have a feeling that Oculus is gonna be dead?
- Rerarom 7y agoReminds me of Paul Cohen working on the Riemann hypothesis.
- joeevans1000 7y agoCarmack likes lisp. I wonder if he'll be using it on this project.
- mantoto 7y agoSo apparently Mark Suckerburg reacted to it and 3000 other people. Anyway he should probably go to Google for this. They look way more advanced than all others. And not sure how much money he has but there should be ml involved and that costs a pretty penny
- Jaruzel 7y agoCarmack's post in full: Starting this week, I’m moving to a "Consulting CTO” position with Oculus. I will still have a voice in the development work, but it will only be consuming a modest slice of my time. As for what I am going to be doing with the rest of my time: When I think back over everything I have done across games, aerospace, and VR, I have always felt that I had at least a vague “line of sight” to the solutions, even if they were unconventional or unproven. I have sometimes wondered how I would fare with a problem where the solution really isn’t in sight. I decided that I should give it a try before I get too old. I’m going to work on artificial general intelligence (AGI). I think it is possible, enormously valuable, and that I have a non-negligible chance of making a difference there, so by a Pascal’s Mugging sort of logic, I should be working on it. For the time being at least, I am going to be going about it “Victorian Gentleman Scientist” style, pursuing my inquiries from home, and drafting my son into the work. Runner up for next project was cost effective nuclear fission reactors, which wouldn’t have been as suitable for that style of work. -- We're at 500 comments at the time of posting this, and no-ones pasted his post in full to save us having to visit Facebook...
- arcturus17 7y agoThanks it didn’t so much save me as it enabled me, since I have it blocked.
- bounded_agent 7y ago+1
- saiya-jin 7y agosame here, can't access it at work, at its damn interesting topic for many of us
- _hao 7y agoI have FB blocked so thanks for sharing!
- davidgrenier 7y ago
- _pmf_ 7y agoI just noticed that this is the first Facebook post I ever read.
- wruza 7y agoQ: Why did John Carmack not worry when his AGI decided to leave a note and go out?
- delegate 7y agoSo the creator of Doom wants to get into artificial general intelligence .. what could possibly go wrong :)).
- phtrivier 7y agoTotally off-topic, but: > Runner up for next project was cost effective nuclear fission reactors, which wouldn’t have been as suitable for that style of work. What would a (high-level) carreer path for that event _look_ like ? (Disclaimer: I'm not Carmack-level smart. Not sure I'm anyone-s-level smart. Asking for a friend.)
- tomerbd 7y agowhy is this so important why do you consider him such a genius?
- protomikron 7y agoWow, that post blew up. Like most engineers I respect JC for his incredible work, but I really think AGI is far off and at the moment would be very surprised if there is significant progress in the next years to come. I also want you to read this extremely well written (old) blog post about the topic and I don't think much has improved since: https://karpathy.github.io/2012/10/22/state-of-computer-vision/ https://karpathy.github.io/2012/10/22/state-of-computer-visi...
- somewhereoutth 7y agoMathematically speaking, current computers run in discrete (integer/countable) space, and the human brain exists in continuous (real/uncountable) space. Cantor showed that continuous space is larger than, and thus cannot be represented in, discrete space (via diagonal method). I suggest that AGI and consciousness lie in continuous space, and thus are unreachable with our current discrete computation model, regardless of how sophisticated we make it. There exists a cardinality barrier.
- hvasilev 7y agoI wonder how these people have decided it is possible with a digital computer, given that the brain is analog. I wouldn't be surprised if the first step through AGI is to actually crack the analog computer hardware problem.
- shrimpx 7y agoUnrelated to AGI, but quantum physics doesn't admit 'continuity'. Continuity is an abstraction. There are no real numbers in reality.
- sebsito 7y agoFirst of all we should at least have a common definition what intelligence even is. Even then I'm not sure we'd know what General Intelligence would be because all we know is Human Intelligence or maybe lower level Animal Intelligences where the problem solving mechanism seems to depend on biological body and it's form. Humans navigate the world with automatic impulses which we evolved over time to deal with way too much signals from the environment so we can filter and react only to those important. We can then use consciousness to slowly map new impulses as the environment changes and go back to autopilot for most of the time. What if our intelligence isn't general but it's just enough to navigate the world we can perceive with out senses? What if we'll never be able to understand e.g. the quantum theory (or at least the part of a world experience which we call this way)? If there's is superset of out intelligence or different sets of intelligences which we just don't undestand? We think that our problem solving can take on any problem but maybe we're only taking on the problems we can take on, limited to our perception of reality which can be limited? So I think instead of calling it AGI the name should be more like Artificial Human-like Intelligence.
- dwheeler 7y agoI am skeptical that he will succeed, but I am happy that he will try. Carmack is smart, and we need smart people to try hard things.
- dharma1 7y agoWhy the announcement? Is it to protect Oculus/FB so it doesn't look like he's bailing out of VR? Couldn't he just have said "I'm taking some time off" and then make an announcement when there's something to announce, ie. "So here's some progress I've made on AGI"
- phyzome 7y agoAh yes, AGI: The idea that eats smart people. https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm
- marvin 7y agoWorking towards more general AI technology is not what the author of this is criticizing. They're criticizing focusing on super-human AI rather than ethical problems of current AI. "The idea that eats smart people" is just a meme at this point, normally quoted outside its context. But even so, history will have to judge whether the author's statements were true regarding the risk of super-human AI. Or whether a lot of _quite_ smart people weren't smart _enough_ to realize that there was a real likelihood of this being possible to achieve faster most thought. Also, using ridicule as a rhetorical technique isn't the most sound type of reasoning, regarding the author of your link ;)
- flipgimble 7y agoLike many others posting here I’ve followed John’s work frOm the mid 90’s so here is my take: * he has an exceptional quality of cutting through the bullshit and shipping practical software, which is arguably what the vague and uncertain field of AGI needs. If you listen to AGI conference talks in recent years they are focused on aspirational single-idea academic frameworks that haven’t produced results in decades. * he is still connected to Facebook with billions in resources, and a world class ML team with Yann LeCun at the head. * his personal brand has been strong enough to have world class developers flock to Oculus. When he is ready to expand his “Victorian Gentleman” alchemy lab with a team, I have no doubt it would be a field-changing think tank. My hope is that he continues to be open and brutally honest with his progres and learnings as he’s been with game development and rocketry.
- hexdrunker 7y agoNice
- bigred100 7y agoIs there any reason to expect he’ll have any success at this? Leaving aside the motivation for a fun hobby that you don’t expect anything out of.
- foobiekr 7y agoI’ve worked with legends in a specific space that is less consumer-y than games and graphics and less weirdly-desperate attention-seeking than ML and so the people therein are every bit Carmack-level but less visible. As they have aged out and wealthy’d out of working time, almost all of them have chosen to retire. These are extremely high performing individuals who have made global impact. Shutting down for people like this is very hard and 100% of them have sent out mails much like Carmack’s Facebook post when the end came. Even the style and verbiage are similar. None of them made a dent in their tinkering-phase project and moved on to normal above-average low-engagement hobbies. They are done. I read his FB post as a pretty standard retirement announcement as a result. I think he’s telling us he is done.
- mav3rick 7y agoYou cant project your colleagues on a third person.
- foobiekr 7y agoYou can observe common properties of the way people at similar skill levels behaved however.
- mav3rick 7y agoHow do I know op's standards are the same as Carmack's ? What I consider great maybe average after all ?. Same could have been said when Carmack said he wanted to work on VR. "it's a retirement letter"
- matz1 7y agoWhy? Its a reasonable assumption.
- staz 7y agoby that standard Carmack retired a few times already and always came back with something interesting
- 7y ago
- acollins1331 7y agoMaybe general intelligence is not a goal to pursue, isn't general intelligence of machines what would eventually lead to the 'singularity' that seems far away now but is plausible in theory? Maybe John's real last name is Connor.
- negamax 7y agoI have seen what happens next in Blackmirror!
- playing_colours 7y agoBased on the interviews with Jon Carmack, he is very intelligent, realistic, reflective. Therefore, he realistically estimates his skills and the complexity of a problem. He said he would possible make an impact - and he can - by potentially attracting more attention to the field from researchers and investors, by building tools, that would help in researching the problem, by, at least, showing promising paths. He may start to dig into the problem in “Victorian Gentleman Scientist” style, but then attract investments and scale up. We can speculate and guess - time will show, fingers crossed. This “Victorian Gentleman Scientist” style is something I am longing for. I cannot go back to the academia now with family, or spend large chunks of my time on any research, but I really want to be able to afford it. Sure, most probably, I ll become soon disillusioned with the routine of a researcher, or jump between topics of research, or just did not contribute anything meaningful, but I'd really wish there was a possibility for me, other people to afford such lifestyle.
- cvaidya1986 7y agoMe too.
- program_whiz 7y agoMaybe I am a naysayer, but John Carmack is going against other companies and teams with not only billions in funding for hardware and data, but also with big staffs of experts. Not only are people who have been in the research for a while more likely to have good ideas, but having the support of engineers to write tests and data wrangling, Neuroscientists to bounce ideas off of, and a whole bevy of support staff is just more likely to produce results. I'm also not a big fan of the kind of hero worship of "well he wrote Doom, so this should be a cakewalk." I'm not saying he won't, but he probably won't. What am I missing here that everyone seems all hyped up about?
- coolassdude6941 7y agoHis “competitors” are irrelevant. It’s obvious that the existing approach (dl/rl) is a dead end, at least for AGI. So the idea of a possible genius working on a new approach with no monetary incentive is exciting.
- krick 7y agoA slightly more on point title would be "John Carmack steps down as Oculuc CTO". The rest is pure hype.
- daodedickinson 7y agoMy dream would be that Carmack would go toward Douglass Hofstadter and they'd move us toward forms that flawed human intelligence has been striving toward, rather than reifying flaws. I wish the people capable of helping with that could help with that and then I hope that the rest of us could all gather together to solve the loneliness CATACLYSM.
- laichzeit0 7y agoWe should begin by giving a clear definition of AGI. I’m talking about an unambiguous universally accepted definition akin to defining what a “continuous function” is. Until that point this is merely an argument about words, to paraphrase John Locke.
- MrZongle2 7y agoI don't know if Carmack has a good chance at achieving this goal, but I certainly wish him the best. There are certainly far worse applications of a sharp mind like his, and if this is where his passion has taken him then I'm sure he will be productive.
- deleted 7y ago[deleted]
- tonfreed 7y agoIs this going to be like the AI in Daikatana?
- tim333 7y agoThis is probably a great time to try to crack AGI. The cost and power of computing has just about become ok for the task and the published work out there like AlphaZero is getting kind of close but probably needs one or two major algorithmic changes to get there.
- alexashka 7y agoIt's interesting that no one has mentioned that this is not unlike Jony Ive changing roles within Apple. It's just John's way of saying 'I'm leaving Facebook' without affecting the stock price.