12 ms·
This is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. They have to operate
by Animats 7y ago
This is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. They have to operate in a world, interact with others, survive, and accomplish goals. Simulator technology is now good enough that you can do quite realistic worlds. Imagine The Sims, with a lot more internal smarts and real physics, as a base for work.
Robotics has the same issues, but you spend all your time fussing with the mechanical machinery. Carmack is a game developer; he can easily connect whatever he's doing to some kind of game engine.
(Back in the 1990s, I was headed in that direction, got stuck because physics engines were no good, made some progress on physics engines, and sold off that technology. Never got back to the AI part. I'd been headed in a direction we now think is a dead end, anyway. I was trying to use adaptive model-based control as a form of machine learning. You observe a black box's inputs and outputs and try to predict the black box. The internal model has delays, multipliers, integrators, and such. All of these have tuning parameters. You try to guess at the internal model, tune it, see what it gets wrong, try some permutations of the model, keep the winners, dump the losers, repeat. It turns out that the road to machine learning is a huge number of dumb nodes, not a small number of complicated ones. Oh well.)
- nikki93 7y agoYeah I believe in this game / simulated world NPC idea too. To get the kind of complexity we want we either need sensors in the real world or interfacing in a virtual world that humans bring complexity to (probably both -- the humans are part of the sensing technology to start). Things like AlphaZero etc. got good cuz they had a simulatable model of the world (just a chess board + next state function in their case). We need increasingly complex and intetesting forms of that. In some sense you can think of interfacing w/ the online world + trying to win attention to yourself as the kind of general game that is being played.
- Animats 7y agoI've long taken the position that intelligence is mostly about getting through the next 10-30 seconds of life without screwing up. Not falling down, not running into stuff, not getting hurt, not breaking things, making some progress on the current task. Common sense. Most of animal brains, and a large fraction of the human brain, is devoted to managing that. On top of that is some kind of coarse planner giving goals to the lower level systems. This area is under-studied. The logicians spent decades on the high level planner part. The machine learning people are mostly at the lower and middle vision level - object recognition, not "what will happen next". There's a big hole in the middle. It's embarrassing how bad robot manipulation is. Manipulation in unstructured situations barely works better than it did 50 years ago. Nobody even seems to be talking about "common sense" any more. "Common sense" can be though of as the ability to predict the consequences of your actions. AI is not very good at this yet, which makes it dangerous. Back when Rod Brooks did his artificial insects, he was talking about jumping to human level AI, with something called "Cog".[1] I asked him "You built a good artificial insect. Why not go for a next step, a good artificial mouse?" He said "Because I don't want to go down in history as the man who created the world's best artificial mouse". Cog was a flop, and Brooks goes down in history as the inventor of the mass market robot vacuum cleaner. Oh well. [1] http://people.csail.mit.edu/brooks/papers/CMAA-group.pdf http://people.csail.mit.edu/brooks/papers/CMAA-group.pdf
- visarga 7y ago> On top of that is some kind of coarse planner giving goals to the lower level systems. There are counterexamples, such as AlphaGo which is all about planning and deep thinking. It also combines learning with evolution (genetic selection).
- alpaca128 7y agoTrue, but AlphaGo is specialized on a very specific task where planning and deep thinking is a basic requirement for high level play. We don't need to think 10 "turns" ahead when trying to walk through a door, we just try to push or pull on it. And if the door is locked or if there's another person coming from the opposite side we'll handle that situation when we come across it.
- toxik 7y agoThat’s not true, human beings plan ahead when opening doors more than many things — should I try to open this bathroom door or will that make it awkward if it’s locked and I have to explain that to my coworker afterwards? Should I keep this door open for a while so the guy behind me gets through as well? Not to mention that people typically route plan at doorways. Doors are basically planning triggers more than many things.
- username90 7y agoHorses don't plan though, and they are much better than computers at a lot of tasks. If we can make a computer as smart as a horse, then we can likely also make it as smart as a human by bolting some planning logic on top of that.
- Mtinie 7y ago“Horses don’t plan though[...]” Can you expand on this statement? While I have no way to “debug” a horse’s brain in real-time, my experiences suggest they absolutely conduct complex decision-making while engaging in activities. Two examples which immediately come to mind where I believe I see evidence of “if this, then that” planning behavior: 1. Equestrian jumping events; horses often balk before a hurdle 2. Herds of wild horses reacting to perceived threats and then using topographic and geographic features to escape the situation.
- visarga 7y agoBoard games have been solved. Now the big boys are working on Starcraft and Dota 2, and it takes a shitload of money to pay for the compute and simulation necessary to train them. No something you can do on the cheap.
- toxik 7y agoThe next arena is multi task learning. Sure, I lose to specialized intelligences in each separate game, but I can beat the computer at basically every other game, including the game of coming up with new fun games.
- marcusverus 7y agoDeepmind's StarCraft AIs are already competing at the Grandmaster level[0], which is the highest tier of competitive play and represents the top 0.6 % of competitors. I am pleasantly surprised by how quickly they have been tackling big new decision spaces. [0]https://deepmind.com/blog/article/AlphaStar-Grandmaster-level-in-StarCraft-II-using-multi-agent-reinforcement-learning https://deepmind.com/blog/article/AlphaStar-Grandmaster-leve...
- defterGoose 7y agoPerhaps the first sentient program will be born in an MMORPG?
- Animats 7y agoRead "Three Laws Lethal", by Walton.
- radicalbyte 7y agoThere's an entire Anime genre about that.. https://en.wikipedia.org/wiki/Isekai https://en.wikipedia.org/wiki/Isekai
- socceroos 7y agoAh, I knew anime would be useful someday
- jack_pp 7y agohttps://www.fimfiction.net/story/62074/friendship-is-optimal https://www.fimfiction.net/story/62074/friendship-is-optimal
- Izkata 7y agoJust imagine all the exploits they'll find and abuse. OpenAI has already done some experiments here [0]. All the way down at the bottom, under the "surprising behaviors" heading, 3 of the 4 examples involve the AIs finding bugs in the simulation and using it to their advantage. The 4th isn't a bug exactly, but a (missing) edge case in their behavior not initially anticipated. [0] https://openai.com/blog/emergent-tool-use/ https://openai.com/blog/emergent-tool-use/
- guelo 7y agoYes but it sounds weird to me because Carmack has spent his whole life involved with games but has not been known for an interest in game AI before.
- a1studmuffin 7y agoGame AI has nothing to do with AGI (or even regular AI) beyond the surface level description OP provided. The reason game AI hasn't progressed in the last few decades isn't because technology is holding us back - after all we can already achieve impressive machine learning feats using current-gen GPUs - it's because bad NPC AI is by design, so players can learn to overcome them and succeed. Very few people want to play a game that always beats them. Most games use simple state machines or behaviour trees with predictable outcomes for their NPCs because it would be a waste of effort to do anything more, and actually negatively impact the game by making it less fun and burning engineering time on things the player won't benefit from.
- PeterStuer 7y agoYou don't optimize for competitive performance (it is trivial to design a game AI that beats every player every time given that you have control over tilting the playing field). You use the AI for bounded response variations (all NPC's act 'natural' and different from the others) and engaging procedural generation (Here is a chapter of a story, now draft an entire zone with landscape, NPC's, cities, quest story lines, etc.). Games like PvE MMO's need to find a way to produce engaging content faster than it can be consumed at a pricepoint that is economically viable. The way they do it now is by having the players repeat the same content over and over again with a diminishing returns variable reward behavioral reinforcement system.
- antris 7y agoOne of the design goals of game AIs is also that they are fun to play against. If they are too smart and coordinated, they try to throw you off in a way that feels "unfair" to the player. You have to hit a spot where they are sometimes a bit surprising, but not in a way that cannot be reacted to quickly on your feet. This throws realism out of the window.
- PeterStuer 7y agoI did my msc thesis in AI back then writing a dedicated simulater for a specific robot used in autonomous systems research. You find that especially when trying to faithfully reproduce sensor signals you need to dive deep into not just the physics of e.g. infrared light, but also the specific electronic operation of the sensor itself. But that kind of realism is not needed for all AGI research. I also spent some years on using evolutionary algorithms to evolve control networks for simple robots. The computational resources available at the time were rather limited though. Should be more promising these days now that your commodity gaming pc can spew out in 30 minutes what back then took all the labs networked machines running each night for a few weeks.
- ivanhoe 7y agoModeling everything realistically is super hard - any interaction with the real-world is so full of the weirdest unexpected electric and mechanical issues. Who hasn't tried it first-hand can't imagine the half of ways that will almost certainly go wrong on the first try :) ... but as you've said for developing the AGI as a concept simplified worlds should work just fine.
- amatic 7y agoTrue. Simulations are always a simplification of reality and leave a lot out of the picture. On the flip side, successful robotics concepts might have more chance of being relevant to AGI.
- joe_the_user 7y agoIndeed, I don't think humans themselves model the world realistically, I think they model the world close enough but are able to adapt their prediction process for situation when their prediction don't work and/or their knowledge is inadequate. To model such a "satisficing" process, you don't need exact simulation.
- tomaskafka 7y agoI love the idea of using the Sims as a platform, as it's a place where it will be blatantly obvious that 'effective' AI without built-in ethics is repulsively inhuman.
- fastbeef 7y ago> Imagine The Sims, with a lot more internal smarts and real physics, as a base for work. Sounds like the start of a truly horrifying Black Mirror episode
- adrianN 7y agohttps://wiki.lesswrong.com/wiki/Roko%27s_basilisk https://wiki.lesswrong.com/wiki/Roko%27s_basilisk
- haffi112 7y ago> Sounds like the start of a truly horrifying Black Mirror episode That episode already exists. https://en.wikipedia.org/wiki/Hang_the_DJ https://en.wikipedia.org/wiki/Hang_the_DJ
- DonHopkins 7y agoOr a Philip K Dick novel that's so weird and prescient, nobody's been able to figure out how to make a movie from it. https://en.wikipedia.org/wiki/The_Three_Stigmata_of_Palmer_Eldritch https://en.wikipedia.org/wiki/The_Three_Stigmata_of_Palmer_E... http://mxmossman.blogspot.com/2013/01/the-three-stigmata-of-palmer-eldritch.html http://mxmossman.blogspot.com/2013/01/the-three-stigmata-of-... >The Perky Pat Layouts itself is an interesting concept. Here's Dick, in the early 60's, coming up with the idea for virtual worlds. I mean, Second Life and other virtual worlds are just a mapping of the Perky Pat Layouts onto cyberspace. Today Facebook acts like the PP Layouts, taking people's minds off toil and work and letting them engage others in a shared virtual hallucination -- you're not actually physically with your friends, and they might not even be your friends. >Dick’s description of the Can-D experience is essentially a description of virtual sex: >“Her husband -- or his wife or both of them or everyone in the entire hovel -- could show up while he and Fran were in the state of translation. And their two bodies would be seated at proper distance one from the other; no wrong-doing could be observed, however prurient the observers were. Legally this had been ruled on: no co-habitation could be proved, and legal experts among the ruling UN authorities on Mars and the other colonies had tried -- and failed. While translated one could commit incest, murder, anything, and it remained from a juridicial standpoint a mere fantasy, an impotent wish only.” >Another character says “when we chew Can-D and leave our bodies we die. And by dying we lose the weight of -- ... Sin.”
- fastball 7y agoSooooo, The Talos Principle? https://en.wikipedia.org/wiki/The_Talos_Principle https://en.wikipedia.org/wiki/The_Talos_Principle
- 0-_-0 7y agoPuzzle game with a great story. I recommend it to the HN people.
- gray_-_wolf 7y agoI loved it, but my issue with that game was severe motion sickness after 20-30 minutes... never finished it :(
- Abishek_Muthian 7y agoThanks for the warning, I cannot even play Minecraft. I wish Carmack had tackled motion sickness in VR/Games before switching to AI; he did talk about it in the interviews as being a limitation though. There's a need gap[1] to solve Simulation Sickness in VR and First Person games. [1]: https://needgap.com/problems/7-simulation-sickness-in-vr-and-first-person-games-virtualreality-motionsickness https://needgap.com/problems/7-simulation-sickness-in-vr-and...
- phaus 7y agoI was under the impression that simulation sickness was largely solved outside of extreme cases (like a vr portal game). I thought we're just waiting for hardware to catch up. Years ago John said if you have 20k and a dedicated room you can make a convincing vr experience that won't make anyone sick.
- MichaelHoste 7y agoI loved and finished the game but I had the same issue on two occasions. I felt sick and I had to stop. Now I know it was not the food I had just eaten but the game itself, thank you!
- anotheryou 7y agoIt however has a huge bias towards human-like ai. Maybe it's not smart to narrow down to copying us so quickly. I mean: maybe it's more efficient to have it read all of wikipedia really well before adding all the other noisy senses.
- simonh 7y agoI don't think that approach is going to work. For any clearly bounded and delineated task, such as a game, the most optimal, lowest energy and lowest cost solution is not AGI but a custom tuned specialist solver. This is why I don't think Deep Blue or Alphago are paths towards AGI. They are just very highly advanced single-task solvers. Now Alphago and it's implementation framework are much more sophisticated than Deep Blue. It's actually a framework for making single-task solvers, but that's all. The fact it can make more than one single-task solver doesn't making it general in the sense we mean it in the term AGI. AlphaGo didn't learn the rules of Go. It has no idea what those rules are, it's just been trained through trial and error not to break them. That's not the same thing. It's not approaching chess or Go as an intelligent thinking being, learning the rules and working out their consequences. It's like an image classifier that can identify an apple, but has no idea what an apple is, or even what things are. To build an AGI we need a way to genuinely model and manipulate objects, concepts and decisions. What's happened in the last few decades is we've skipped past all that hard work, to land on quick solutions to specific problems. That's achieved impressive, valuable results but I don't think it's a path to AGI. We need to go back to the hard problems of “computer models of the fundamental mechanisms of thought.”[0] [0]https://www.theatlantic.com/magazine/archive/2013/11/the-man-who-would-teach-machines-to-think/309529/ https://www.theatlantic.com/magazine/archive/2013/11/the-man...
- lawlessone 7y agoThe only way i think it can be done is simulated evolution.. be that simulated evolution of neural nets or something else. As others have mentioned here though.. this becomes horrifying if we've created something sentient to kill in games or enslave.
- arethuza 7y agoReverse engineering a human mind would be another approach.
- olalonde 7y agoThis is often overlooked but it's the only approach that is pretty much guaranteed to succeed given enough time. That said, it's also likely that AGI will come about way earlier from another approach (just as planes came before the "robot birds").
- goatinaboat 7y agoSimulator technology is now good enough that you can do quite realistic worlds. It is nowhere near good enough to avoid running into Moravec’s Paradox like a brick wall as soon as you try and apply it outside the simulator.
- the_af 7y ago> This is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. I don't think so. Game NPCs don't need AI, which would be way overkill; they just need to provide the illusion of agency. I think for general AI you need a field where any other option else would be suboptimal or inadequate, but in videogames general AI is the suboptimal option... more cost effective is to just fake it!
- Aeolun 7y agoIf you have general AI of enough sophistication to use in a game, haven’t you just created a virtual human?
- the_af 7y agoYes, probably. At which point, the "videogame" part is irrelevant.
- seanwilson 7y ago> Game NPCs don't need AI > ... more cost effective is to just fake it! Many players complain in story heavy games that their choices have no consequences to the story - this is largely because building stories with meaningful branches isn't economically feasible. A game that could make NPCs react to the what the player does dynamically while also creating a cohesive story for the player to experience would be absolutely groundbreaking in my opinion. This is more in the realms of AI story generation but I haven't seen any work on this that generates stories you would ever mistake as coming from a human (please correct me if I'm wrong) so it would be amazing to see some progress here.
- the_af 7y agoAgreed about the meaningful choices and dynamically generated reactions from NPC, but general AI is not needed for this in my opinion. You also have to consider whether the complaints of "many" players matter when publishing a game. A percentage of vocal players will complain no matter what. Yes, they will complain even if you somehow implement true AI!
- iandanforth 7y agoHi John! Animats was very cool! As you know game physics still kinda sucks for this work. Unity/Bullet/MuJoCo are the best we have and even they have limited body collision counts. Luckily we've now got some GPU physics acceleration, but IMO it's not enough. What we really need is a scalable, distributed, physics pipeline so we can scale sims to 1000x realtime with billions of colliding objects. My guess is that Google/Stadia or Unity/UnityML are better places to do that work than Facebook, but if Carmack decides to learn physics engines* and make a dent I'm sure he will. Until our environments are rich and diverse our agents will remain limited. *More, I'm sure his knowledge already exceeds most people's.
- Animats 7y agoWhat we really need is a scalable, distributed, physics pipeline so we can scale sims to 1000x realtime with billions of colliding objects. Improbable tried to do that with Spatial OS. They spent $500 million on it.[1] Read the linked article. No big game company uses it, because they cut a deal with Google so their system has to run on Google's servers. It costs too much there, and Google can turn off your air supply any time they want to, so there's a huge business risk. [1] https://improbable.io/blog/the-future-of-the-game-engine https://improbable.io/blog/the-future-of-the-game-engine
- iandanforth 7y agoAgree, as a game engine this might power some high-end Stadia games with crazy physics, but the real value is in high-complexity environments for virtual agents. Interestingly companies like SideFX are also doing really interesting work in distributed simulations. (e.g. Houdini)
- aruggirello 7y agoAs a side note, if we're living in a simulation [0], I'd really like to know who's "real" vs. who's an AI bot out there... [0] https://en.wikipedia.org/wiki/Simulation_hypothesis https://en.wikipedia.org/wiki/Simulation_hypothesis
- mindfulmonkey 7y agoHate to break it to you, but we're all NPCs