15 ms·

Integrating GTA V into Universe

805 points by gdb 10y ago

deleted 10y ago
[deleted]
9erdelta 10y ago
Just a heads up to anybody who tries to get this running, you may have to request permission from Amazon to get the required instance going (g2.2xlarge). Apparently they are making sure people aren't trying to set up botnets and bitcoin mining. Just sent in my request, we'll see what happens.
felixthursday 10y ago
What happened to the link? It just redirects
learnf007 10y ago
looks like the blog post got deleted? why???? Sad face :(
LeanderK 10y ago
too bad GTA V is not open-source for researchers. I don't know how customisable it is, but e.g. experiencing snow/ice or a failure like spontaneous pressure loss in one of the tires would be interesting to simulate.
- cr4zy 10y ago
  Yes, it would be great to have source! The modding community does make up for this in some ways. For example, mods provide access to things like finding memory addresses for things like steering angle that weren't available in the API for game scriptors working on GTAV: https://github.com/crosire/scripthookvdotnet/blob/dev_v3/source/scripting/Vehicle.cs#L841-L856 https://github.com/crosire/scripthookvdotnet/blob/dev_v3/sou... re snow: It is possible to lay fresh snow anywhere in addition to drive through this area with snow burms, etc.. https://www.youtube.com/watch?v=rOo2mtuPkKQ&feature=youtu.be https://www.youtube.com/watch?v=rOo2mtuPkKQ&feature=youtu.be Traction is definitely affected in snowy areas. re tires bursting: it seems possible https://github.com/crosire/scripthookvdotnet/blob/2b14a22ad3095e6147289af45c0bf49797a3c39e/source/scripting/Vehicle.cs#L1650 https://github.com/crosire/scripthookvdotnet/blob/2b14a22ad3...
captainarab 10y ago
This is incredible! Huge fan of GTA V / Rockstar as well as Open.AI Will be super interesting to see how all of the randomness in GTA V driving influences autonomous driving AIs. I could see them being a bit more "cautious" in order to deal with all of the crazy drivers (and events) sharing the roads of Los Santos.
- corobo 10y ago
  I'd be super interested in seeing how this reacts to having a single star wanted level. At that level police cars will come to a stop almost perpendicular in front of your car. Will the AI be able to avoid an aggressive obstacle. Also it'd be interesting to see how it reacts to getting shot at
  - mtrycz 10y ago
    I'm missing (and the article doesn't exactly state) what the fitness function is here. It might just as well don't care about being shot at.
    - cr4zy 10y ago
      We currently don't look at being shot at (although it's an interesting suggestion!) - the current reward function is basically this: distance_reward - off_road_penalty - speeding_penalty - slow_penalty - discomfort_penalty; Plus if you collide or drive against traffic, the episode terminates. https://github.com/openai/universe-windows-envs/blob/f5aad9656661e8822c6dbe8dc212bf5f304a6c6e/vnc-gtav/GTAVController/src/GTAVRewardCalculator.cpp#L310 https://github.com/openai/universe-windows-envs/blob/f5aad96...
    - corobo 10y ago
      Well that part was mostly a bit of a joke - It could add in a few interesting variables such as deflated tyres causing the car to drift more than the AI is expecting.
- cr4zy 10y ago
  Thanks! Yes, the game is highly scriptable[1], so you can test the types of scenarios you'd want. We've disabled violence against the driver, but could enable[1] it on-demand if it's useful for training. [1] http://www.dev-c.com/nativedb/ http://www.dev-c.com/nativedb/ [2] http://gtaforums.com/topic/822314-guide-driving-styles/ http://gtaforums.com/topic/822314-guide-driving-styles/
madez 10y ago
I wonder if AI is where games can make use of many cores. If the game logic can not be reasonibly parallelized anymore, the unused computing capacity can be used to add "intelligent" agents into the game. They don't need to do revolutionary things, just being more complex, more "reasonable" and less predictable than the usual agents would be enough. I think that has the potential to make some games more interesting. Imagine a game like Warcraft 3 where the units actually seem to have a personality and follow "their guts" in battles rather than following simple patterns like attacking the nearest enemy. Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?
- ge96 10y ago
  Mr Anderson...
- dtromero 10y ago
  Interested in this as well. Applying machine learning in this way could create some interesting game play.
- mtrycz 10y ago
  I was told (but didn't verify it myself, nor have any sources on this) that in Half Life 2 (2004), your enemies would "learn" from your attack patterns and adapt to it. At the same time, as to make the game enjoyable/chellenging and not frustratingly difficult, the agents would adjust their difficulty level to your skill. Take it as an anecdote.
  - selectodude 10y ago
    That was actually Left 4 Dead. https://en.wikipedia.org/wiki/Left_4_Dead#AI_Director https://en.wikipedia.org/wiki/Left_4_Dead#AI_Director
    - umurkontaci 10y ago
      The director in L4D was had a very simple logic with inputs like number of alive special infecteds and number of zombies to control spawns. It's like seeing a health kit when your health is low.
empath75 10y ago
I'm not sure I would like to live in a world with autonomous vehicles that were trained in Grand Theft Auto.
- anotheryou 10y ago
  I guess trianing/testing the training might work well though
- ilya_su 10y ago
  This is a modified version of GTA where the behavior of people is non-violent.
  - dk1138 10y ago
    Maybe the most unrealistic part about the simulator, then.
    - ceedan 10y ago
      Why?
      - kazga 10y ago
        They're implying that lack of road rage makes it unrealistic.
- midolzzzz 10y ago
  Sounds a bit crazy until you look at it from the perspective of being able to visually observe how an AI driver would react to erratic behavior like pedestrians suddenly running onto the road or conditions like gunfire, accidents, etc. I think this is just a small step in the positive direction of AI development.
- deleted 10y ago
  [deleted]
- cr4zy 10y ago
hacker_9 10y ago
The video shown is a bit underwhelming; the car moves at 1 mph with pretty much just one car behind it the whole time.
- ph0rque 10y ago
  How much real-time is needed to get 1k (or 1M) iterations, to improve it so that it moves as fast as the speed limit allows while obeying all the laws?
  - cr4zy 10y ago
    Convergence on target speeds and steering is actually very quick (a matter of hours on a modern GPU / AlexNet / 227x227 RGB) once you have the data. Generating training data with the correct speed limit and providing speed limit based rewards is the missing step. The training car already obeys traffic laws.
    - ph0rque 10y ago
      Cool! You've proved my point :)
- cr4zy 10y ago
  So the training car has a max speed of 10 meters per second or 22mph which the net matches. Future datasets will contain the road's speed limit as well as throttle / brake targets required to reach that speed limit.
deleted 10y ago
[deleted]
aipyle 10y ago
Open AI is the new Willow Garage. https://en.wikipedia.org/wiki/Willow_Garage https://en.wikipedia.org/wiki/Willow_Garage
wrsh07 10y ago
This feels like an important step in the democratization of AI. We have algorithms, data, and computing power which are unevenly distributed. Providing easy-to-access training data for a self-driving car feels like a huge step in democratizing the data aspect of car AI.
- drcross 10y ago
  With all respect I think "easy-to-access" is a bit of an understatement when you are comparing a 3D game to real-world sensor data. They are barely comparable.
  - wrsh07 10y ago
    The fact that you can even consider training an NN for driving after under an hour of setup is absurd. Did you see the "self driving mario kart"? [1] That's a lot of complexity just to try out some algorithms for driving. Is GTA V the same as setting up a rig which controls I/O for a real car? No. Of course not. You're not being fed data from LIDAR, which Waymo [Google X driving] seems quite invested in. [2] However, much of Waymo's self-driving training is simulated [of course it is, how else would you get enough data?]. This is filling out a gap that is currently otherwise only filled by proprietary solutions. That's a big deal. [1] http://kevinhughes.ca/blog/tensor-kart http://kevinhughes.ca/blog/tensor-kart [2] https://news.ycombinator.com/item?id=13352385 https://news.ycombinator.com/item?id=13352385
    - cr0sh 10y ago
      > Is GTA V the same as setting up a rig which controls I/O for a real car? No. Of course not. You're not being fed data from LIDAR, which Waymo [Google X driving] seems quite invested in. I've been coming around to the conclusion that LIDAR isn't going to be the way forward, unless prices come down greatly and mechanical scanning is done away with. Perhaps something like Flash LIDAR, if it could be made 360 degree FOV and cost under $500.00 in quantity - that might be worthwhile for manufacturing a vehicle with, but currently even the cheapest 3D LIDAR is still way too expensive. I honestly think we'll see camera-based solutions becoming the norm; high-speed, high-resolution machine-vision cameras are already an established technology (though mainly black-and-white; pair those with lower-res color cameras and fuse the output as needed). Set up multiple stereo-camera pairs for overlapping 360 coverage - it would likely be a fairly cheap solution. At that point - the frame-buffer view data from GTA (or some other simulator) could be considered close to "real world" for data gathering and model training.
iagooar 10y ago
So imagine I'd like to start doing cool stuff like this by myself. Where do I start?
AbenezerMamo 10y ago
This is actually pretty exciting!!!!
mysterydip 10y ago
Open world games seem like a great place to work with AI. What has traditionally prevented more intelligence, in my mind, was the ability of AI to understand and integrate with its environment. For example, if I say I threw a baseball, you can visualize that. In a virtual world, the AI can "see" and potentially interact with every object to increase understanding. Great experiment and I look forward to seeing how far you can take it!
zxcvvcxz 10y ago
Who's ready for some "Deep Pimping"? http://gta.wikia.com/wiki/Pimping http://gta.wikia.com/wiki/Pimping ABSTRACT: We explore the adage "pimpin' ain't easy" in the context of deep reinforcement learning in the OpenAI environment GTAV. We train a Deep Q-Network (DQN) to optimize player cash flows. Over the course of training, our network learns to keep these hoes in line, and discipline misbehaving customers.
zitterbewegung 10y ago
It looks like it has driving support but I read the post and can I make a pedestrian agent? It doesn't appear that I could. This looks like something I would like to play with and an awesome project.
kuprel 10y ago
Is it possible to return observations that a well equipped driverless car would receive (e.g. lidar, sonar, multiple cameras, IMU, etc)?
- kuprel 10y ago
  It seems like it would almost be easier to hire some of rockstar's game designers and build a realistic self driving car environment from the ground up, given the amount of funding OpenAI has
  - JabavuAdams 10y ago
    Nah. This is second-system syndrome thinking. Definitely easier to modify the existing huge working system.
    - kuprel 10y ago
      GTAV is closed source
  - cr0sh 10y ago
    This could have been done completely open source, but you wouldn't have gotten all the extras that GTA V gives you (basically the world and all of its objects, npcs, etc - which is a huge plus). For instance, they could have gone with TORCS: http://torcs.sourceforge.net/ http://torcs.sourceforge.net/ ...or Stunt Rally: http://stuntrally.tuxfamily.org/ http://stuntrally.tuxfamily.org/ ...or Trigger Rally: http://trigger-rally.sourceforge.net/ http://trigger-rally.sourceforge.net/ (i'm not sure - but Stunt Rally and Trigger Rally may be from the same Trigger codebase) Trigger Rally also has a webgl version: https://github.com/CodeArtemis/TriggerRally https://github.com/CodeArtemis/TriggerRally All of this is just to say that if it were wanted, a completely open-source driving AI simulator could be easily made.
vyacheslavl 10y ago
now the interesting part: put it in front of ethical problem (take one from http://moralmachine.mit.edu/ http://moralmachine.mit.edu/) and lets see what it does
sytelus 10y ago
This is super cool. Are there any good resources/how-to modify GTA V for things like this? I'm looking in to GTA main website and wikies but info for developers looks thin. Here's few links I found so far: https://en.wikipedia.org/wiki/Modding_in_Grand_Theft_Auto https://en.wikipedia.org/wiki/Modding_in_Grand_Theft_Auto http://www.dev-c.com/gtav/scripthookv/ http://www.dev-c.com/gtav/scripthookv/ https://www.youtube.com/watch?v=wS8oJTHqf8Q https://www.youtube.com/watch?v=wS8oJTHqf8Q https://www.reddit.com/r/GrandTheftAutoV_PC/comments/38wct0/guide_to_modding_gta_5_safely_and_effectively/ https://www.reddit.com/r/GrandTheftAutoV_PC/comments/38wct0/... https://steamcommunity.com/sharedfiles/filedetails/?id=558079253 https://steamcommunity.com/sharedfiles/filedetails/?id=55807... http://gtaforums.com/forum/109-tutorials/ http://gtaforums.com/forum/109-tutorials/ Also I'm reading that GTA folks went out of their way to prevent modding. So may be everything you do with GTA is a hack and there is no supported APIs for developers?
- drivingmenuts 10y ago
  Rockstar's policy is that you're free to mod the single-player game, but any modding whatsoever of the online game is forbidden, no matter how miniscule. They detect it and it's a potential ban. That said, they don't publish anything about modding and don't have an official API available. Everything has been done third-party. The modding has caused real problems in the online play - player in godmode, handing out in-game money, etc. No matter the efforts of R*, they seem to be unable to eradicate it for good. The group of players I previously played the game with, including myself, have all become extremely disenchanted with the game by the online modders.
  - yarou 10y ago
    It ultimately boils down to a cat and mouse game. What you end up with are extremely intrusive anticheat tools that do all sorts of nasty Trojan-like things; for example, injecting DLLs into a player's host machine to determine if they have modified the memory of the game. The anticheat process itself will hide and run in kernelmode. I don't know about you, but I don't feel terribly comfortable having a closed source binary running in ring0.
squeaky-clean 10y ago
I love GTAV so much, it's the most real single player world I've ever experienced. I once rear-ended an NPC because they suddenly stopped in the middle of the street. I got out of my car (because it's GTA and you have to shoot everyone, duh) I saw the NPC had stopped because he hit a cat with his car! I'd be lying if I said OpenAI Universe / GTAV wasn't one of the primary drivers in me trying to learn ML. Anyone know what kind of hardware I will need to play with this? The AMI page on github mentions an AWS g2.2xlarge (iirc a GTX 1060 is slightly better than this?). And it seems like GTA is actually running on a different system you use VNC to share the screen/input? Any estimates about whether a 1060 or 1080 on a beefy gaming PC could handle both at once?
- cr4zy 10y ago
  It's technically possible to run everything on one Windows machine (using VNC loopback mode), however we expect most people to run ML on Linux / Mac so that is the best supported configuration. Before Universe, I ran a modified version of Caffe in Windows along with the game on a GTX 980 and there were more than enough GPU resources for that. Running ML / ROS / other self-driving software in Linux is much more ideal though.
  - squeaky-clean 10y ago
    Thanks for this info. My gaming PC has a 7950, so I'd need to buy a Nvidia card anyways to do this. I was hoping to avoid the need to put together a second PC for ML, but I think I have enough spare parts lying around that I could do it cheaply, and I believe you about Linux being ideal over Windows. Sounds like a 1060 should be a good start for me to play with Universe then.
- dTal 10y ago
  Hmpth. Conversely, while playing GTAV in completely innocent explore-the-world mode, I accidentally bumped a police boat with my jetski. Apparently this warranted being shot in the head. Most reviews I've read actually complain that GTAV is a step back in realism from GTA4 (for example, body-part specific injuries were dropped), but let's not beat about the bush - no GTA deserves the title of "realistic", because as you say "it's GTA and you have to shoot everyone, duh". They're made to feel realistic, but in fact they powerfully reward violence and punish being law-abiding, in precisely the opposite way that the real world does.
ex3ndr 10y ago
Next Step: Build Street View map of GTA V.
justifier 10y ago
> artificially slowed to 8FPS i'm sure this is to offer relief to the nn but does this further simulate 'reality'? are our autonomous vehicles seeing at 8 frames per second?
- alderz 10y ago
  As the average human reaction time is 0.19 seconds, I don't see 8FPS too bad for a start. More FPS is always nice, of course.
  - justifier 10y ago
    interesting comparison but it is a bit disingenuous there is more going on than just input in that reaction time figure 8fps fails to assume a reaction time of 0.125s for the same reason that the human eye's ~220fps.. now this is a mental stretch of the true underlying mechanisms but i'll make it for sake of consistency of argument(o).. fails to imply a reaction time of .00454545s i guess i assumed autonomous vehicles sensor data would be measured in hertz, but now am intrigued by what framerate the computer vision system works (o) http://www.100fps.com/how_many_frames_can_humans_see.htm http://www.100fps.com/how_many_frames_can_humans_see.htm
  - will_pseudonym 10y ago
    Reaction times and flicker fusion rates[0] are orthogonal. 0 ms latency at 8 FPS is different than 190 ms latency at 120 FPS. E.g. If there is an event that takes less than 125 ms, at 8 FPS there is a probability that you would miss it entirely! At 190 ms latency with 120 FPS, you would be guaranteed of seeing a 125 ms event, but processing of that information would take 190 ms longer. [0] https://en.wikipedia.org/wiki/Flicker_fusion_threshold https://en.wikipedia.org/wiki/Flicker_fusion_threshold
cosinetau 10y ago
I'm not sure how anti cheat methods will work now.
delegate 10y ago
This is cool. Next step is obviously teaching it how to kill people in the game and get away from police. I'd like to see the AI go on a rampage and win :). That should be enough to wipe off a lot of smiles off our idealistic faces :)
- smrtinsert 10y ago
  It's all fun and games until the Second Amendment is used to protect the ownership of armed drones that can be used to assassinate anyone anonymously simply by scratching the serial numbers off.
  - 234tq3w4gfase 10y ago
    yeah that's definitely a scary possibility, though I would think explosives would be the more realistic threat anyways, electronic actuation of a firearm is already illegal in the US. if you're firing a gun in any way besides actually pulling a trigger with a finger, chances are good it's illegal (with some small leeway such as turning a handcrank)
    - logfromblammo 10y ago
      If you're going to murder someone, you're probably not too worried about arms-control laws. You're already planning on doing something way more illegal. But if you're going for some sort of obscure criminal achievement by only breaking one law at a time, then just make your lethal device something that does not legally qualify as a firearm. A black-powder muzzle-loader or a Bulgarian ricin-umbrella would do the trick, depending on whether you care about noise or not. Or, like the parent mentions, make a grenade, and deliver it via remote-piloted aircraft. Realistically, though, very few people on this planet would bother with any of that when they can often just kick down the door of your home, kill you with a tool purchased from a hardware or sporting goods store, and just leave before police can respond. The clearance rate for violent crimes in the US is already low enough, even for people who give no thought whatsoever to covering their tracks. I'll continue to assume that unmanned aircraft are carrying cameras, and assume that if someone wants to murder me, they need a compelling reason to have a more complicated plan than to just bash my skull in with a big rock when I'm not looking.
ChuckMcM 10y ago
Ok, I am so glad that we're not training AI to "win" at GTA V. That is not an android I want to meet on the street :-) The transitive property of teaching an AI to drive given a world simulation, makes me wonder if you can train a network to recognize speech by feeding War and Peace through a text to speech program and then sending the audio data generated into an RNN. Where is the ground truth in such a system which you can calibrate your results by?
- semi-extrinsic 10y ago
  > Where is the ground truth in such a system which you can calibrate your results by? Answering standard reading comprehension test questions? - "Why did Pierre meet Natasha at the church?"
uptown 10y ago
Off-topic, but does anyone know whether it's the GTA engine used to record these videos: https://www.youtube.com/watch?v=T5kooJeXGiI https://www.youtube.com/watch?v=T5kooJeXGiI There's a ton of them on YouTube, but I can't figure out how they make them.
- PhilipTrauner 10y ago
  Yep, you can tell by the way the helicopter sounds and the walking animation for spiderman. The motion blur also looks very similar. The terrible looking ENB mod makes it difficult to notice, tough.
- kowdermeister 10y ago
  It's just a normal stock game or some mod added. Some use the GTA V. director: http://kotaku.com/on-youtube-millions-watch-shows-for-children-made-enti-1788282618 http://kotaku.com/on-youtube-millions-watch-shows-for-childr... The creepy thing is that this video is targeted at 0-3 year olds who randomly tap on mom's iPad. If you think like that now it's clear why the title is just keyword stuffing, the lame music and the spiderman outfit. 80 million views in a month is a lot of money: https://www.youtube.com/watch?v=ipTJNNvW-Gw https://www.youtube.com/watch?v=ipTJNNvW-Gw Just checked their channel: 4,316,098,805 views
  - uptown 10y ago
    "80 million views in a month is a lot of money" What would 80 million views translate to in dollar terms?
    - keypusher 10y ago
      I have seen $2,000 / million views, so $160,000 / month?
  - uptown 10y ago
    >Just checked their channel: 4,316,098,805 views Where do you see that count?
deepnotderp 10y ago
Damn, OpenAI is a house
flippyhead 10y ago
I live in the future.
dsugarman 10y ago
It would be really great to see OpenAI sponsor AI contests, the # of people contributing / progressing on specific goals would be enormous. This is really awesome, great work to those involved.
celerrimus 10y ago
Probably Uber should practice in GTAV, rather then on real roads:)
deleted 10y ago
[deleted]
IMTDb 10y ago
Did anyone try to feed this AI with actual dashcam images and see if the driving learnt in GTA would be applicable in real life situations ?
lxe 10y ago
It looks like it's not just using the camera/vnc images as input, but also uses depth data to simulate LIDAR input (http://deepdrive.io/ http://deepdrive.io/). I wonder, what makes it hard to rely on solely the 2 cameras to calculate/guess the depth, similarly to the way us non-robotic drivers do it, when it comes to machine sensing. EDIT: found this: https://web.stanford.edu/class/ee368/Project_Autumn_1516/Reports/Appiah_Bandaru.pdf https://web.stanford.edu/class/ee368/Project_Autumn_1516/Rep... on the topic. Is there anything else I should read up on? EDIT: found lots of articles: https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=optical+flow+depth+estimation https://www.google.com/webhp?sourceid=chrome-instant&ion=1&e...
kobeya 10y ago
I'm probably one of the most critical people of the new AI-safety craze, but even I wonder if training AI on games like GTA is a good thing to do... if it is something I'd be hesitant to let my kids play, should we really be training our mechanical offspring on it?
- miguelrochefort 10y ago
  Surely you're not that naive.
  - kobeya 10y ago
    What is naïve? OpenAI is not a domain specific company working on some limited application like self driving cars. They are interested in developing general intelligence. Unlike a racing game, GTA has non player characters (agents) that can be manipulated to accomplish goals. But the lessons learned in doing so are not things I would want to generalize or carry forward into future experiments. E.g. Maybe it learns that pedestrians made braking instruments in a tight turn.
- megablast 10y ago
  It is not playing the game, it is using the game as a driving simulator.
  - kobeya 10y ago
    It's using the game as a driving simulator by playing it.
nojvek 10y ago
This would make an interesting plot for a movie. Someday hacks GTAV ai into Google self driving cars. Suddenly "do no evil" company becomes "fuck the world up" and man tries to fight cars. Anything that has wheels is dangerous to humans!
rubyfan 10y ago
What could go wrong
KennyCason 10y ago
Imagine using this to train and to "unit" test a self driving car. With realistic graphics and the ability to control events, I wonder how useful this could be. (e.g. simulate near accident situations)
9erdelta 10y ago
This is super intriguing! Didn't have GTA V for PC but now I do!
fb03 10y ago
I know this is totally unrelated, but i'd love to see a ML framework have a go at NetHack!

805 points by gdb 10y ago

deleted 10y ago

[deleted]

Just a heads up to anybody who tries to get this running, you may have to request permission from Amazon to get the required instance going (g2.2xlarge). Apparently they are making sure people aren't trying to set up botnets and bitcoin mining. Just sent in my request, we'll see what happens.

felixthursday 10y ago

What happened to the link? It just redirects

learnf007 10y ago

looks like the blog post got deleted? why???? Sad face :(

LeanderK 10y ago

too bad GTA V is not open-source for researchers. I don't know how customisable it is, but e.g. experiencing snow/ice or a failure like spontaneous pressure loss in one of the tires would be interesting to simulate.

cr4zy 10y ago
Yes, it would be great to have source! The modding community does make up for this in some ways. For example, mods provide access to things like finding memory addresses for things like steering angle that weren't available in the API for game scriptors working on GTAV: https://github.com/crosire/scripthookvdotnet/blob/dev_v3/source/scripting/Vehicle.cs#L841-L856 https://github.com/crosire/scripthookvdotnet/blob/dev_v3/sou... re snow: It is possible to lay fresh snow anywhere in addition to drive through this area with snow burms, etc.. https://www.youtube.com/watch?v=rOo2mtuPkKQ&feature=youtu.be https://www.youtube.com/watch?v=rOo2mtuPkKQ&feature=youtu.be Traction is definitely affected in snowy areas. re tires bursting: it seems possible https://github.com/crosire/scripthookvdotnet/blob/2b14a22ad3095e6147289af45c0bf49797a3c39e/source/scripting/Vehicle.cs#L1650 https://github.com/crosire/scripthookvdotnet/blob/2b14a22ad3...

captainarab 10y ago

This is incredible! Huge fan of GTA V / Rockstar as well as Open.AI Will be super interesting to see how all of the randomness in GTA V driving influences autonomous driving AIs. I could see them being a bit more "cautious" in order to deal with all of the crazy drivers (and events) sharing the roads of Los Santos.

corobo 10y ago
I'd be super interested in seeing how this reacts to having a single star wanted level. At that level police cars will come to a stop almost perpendicular in front of your car. Will the AI be able to avoid an aggressive obstacle. Also it'd be interesting to see how it reacts to getting shot at
- mtrycz 10y ago
  I'm missing (and the article doesn't exactly state) what the fitness function is here. It might just as well don't care about being shot at.
  - cr4zy 10y ago
    We currently don't look at being shot at (although it's an interesting suggestion!) - the current reward function is basically this: distance_reward - off_road_penalty - speeding_penalty - slow_penalty - discomfort_penalty; Plus if you collide or drive against traffic, the episode terminates. https://github.com/openai/universe-windows-envs/blob/f5aad9656661e8822c6dbe8dc212bf5f304a6c6e/vnc-gtav/GTAVController/src/GTAVRewardCalculator.cpp#L310 https://github.com/openai/universe-windows-envs/blob/f5aad96...
  - corobo 10y ago
    Well that part was mostly a bit of a joke - It could add in a few interesting variables such as deflated tyres causing the car to drift more than the AI is expecting.
cr4zy 10y ago
Thanks! Yes, the game is highly scriptable[1], so you can test the types of scenarios you'd want. We've disabled violence against the driver, but could enable[1] it on-demand if it's useful for training. [1] http://www.dev-c.com/nativedb/ http://www.dev-c.com/nativedb/ [2] http://gtaforums.com/topic/822314-guide-driving-styles/ http://gtaforums.com/topic/822314-guide-driving-styles/

madez 10y ago

I wonder if AI is where games can make use of many cores. If the game logic can not be reasonibly parallelized anymore, the unused computing capacity can be used to add "intelligent" agents into the game. They don't need to do revolutionary things, just being more complex, more "reasonable" and less predictable than the usual agents would be enough. I think that has the potential to make some games more interesting. Imagine a game like Warcraft 3 where the units actually seem to have a personality and follow "their guts" in battles rather than following simple patterns like attacking the nearest enemy. Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?

ge96 10y ago
Mr Anderson...
dtromero 10y ago
Interested in this as well. Applying machine learning in this way could create some interesting game play.
mtrycz 10y ago
I was told (but didn't verify it myself, nor have any sources on this) that in Half Life 2 (2004), your enemies would "learn" from your attack patterns and adapt to it. At the same time, as to make the game enjoyable/chellenging and not frustratingly difficult, the agents would adjust their difficulty level to your skill. Take it as an anecdote.
- selectodude 10y ago
  That was actually Left 4 Dead. https://en.wikipedia.org/wiki/Left_4_Dead#AI_Director https://en.wikipedia.org/wiki/Left_4_Dead#AI_Director
  - umurkontaci 10y ago
    The director in L4D was had a very simple logic with inputs like number of alive special infecteds and number of zombies to control spawns. It's like seeing a health kit when your health is low.

empath75 10y ago

I'm not sure I would like to live in a world with autonomous vehicles that were trained in Grand Theft Auto.

anotheryou 10y ago
I guess trianing/testing the training might work well though
ilya_su 10y ago
This is a modified version of GTA where the behavior of people is non-violent.
- dk1138 10y ago
  Maybe the most unrealistic part about the simulator, then.
  - ceedan 10y ago
    Why?
    - kazga 10y ago
      They're implying that lack of road rage makes it unrealistic.
midolzzzz 10y ago
Sounds a bit crazy until you look at it from the perspective of being able to visually observe how an AI driver would react to erratic behavior like pedestrians suddenly running onto the road or conditions like gunfire, accidents, etc. I think this is just a small step in the positive direction of AI development.
deleted 10y ago
[deleted]
cr4zy 10y ago

hacker_9 10y ago

The video shown is a bit underwhelming; the car moves at 1 mph with pretty much just one car behind it the whole time.

ph0rque 10y ago
How much real-time is needed to get 1k (or 1M) iterations, to improve it so that it moves as fast as the speed limit allows while obeying all the laws?
- cr4zy 10y ago
  Convergence on target speeds and steering is actually very quick (a matter of hours on a modern GPU / AlexNet / 227x227 RGB) once you have the data. Generating training data with the correct speed limit and providing speed limit based rewards is the missing step. The training car already obeys traffic laws.
  - ph0rque 10y ago
    Cool! You've proved my point :)
cr4zy 10y ago
So the training car has a max speed of 10 meters per second or 22mph which the net matches. Future datasets will contain the road's speed limit as well as throttle / brake targets required to reach that speed limit.

deleted 10y ago

[deleted]

aipyle 10y ago

Open AI is the new Willow Garage. https://en.wikipedia.org/wiki/Willow_Garage https://en.wikipedia.org/wiki/Willow_Garage

wrsh07 10y ago

This feels like an important step in the democratization of AI. We have algorithms, data, and computing power which are unevenly distributed. Providing easy-to-access training data for a self-driving car feels like a huge step in democratizing the data aspect of car AI.

drcross 10y ago
With all respect I think "easy-to-access" is a bit of an understatement when you are comparing a 3D game to real-world sensor data. They are barely comparable.
- wrsh07 10y ago
  The fact that you can even consider training an NN for driving after under an hour of setup is absurd. Did you see the "self driving mario kart"? [1] That's a lot of complexity just to try out some algorithms for driving. Is GTA V the same as setting up a rig which controls I/O for a real car? No. Of course not. You're not being fed data from LIDAR, which Waymo [Google X driving] seems quite invested in. [2] However, much of Waymo's self-driving training is simulated [of course it is, how else would you get enough data?]. This is filling out a gap that is currently otherwise only filled by proprietary solutions. That's a big deal. [1] http://kevinhughes.ca/blog/tensor-kart http://kevinhughes.ca/blog/tensor-kart [2] https://news.ycombinator.com/item?id=13352385 https://news.ycombinator.com/item?id=13352385
  - cr0sh 10y ago
    > Is GTA V the same as setting up a rig which controls I/O for a real car? No. Of course not. You're not being fed data from LIDAR, which Waymo [Google X driving] seems quite invested in. I've been coming around to the conclusion that LIDAR isn't going to be the way forward, unless prices come down greatly and mechanical scanning is done away with. Perhaps something like Flash LIDAR, if it could be made 360 degree FOV and cost under $500.00 in quantity - that might be worthwhile for manufacturing a vehicle with, but currently even the cheapest 3D LIDAR is still way too expensive. I honestly think we'll see camera-based solutions becoming the norm; high-speed, high-resolution machine-vision cameras are already an established technology (though mainly black-and-white; pair those with lower-res color cameras and fuse the output as needed). Set up multiple stereo-camera pairs for overlapping 360 coverage - it would likely be a fairly cheap solution. At that point - the frame-buffer view data from GTA (or some other simulator) could be considered close to "real world" for data gathering and model training.

iagooar 10y ago

So imagine I'd like to start doing cool stuff like this by myself. Where do I start?

AbenezerMamo 10y ago

This is actually pretty exciting!!!!

mysterydip 10y ago

Open world games seem like a great place to work with AI. What has traditionally prevented more intelligence, in my mind, was the ability of AI to understand and integrate with its environment. For example, if I say I threw a baseball, you can visualize that. In a virtual world, the AI can "see" and potentially interact with every object to increase understanding. Great experiment and I look forward to seeing how far you can take it!

zxcvvcxz 10y ago

Who's ready for some "Deep Pimping"? http://gta.wikia.com/wiki/Pimping http://gta.wikia.com/wiki/Pimping ABSTRACT: We explore the adage "pimpin' ain't easy" in the context of deep reinforcement learning in the OpenAI environment GTAV. We train a Deep Q-Network (DQN) to optimize player cash flows. Over the course of training, our network learns to keep these hoes in line, and discipline misbehaving customers.

zitterbewegung 10y ago

It looks like it has driving support but I read the post and can I make a pedestrian agent? It doesn't appear that I could. This looks like something I would like to play with and an awesome project.

kuprel 10y ago

Is it possible to return observations that a well equipped driverless car would receive (e.g. lidar, sonar, multiple cameras, IMU, etc)?

kuprel 10y ago
It seems like it would almost be easier to hire some of rockstar's game designers and build a realistic self driving car environment from the ground up, given the amount of funding OpenAI has
- JabavuAdams 10y ago
  Nah. This is second-system syndrome thinking. Definitely easier to modify the existing huge working system.
  - kuprel 10y ago
    GTAV is closed source
- cr0sh 10y ago
  This could have been done completely open source, but you wouldn't have gotten all the extras that GTA V gives you (basically the world and all of its objects, npcs, etc - which is a huge plus). For instance, they could have gone with TORCS: http://torcs.sourceforge.net/ http://torcs.sourceforge.net/ ...or Stunt Rally: http://stuntrally.tuxfamily.org/ http://stuntrally.tuxfamily.org/ ...or Trigger Rally: http://trigger-rally.sourceforge.net/ http://trigger-rally.sourceforge.net/ (i'm not sure - but Stunt Rally and Trigger Rally may be from the same Trigger codebase) Trigger Rally also has a webgl version: https://github.com/CodeArtemis/TriggerRally https://github.com/CodeArtemis/TriggerRally All of this is just to say that if it were wanted, a completely open-source driving AI simulator could be easily made.

vyacheslavl 10y ago

now the interesting part: put it in front of ethical problem (take one from http://moralmachine.mit.edu/ http://moralmachine.mit.edu/) and lets see what it does

sytelus 10y ago

This is super cool. Are there any good resources/how-to modify GTA V for things like this? I'm looking in to GTA main website and wikies but info for developers looks thin. Here's few links I found so far: https://en.wikipedia.org/wiki/Modding_in_Grand_Theft_Auto https://en.wikipedia.org/wiki/Modding_in_Grand_Theft_Auto http://www.dev-c.com/gtav/scripthookv/ http://www.dev-c.com/gtav/scripthookv/ https://www.youtube.com/watch?v=wS8oJTHqf8Q https://www.youtube.com/watch?v=wS8oJTHqf8Q https://www.reddit.com/r/GrandTheftAutoV_PC/comments/38wct0/guide_to_modding_gta_5_safely_and_effectively/ https://www.reddit.com/r/GrandTheftAutoV_PC/comments/38wct0/... https://steamcommunity.com/sharedfiles/filedetails/?id=558079253 https://steamcommunity.com/sharedfiles/filedetails/?id=55807... http://gtaforums.com/forum/109-tutorials/ http://gtaforums.com/forum/109-tutorials/ Also I'm reading that GTA folks went out of their way to prevent modding. So may be everything you do with GTA is a hack and there is no supported APIs for developers?

drivingmenuts 10y ago
Rockstar's policy is that you're free to mod the single-player game, but any modding whatsoever of the online game is forbidden, no matter how miniscule. They detect it and it's a potential ban. That said, they don't publish anything about modding and don't have an official API available. Everything has been done third-party. The modding has caused real problems in the online play - player in godmode, handing out in-game money, etc. No matter the efforts of R*, they seem to be unable to eradicate it for good. The group of players I previously played the game with, including myself, have all become extremely disenchanted with the game by the online modders.
- yarou 10y ago
  It ultimately boils down to a cat and mouse game. What you end up with are extremely intrusive anticheat tools that do all sorts of nasty Trojan-like things; for example, injecting DLLs into a player's host machine to determine if they have modified the memory of the game. The anticheat process itself will hide and run in kernelmode. I don't know about you, but I don't feel terribly comfortable having a closed source binary running in ring0.

squeaky-clean 10y ago

I love GTAV so much, it's the most real single player world I've ever experienced. I once rear-ended an NPC because they suddenly stopped in the middle of the street. I got out of my car (because it's GTA and you have to shoot everyone, duh) I saw the NPC had stopped because he hit a cat with his car! I'd be lying if I said OpenAI Universe / GTAV wasn't one of the primary drivers in me trying to learn ML. Anyone know what kind of hardware I will need to play with this? The AMI page on github mentions an AWS g2.2xlarge (iirc a GTX 1060 is slightly better than this?). And it seems like GTA is actually running on a different system you use VNC to share the screen/input? Any estimates about whether a 1060 or 1080 on a beefy gaming PC could handle both at once?

cr4zy 10y ago
It's technically possible to run everything on one Windows machine (using VNC loopback mode), however we expect most people to run ML on Linux / Mac so that is the best supported configuration. Before Universe, I ran a modified version of Caffe in Windows along with the game on a GTX 980 and there were more than enough GPU resources for that. Running ML / ROS / other self-driving software in Linux is much more ideal though.
- squeaky-clean 10y ago
  Thanks for this info. My gaming PC has a 7950, so I'd need to buy a Nvidia card anyways to do this. I was hoping to avoid the need to put together a second PC for ML, but I think I have enough spare parts lying around that I could do it cheaply, and I believe you about Linux being ideal over Windows. Sounds like a 1060 should be a good start for me to play with Universe then.
dTal 10y ago
Hmpth. Conversely, while playing GTAV in completely innocent explore-the-world mode, I accidentally bumped a police boat with my jetski. Apparently this warranted being shot in the head. Most reviews I've read actually complain that GTAV is a step back in realism from GTA4 (for example, body-part specific injuries were dropped), but let's not beat about the bush - no GTA deserves the title of "realistic", because as you say "it's GTA and you have to shoot everyone, duh". They're made to feel realistic, but in fact they powerfully reward violence and punish being law-abiding, in precisely the opposite way that the real world does.

ex3ndr 10y ago

Next Step: Build Street View map of GTA V.

justifier 10y ago

> artificially slowed to 8FPS i'm sure this is to offer relief to the nn but does this further simulate 'reality'? are our autonomous vehicles seeing at 8 frames per second?

alderz 10y ago
As the average human reaction time is 0.19 seconds, I don't see 8FPS too bad for a start. More FPS is always nice, of course.
- justifier 10y ago
  interesting comparison but it is a bit disingenuous there is more going on than just input in that reaction time figure 8fps fails to assume a reaction time of 0.125s for the same reason that the human eye's ~220fps.. now this is a mental stretch of the true underlying mechanisms but i'll make it for sake of consistency of argument(o).. fails to imply a reaction time of .00454545s i guess i assumed autonomous vehicles sensor data would be measured in hertz, but now am intrigued by what framerate the computer vision system works (o) http://www.100fps.com/how_many_frames_can_humans_see.htm http://www.100fps.com/how_many_frames_can_humans_see.htm
- will_pseudonym 10y ago
  Reaction times and flicker fusion rates[0] are orthogonal. 0 ms latency at 8 FPS is different than 190 ms latency at 120 FPS. E.g. If there is an event that takes less than 125 ms, at 8 FPS there is a probability that you would miss it entirely! At 190 ms latency with 120 FPS, you would be guaranteed of seeing a 125 ms event, but processing of that information would take 190 ms longer. [0] https://en.wikipedia.org/wiki/Flicker_fusion_threshold https://en.wikipedia.org/wiki/Flicker_fusion_threshold

cosinetau 10y ago

I'm not sure how anti cheat methods will work now.

delegate 10y ago

This is cool. Next step is obviously teaching it how to kill people in the game and get away from police. I'd like to see the AI go on a rampage and win :). That should be enough to wipe off a lot of smiles off our idealistic faces :)

smrtinsert 10y ago
It's all fun and games until the Second Amendment is used to protect the ownership of armed drones that can be used to assassinate anyone anonymously simply by scratching the serial numbers off.
- 234tq3w4gfase 10y ago
  yeah that's definitely a scary possibility, though I would think explosives would be the more realistic threat anyways, electronic actuation of a firearm is already illegal in the US. if you're firing a gun in any way besides actually pulling a trigger with a finger, chances are good it's illegal (with some small leeway such as turning a handcrank)
  - logfromblammo 10y ago
    If you're going to murder someone, you're probably not too worried about arms-control laws. You're already planning on doing something way more illegal. But if you're going for some sort of obscure criminal achievement by only breaking one law at a time, then just make your lethal device something that does not legally qualify as a firearm. A black-powder muzzle-loader or a Bulgarian ricin-umbrella would do the trick, depending on whether you care about noise or not. Or, like the parent mentions, make a grenade, and deliver it via remote-piloted aircraft. Realistically, though, very few people on this planet would bother with any of that when they can often just kick down the door of your home, kill you with a tool purchased from a hardware or sporting goods store, and just leave before police can respond. The clearance rate for violent crimes in the US is already low enough, even for people who give no thought whatsoever to covering their tracks. I'll continue to assume that unmanned aircraft are carrying cameras, and assume that if someone wants to murder me, they need a compelling reason to have a more complicated plan than to just bash my skull in with a big rock when I'm not looking.

ChuckMcM 10y ago

Ok, I am so glad that we're not training AI to "win" at GTA V. That is not an android I want to meet on the street :-) The transitive property of teaching an AI to drive given a world simulation, makes me wonder if you can train a network to recognize speech by feeding War and Peace through a text to speech program and then sending the audio data generated into an RNN. Where is the ground truth in such a system which you can calibrate your results by?

semi-extrinsic 10y ago
> Where is the ground truth in such a system which you can calibrate your results by? Answering standard reading comprehension test questions? - "Why did Pierre meet Natasha at the church?"

uptown 10y ago

Off-topic, but does anyone know whether it's the GTA engine used to record these videos: https://www.youtube.com/watch?v=T5kooJeXGiI https://www.youtube.com/watch?v=T5kooJeXGiI There's a ton of them on YouTube, but I can't figure out how they make them.

PhilipTrauner 10y ago
Yep, you can tell by the way the helicopter sounds and the walking animation for spiderman. The motion blur also looks very similar. The terrible looking ENB mod makes it difficult to notice, tough.
kowdermeister 10y ago
It's just a normal stock game or some mod added. Some use the GTA V. director: http://kotaku.com/on-youtube-millions-watch-shows-for-children-made-enti-1788282618 http://kotaku.com/on-youtube-millions-watch-shows-for-childr... The creepy thing is that this video is targeted at 0-3 year olds who randomly tap on mom's iPad. If you think like that now it's clear why the title is just keyword stuffing, the lame music and the spiderman outfit. 80 million views in a month is a lot of money: https://www.youtube.com/watch?v=ipTJNNvW-Gw https://www.youtube.com/watch?v=ipTJNNvW-Gw Just checked their channel: 4,316,098,805 views
- uptown 10y ago
  "80 million views in a month is a lot of money" What would 80 million views translate to in dollar terms?
  - keypusher 10y ago
    I have seen $2,000 / million views, so $160,000 / month?
- uptown 10y ago
  >Just checked their channel: 4,316,098,805 views Where do you see that count?

deepnotderp 10y ago

Damn, OpenAI is a house

flippyhead 10y ago

I live in the future.

dsugarman 10y ago

It would be really great to see OpenAI sponsor AI contests, the # of people contributing / progressing on specific goals would be enormous. This is really awesome, great work to those involved.

celerrimus 10y ago

Probably Uber should practice in GTAV, rather then on real roads:)

deleted 10y ago

[deleted]

IMTDb 10y ago

Did anyone try to feed this AI with actual dashcam images and see if the driving learnt in GTA would be applicable in real life situations ?

lxe 10y ago

It looks like it's not just using the camera/vnc images as input, but also uses depth data to simulate LIDAR input (http://deepdrive.io/ http://deepdrive.io/). I wonder, what makes it hard to rely on solely the 2 cameras to calculate/guess the depth, similarly to the way us non-robotic drivers do it, when it comes to machine sensing. EDIT: found this: https://web.stanford.edu/class/ee368/Project_Autumn_1516/Reports/Appiah_Bandaru.pdf https://web.stanford.edu/class/ee368/Project_Autumn_1516/Rep... on the topic. Is there anything else I should read up on? EDIT: found lots of articles: https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=optical+flow+depth+estimation https://www.google.com/webhp?sourceid=chrome-instant&ion=1&e...

kobeya 10y ago

I'm probably one of the most critical people of the new AI-safety craze, but even I wonder if training AI on games like GTA is a good thing to do... if it is something I'd be hesitant to let my kids play, should we really be training our mechanical offspring on it?

miguelrochefort 10y ago
Surely you're not that naive.
- kobeya 10y ago
  What is naïve? OpenAI is not a domain specific company working on some limited application like self driving cars. They are interested in developing general intelligence. Unlike a racing game, GTA has non player characters (agents) that can be manipulated to accomplish goals. But the lessons learned in doing so are not things I would want to generalize or carry forward into future experiments. E.g. Maybe it learns that pedestrians made braking instruments in a tight turn.
megablast 10y ago
It is not playing the game, it is using the game as a driving simulator.
- kobeya 10y ago
  It's using the game as a driving simulator by playing it.

nojvek 10y ago

This would make an interesting plot for a movie. Someday hacks GTAV ai into Google self driving cars. Suddenly "do no evil" company becomes "fuck the world up" and man tries to fight cars. Anything that has wheels is dangerous to humans!

rubyfan 10y ago

What could go wrong

KennyCason 10y ago

Imagine using this to train and to "unit" test a self driving car. With realistic graphics and the ability to control events, I wonder how useful this could be. (e.g. simulate near accident situations)

9erdelta 10y ago

This is super intriguing! Didn't have GTA V for PC but now I do!

fb03 10y ago

I know this is totally unrelated, but i'd love to see a ML framework have a go at NetHack!