8 ms·
ChosunTruck – Euro Truck Simulator 2 autonomous driving solution
- djsumdog 10y agoThere was a GTA5 one earlier this month wasn't there? These devs should really start/join a DARPA challenge team; try out their ideas on real courses.
- mastazi 10y agoYes, I remember seeing the GTA V one, I think it's part of OpenAI's Universe https://universe.openai.com/ https://universe.openai.com/
- citruspi 10y agoYep, it was on integrating GTA V into Universe[0]. Interestingly, that post links to a specific blog post on GTA V integration[1], but going to that link redirects to the initial "Introducing OpenAI" post[2]. I can't find the GTA V blog post[3] on their blog anymore. Edit: In addition, the GitHub repository[4] for spinning up a GTA V server instance from the post[3] no longer exists. [0]: https://news.ycombinator.com/item?id=13375543 https://news.ycombinator.com/item?id=13375543 [1]: https://openai.com/blog/GTA-V-plus-Universe/ https://openai.com/blog/GTA-V-plus-Universe/ [2]: https://openai.com/blog/introducing-openai/ https://openai.com/blog/introducing-openai/ [3]: http://web.archive.org/web/20170112173923/https://openai.com/blog/GTA-V-plus-Universe/ http://web.archive.org/web/20170112173923/https://openai.com... [4]: https://github.com/openai/universe-windows-envs/blob/master/vnc-gtav/README.md#using-the-prebuilt-ami https://github.com/openai/universe-windows-envs/blob/master/...
- HappyTypist 10y agoC&D from Blizz is what I heard. Research/commercial use is prohibited
- benkant 10y agoBy Blizzard you mean Rockstar? edit: An issue was opened[0] but there has been no response. I find it hard to believe a C&D comes with a gag order but IANAL. I find it frustrating there's no transparency on this, and I hope I'm not being overly dramatic, but I would have expected more openness here. There could be a very reasonable explanation for keeping mum on this, I guess. [0] https://github.com/openai/universe/issues/126 https://github.com/openai/universe/issues/126
- jaimex2 10y agoAnd there was also a Mario Kart 64 one before that. http://kevinhughes.ca/blog/tensor-kart http://kevinhughes.ca/blog/tensor-kart
- joshu 10y agoI am running http://selfracingcars.com/events http://selfracingcars.com/events in a few months... Udacity has arranged for some folks to run code on their vehicles as well.
- nandhp 10y agoVideo: https://www.youtube.com/watch?v=w6H2eGEvzvw https://www.youtube.com/watch?v=w6H2eGEvzvw
- overcast 10y agoQuite the musical accompaniment.
- nether 10y agoI came to assess the self-driving tech, but now I'm jamming.
- chillaxtian 10y agoagreed - anyone know the source ?
- jeron 10y agoFly Away - Otis McDonald https://www.youtube.com/watch?v=fl6M8FRo2NI https://www.youtube.com/watch?v=fl6M8FRo2NI
- pragmatick 10y agoAs a german I can only think "Gah, why isn't it driving on the right lane?"
- sundvor 10y agoMy thought was, driver must be Australian - as fast lane hogs and a complete, fundamental lack of lane discipline is unfortunately the order of the day here. Still a pretty cool idea to see Euro Truck Sim being used for AI training. I'm impressed.
- yellowapple 10y agoNot only that, but driving in the left lane at 5 kilometers per hour.
- 10y ago
- kijin 10y agoInteresting choice of a name. "Chosun" is not widely used in contemporary South Korea unless you're in a self-deprecating mood ("Hell Chosun") or referring to the Kim dynasty up north. You guys planning to drive to Pyongyang one day in an autonomous truck? :) Anyway, cool project! It's really nice to see what used to be very expensive proprietary technologies become available for anyone with an Ubuntu box to play and experiment with.
- lifthrasiir 10y ago> "Chosun" is not widely used in contemporary South Korea unless you're in a self-deprecating mood ("Hell Chosun") or referring to the Kim dynasty up north. I rather see it as a tongue-in-cheek name, it doesn't necessarily convey a self-deprecating mood (though it somehow implies the comparison itself).
- pmontra 10y agoA mostly off topic side note. It's amazing to see a game with the roads of half continent. I've been playing driving games for so many years that I become fed up with all the grinding (licenses, career path, races blocked by other races, etc) so I'm not buying those games any more. I want all the cars, all the roads, no traffic, no police, only driving. I could pay an extra for that. I wish there is something like Euro Truck with only roads, some fast car and curated lists of interesting road to drive on. And kudos to the authors of ChosunTruck, obviously.
- detaro 10y agoI'm not sure if the engine of ETS lends itself to "fast car", but you could check if there are appropriate mods for it.
- probablybroken 10y agoIt actually only contains a representation of the key roads required for the game; there's really not that much detail in there.
- kruczek 10y agoI've seen normal cars (not trucks) on some servers, so there is certainly an option to drive such cars, although I don't know how they handle.
- dharma1 10y agoWhich servers? ETS2 looks good but not too keen on trucking
- kruczek 10y agoI've played on Europe Server 1 and 3 and on both I've seen normal cars available (although from what I've seen ES1 kicks people back into singleplayer, if they attempt to use a car in multiplayer); ES3 supposedly allows to drive cars in multiplayer as well.
- kogepathic 10y ago
- hasperdi 10y agoIf you want to start learning programming autonomous driving, you might start with Desert Bus* *) http://www.newyorker.com/tech/elements/desert-bus-the-very-worst-video-game-ever-created http://www.newyorker.com/tech/elements/desert-bus-the-very-w...
- notjustanymike 10y agoAlso, if you haven't played ETS2, it's actually quite fantastic.
- doozler 10y agoI appreciate this is a big ask but I find this incredibly interesting and would love to know how it works, would someone mind walking me through some of the main functions please? Which files on GitHub are actually doing the work to keep the truck within the correct lines? Does this learn as it goes or have they somehow trained it before hand?
- omegaham 10y agoI'm doing my part![1] It's not trained; it's entirely programmed. Looking at /windows/src/main.cpp: 1. Grab the game window with `FindWindow("prism3d", NULL)`. 2. Get an image of said game window with `hwnd2mat` (found in windows/src/hwnd2mat.cpp). This part is complicated, as it deals with the Windows API. 2. Apply Median Blur to the image, as shown with this transformation[2]. 3. After creating the 4-point correspondence between the input image (`image`) and the output overlaid image, (`outputImg`), create an IPM object that will perform the Inverse Perspective Mapping. This is a function that turns a flat-angle shot into an overhead shot [3]. 4. Resize the `outputImg` to 320x240 for the smaller window. 5. Convert `outputImg` to grayscale, storing the result in `gray`[4]. 6. Perform the `cv::blur`[5] operation on `gray`, storing the result in `blur`. This reduces noise in the image; there are artifacts gotten from the Inverse Perspective Mapping. 7. Perform the `cv::Sobel`[6] operation on `gray`, storing the result in `sobel`. This gets all of the outlines, including (surprise!) the lines on the road. 8. Perform the `cv::threshold`[7] operation on `sobel`, storing the result in `contours`. Result at this point: A flat perspective of the road has been turned into an overhead perspective, converted to grayscale, blurred, turned into an outline, and then thresholded to solely have the lines on the road. Note that there's an option to perform Canny[8] as well, which is another alternative. 9. Create a `LineFinder` object `ld` (found in /windows/src/linefinder.cpp). 10. Perform `ld.findLines` on `contours`. This performs the `cv::HoughLinesP` operation[9], returning a vector of lines (`vec4i` is a vector of 4 integers that stores x_1, y_1, x_2, y_2). It'll use these to draw the detected lines for informational purposes. --- Okay, now we have a black-and-white image of road-line contours, stored in `contours`. Unfortunately, keyboard and mouse input doesn't work in Windows, so we have to go over to linux/src/main2.cc. Note that the line-finding part is the same thing but requires some Linux-specific stuff to get the original image of the game. Starting from Line 198: 11. Define the center of the image to be `bottom_center` at column 160. Our x-value for the pixel values is going to be `j`. Our y-value is going to be `i`. 12. Create a for loop that iterates through all values from y = 240 to y = 30. Inside this is effectively two loops - one that acts on the x-values from 160 to 10, and one that acts on the x-values from 160 to 310. This loop looks at the pixel in the (j, i) spot in `contours` and sees if it detects a line there. If it does, and it hasn't found a line yet, it sets the `center_to_right` and `center_to_left` values, respectively. This finds the x-values of the left and right lines. I think you know where we're going from here... 13. Create a `centerline`, which is going to be the average of the two. 14. For informational purposes, gather the average `centerline` values and the average `center_to_left` and `center_to_right` values. 15. Move the mouse according to how far away `centerline` is from the center of the image. I don't know how to move in ETS, so I'm going to take their word for it that their gnarly-looking trig function is on-point. [1] https://i.imgur.com/cMfkFGP.gif https://i.imgur.com/cMfkFGP.gif [2] http://docs.opencv.org/3.1.0/median.jpg http://docs.opencv.org/3.1.0/median.jpg [3] https://marcosnietoblog.files.wordpress.com/2014/02/sample.png https://marcosnietoblog.files.wordpress.com/2014/02/sample.p... [4] http://docs.opencv.org/2.4/_images/Load_Save_Image_Result_1.jpg http://docs.opencv.org/2.4/_images/Load_Save_Image_Result_1.... [5] http://docs.opencv.org/3.1.0/blur.jpg http://docs.opencv.org/3.1.0/blur.jpg [6] http://www.mon-club-elec.fr/mes_images/javacvpro/javacvpro_exemple_sobel.png http://www.mon-club-elec.fr/mes_images/javacvpro/javacvpro_e... [7] http://docs.opencv.org/2.4/_images/Threshold_Tutorial_Theory_Example.jpg http://docs.opencv.org/2.4/_images/Threshold_Tutorial_Theory... [8] https://3.bp.blogspot.com/-ybDW9YBG05s/UsvD3HRj7JI/AAAAAAAABWI/5bUbVWpXx-U/s1600/canny.JPG https://3.bp.blogspot.com/-ybDW9YBG05s/UsvD3HRj7JI/AAAAAAAAB... [9] http://answers.opencv.org/upfiles/13661345957308836.jpg http://answers.opencv.org/upfiles/13661345957308836.jpg
- boksiora 10y agowow man this is really amazing!!!!
- zappybiby 10y agoThank you for your interest guys! We went from about 20 stars to 200+ stars, landing us on the 'trending' section of github! We could vastly improve the program by hooking into the game's memory (Such as the totally sick mapping done in https://www.youtube.com/watch?v=3YYyGGgoWfs https://www.youtube.com/watch?v=3YYyGGgoWfs). But this project really just explores computer vision and ML (Kinda, tensorflow is still being worked on).