3 ms·

Not everything seems to be open source. I can't seem to find the vision part, only the binary, see: https://github.com/commaai/openpilot/tree/master/selfdrive/v

Not everything seems to be open source. I can't seem to find the vision part, only the binary, see: https://github.com/commaai/openpilot/tree/master/selfdrive/visiond https://github.com/commaai/openpilot/tree/master/selfdrive/v...

bri3d 10y ago
Based on a really cursory inspection of the binary and symbols, the binary seems to be an OpenCV project which uses some common image processing algorithms (Sobel edge detection, Hough to find straight lines). There doesn't seem to be a large amount of embedded data indicative of a trained model or neural network. So, it seems that the vision algorithms probably mirror those in the common literature and represent a reimplementation or iterative improvement on existing lane-finding systems. More reversing would probably yield greater insight :)
- paulbaumgart 10y ago
  I think it's also using Qualcomm's FastCV, judging by the fcv* prefix on a lot of the symbols. Which makes sense, since FastCV is optimized for the Snapdragon processor in the OnePlus 3 phone they use.
- Voloskaya 10y ago
  I did the same thing as you, and found reference to RNNs, ReLus etc... That was the part I was really looking forward to see open sourced, but well :)
- commaai 10y ago
  There's a neural network in there :)
  - bri3d 10y ago
    Ha, straight from the horse's mouth - I stand corrected :) I think this will be the first binary I've pulled apart with one, look forward to it!
Animats 10y ago
Looking at the code, the good stuff is in the binary blob. The binary blob returns polygons for the edges of the road, with a probability for each. Path planning, in "pathplanner.py", consists of a probability-weighted average to find the center of the lane, using an assumption of 3.6m lane width. There's no map-building at all. It's purely reactive. Speed control uses data from a radar and from the vision system. The path info is used to decide which targets the radar should consider important. The big thing seems to be identifying the "car ahead", if any, and maintaining an appropriate distance. There's a hard-coded 6 minute timeout to make the driver take control once in a while. On a good day on a well-marked freeway, this might work. It would be interesting to modify this to process pre-recorded dashcam data and see how well it tracks the road.
DrWhat 10y ago
visiond appears to be linked against libgsl.so. To my understanding, distributing a binary blob using the GNU Scientific Library is against the license of this library. That written there does not seem to be any symbol used from libgsl.so.
- espes 10y ago
  It links agains the Adreno-GSL, part of the GPU driver, not the GNU Scientific library.
guomanmin 10y ago
Participate in Atlassian Research My name is Angela and I do research for Bitbucket. I’m kicking off a round of discussions with people who use Git tools. Ideally, I’d like to talk to people that sit on a team of 3 or more. If this is you, I would love to talk to you about your experience with <using> Git tools, or just some of the pain points that are keeping you up at night when doing your jobs.    We’ll just need 30 mins of your time, and as a token of my thanks to those that participate, I’d like to offer a US$50 Amazon gift voucher.    If you’re interested, just shoot me an email with your availability over the next few weeks and we can set up a time to chat for 30 minutes. Please also include your timezone so we can schedule a suitable time (as I’m located in San Francisco). Hope to talk to you soon! Cheers,   Angela Guo aguo@atlassian.com

bri3d 10y ago

Based on a really cursory inspection of the binary and symbols, the binary seems to be an OpenCV project which uses some common image processing algorithms (Sobel edge detection, Hough to find straight lines). There doesn't seem to be a large amount of embedded data indicative of a trained model or neural network. So, it seems that the vision algorithms probably mirror those in the common literature and represent a reimplementation or iterative improvement on existing lane-finding systems. More reversing would probably yield greater insight :)

paulbaumgart 10y ago
I think it's also using Qualcomm's FastCV, judging by the fcv* prefix on a lot of the symbols. Which makes sense, since FastCV is optimized for the Snapdragon processor in the OnePlus 3 phone they use.
Voloskaya 10y ago
I did the same thing as you, and found reference to RNNs, ReLus etc... That was the part I was really looking forward to see open sourced, but well :)
commaai 10y ago
There's a neural network in there :)
- bri3d 10y ago
  Ha, straight from the horse's mouth - I stand corrected :) I think this will be the first binary I've pulled apart with one, look forward to it!

Animats 10y ago

Looking at the code, the good stuff is in the binary blob. The binary blob returns polygons for the edges of the road, with a probability for each. Path planning, in "pathplanner.py", consists of a probability-weighted average to find the center of the lane, using an assumption of 3.6m lane width. There's no map-building at all. It's purely reactive. Speed control uses data from a radar and from the vision system. The path info is used to decide which targets the radar should consider important. The big thing seems to be identifying the "car ahead", if any, and maintaining an appropriate distance. There's a hard-coded 6 minute timeout to make the driver take control once in a while. On a good day on a well-marked freeway, this might work. It would be interesting to modify this to process pre-recorded dashcam data and see how well it tracks the road.

DrWhat 10y ago

visiond appears to be linked against libgsl.so. To my understanding, distributing a binary blob using the GNU Scientific Library is against the license of this library. That written there does not seem to be any symbol used from libgsl.so.

espes 10y ago
It links agains the Adreno-GSL, part of the GPU driver, not the GNU Scientific library.

guomanmin 10y ago

Participate in Atlassian Research My name is Angela and I do research for Bitbucket. I’m kicking off a round of discussions with people who use Git tools. Ideally, I’d like to talk to people that sit on a team of 3 or more. If this is you, I would love to talk to you about your experience with <using> Git tools, or just some of the pain points that are keeping you up at night when doing your jobs.    We’ll just need 30 mins of your time, and as a token of my thanks to those that participate, I’d like to offer a US$50 Amazon gift voucher.    If you’re interested, just shoot me an email with your availability over the next few weeks and we can set up a time to chat for 30 minutes. Please also include your timezone so we can schedule a suitable time (as I’m located in San Francisco). Hope to talk to you soon! Cheers,   Angela Guo aguo@atlassian.com