6 ms·

Berkeley Deep Drive Dataset

300 points by MuhammedAbiola 8y ago

arielbaz 8y ago
First, thanks for sharing this data. Second - why on earth would anyone create a 1.8TB zip file of 100k videos? Likely the video encoder already compressed every possible bit out of these videos, zip is not going to make it smaller. It is, however, going to make it mandatory for everyone to download the full 1.8TB file even to get a single video out of this archive. Makes me wonder what else is happening here (like the chrome only download link which is hosted on another domain, and the non-https login, and that escalator to nowhere..*)
adpirz 8y ago
In case you don't want to register and are curious about some metadata: Videos: 100K video clips: Size:1.8TB Info: The GPS/IMU information recorded along with the videos: Size: 3.9GB Images It has two subfolders. 1) 100K labeled key frame images extracted from the videos at 10th second 2) 10K key frames for full-frame semantic segmentation.: Size: 6.5GB Labels: Annotations of road objects, lanes, and drivable areas in JSON format. Details at Github repo.: Size: 147MB Drivable Maps: Segmentation maps of Drivable areas.: Size: 661MB Segmentation: Full-frame semantic segmentation maps. The corresponding images are in the same folder.: Size: 1.2GB
- sytelus 8y ago
  While this is largest public annotated dataset, it's still only from 2 or 3 cities if I'm reading this right.
  - Isamu 8y ago
    4 regions - see page 7 of the paper. New York, Berkeley, San Francisco, and "Bay Area" https://arxiv.org/pdf/1805.04687.pdf https://arxiv.org/pdf/1805.04687.pdf
quickben 8y ago
The website doesn't work in Firefox. "FAQ: The download buttons do not work". "The website is fully supported by Chrome now" The current state of the internet, has reached a very low point. One would expect more from berkley.edu. EDIT: I find the downvotes preposterous. Are we somehow supposed to expect requiring a proprietary browser to simply download a file from an educational institution now?
- ironjunkie 8y ago
  Since when is Chrome a reference? If you want something not proprietary, why don't you use Firefox?
  - quickben 8y ago
    That's what I accessed it with actually. It didn't work in Firefox. I clarified my post above.
  - MBCook 8y ago
    It’s in DOWNLOAD button. How does it not work in EVERY browser? A link would. A form submission would. Ultra simple JavaScript would. It’s not a question of “why doesn’t chrome work“ but more a question of “how is this even an issue“.
    - ironjunkie 8y ago
      In this case I agree with you. I was just pointing out that we should be careful with Chrome being used as a "reference".
      - r1nkgrl 8y ago
        FWIW Chrome does have the largest market share of any desktop internet browser. That said, I understand what you are getting at and agree.
      - MBCook 8y ago
        That part I totally agree with. I’m really tired of people declaring other browsers “broken“ because they don’t implement the future-of-the-minute that chrome has already added.
trillic 8y ago
License: """ Copyright ©2018. The Regents of the University of California (Regents). All Rights Reserved. Permission to use, copy, modify, and distribute this software and its documentation for educational, research, and not-for-profit purposes, without fee and without a signed licensing agreement; and permission use, copy, modify and distribute this software for commercial purposes (such rights not subject to transfer) to BDD member and its affiliates, is hereby granted, provided that the above copyright notice, this paragraph and the following two paragraphs appear in all copies, modifications, and distributions. Contact The Office of Technology Licensing, UC Berkeley, 2150 Shattuck Avenue, Suite 510, Berkeley, CA 94720-1620, (510) 643-7201, otl@berkeley.edu, http://ipira.berkeley.edu/industry-info http://ipira.berkeley.edu/industry-info for commercial licensing opportunities. IN NO EVENT SHALL REGENTS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF REGENTS HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. REGENTS SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE AND ACCOMPANYING DOCUMENTATION, IF ANY, PROVIDED HEREUNDER IS PROVIDED "AS IS". REGENTS HAS NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. """
- paxy 8y ago
  BDD = Berkeley Deep Drive (https://deepdrive.berkeley.edu/ https://deepdrive.berkeley.edu/)
- rectang 8y ago
  If I'm reading that right, it's not an open source license. It has a field-of-use restriction because only Berkeley Deep Dive members can use it for commercial purposes. EDIT: The title of this HN topic is wrong. It's not what's in the source and it to be changed. (I'm relieved that it's just a submitter summarizing incorrectly and that Berkeley Deep Dive was not responsible for this mistake.)
  - mehrdadn 8y ago
    > If I'm reading that right, it's not an open source license. It has a field-of-use restriction How do usage terms dictate whether the source is open?
faitswulff 8y ago
Tangent - does anyone know if Waymo is user-generated driving data from Navigation mode on Google Maps for their self-driving data? Or if that would even be feasible or useful?
- henryfjordan 8y ago
  GPS + IMU data from phones might be useful but neither is particularly accurate compared to video data. Maybe the IMU data would be useful to make the car feel more "human"
dominias 8y ago
Interesting to see China winning the competition
- mycall 8y ago
  Next up, public bus footage, if all the ducks become aligned.
Animats 8y ago
This is nice, but it assumes that grinding on lots of successful driving video is a valid path to automatic driving.
- taneq 8y ago
  Presumably lots of successful driving video is at least a prerequisite to validating an automatic driving system, even if you're not using it for training?
itchyjunk 8y ago
I am curious, does training the AI on other driving data sets help? What I means is, not just sedan data set. Trucks, buses, maybe 2 wheelers? Would this help the model generalize more and make better prediction of how other vehicles work or would it just add noise?
- cr4zy 8y ago
  It helps a lot in my experience. So in simulation I tried this with imitation learning by training on a hood camera, a camera at the height of what a semi-truck hood would be, and another camera offset to the left 1.5m along with steering and throttle for labels. I also added random noise to the position (less than a meter), rotation (less than degree), fov (less than a degree), capture height (< 1%), and capture width (< 1%). The result was a 3x higher average score on a driving benchmark where the score was meters driven minus seconds taken, second-meters of lane deviation, and seconds where acceleration surpassed 0.5g forces (to measure comfort). The dataset, training code, and sim are at deepdrive.io - a different entity with the same name :)
sanxiyn 8y ago
Is there a similar dataset for lidar? My understanding was that lidar is more important than video. Has it changed?
rck 8y ago
The license seems a bit odd. It refers to software, but I can't see any software in any of the downloads - just data. Clearly the license is intended to cover the contents of the downloads, but the wording seems wrong then. I'm not a lawyer, so maybe someone with more expertise could chime in...
bytematic 8y ago
This made me wonder, how will driver-less cars interpret roads that have very faded, non-existent, or unique lane markings? I imagine rural, park, construction, and weather like snow/dust can really obscure things.
jamesblonde 8y ago
I am downloading this dataset. It will take about 40 days, is my guess....
- nerdponx 8y ago
  How would one go about creating a torrent for it? Or uploading to IPFS?
  - web007 8y ago
    I already emailed the creator a couple weeks back to request / offer a torrent, but haven't heard anything back. The problem here is that both of your suggestions involve a 2-step process: 1. Download the file 2. Create a torrent from it, or upload it to IPFS Since step 1 is already a 2TB download, getting to either version of step 2 is untenable. I agree with one of the other posters in this thread, the default for something like this should be torrent since you get both distribution and checksumming for free. It would also be nice if it wasn't a 2TB zip file, which then has to be unzipped onto another 2TB of storage for practical use.
    - jamesblonde 8y ago
      Subject to licensing, we intend to make the dataset available (along with loads of other big datasets for ML) using a bit-torrent like program called Dela for the Hops Hadoop platform. Maybe in 3 weeks or so, it will all be released - with this dataset. Dela integrates with HDFS/S3/GCS backends, and it supports NAT traversal, and a delay-based congestion control over UDP - good for high bandwidth/high latency networks. See http://www.hops.io http://www.hops.io and our paper - https://ieeexplore.ieee.org/document/7980225/ https://ieeexplore.ieee.org/document/7980225/
syntaxing 8y ago
Before I register to download the data, is there a smaller dataset for you to play with on the portal? I've been itching to do something fun after taking my SDC from Udacity. Nut1.8TB is way more than I can handle right now. Can someone upload a portion of this? (<10GB)?
syntaxing 8y ago
I'm confused...are the datasets free (as in for profit and nonprofit) to use and the pretrained models (mentioned in the paper) are under the UC license?
supergirl 8y ago
good luck downloading them with 10KB/s.
Uberphallus 8y ago
Does it include accidents and incidents with correct maneuvers? That's the training set hard to come by.

First, thanks for sharing this data. Second - why on earth would anyone create a 1.8TB zip file of 100k videos? Likely the video encoder already compressed every possible bit out of these videos, zip is not going to make it smaller. It is, however, going to make it mandatory for everyone to download the full 1.8TB file even to get a single video out of this archive. Makes me wonder what else is happening here (like the chrome only download link which is hosted on another domain, and the non-https login, and that escalator to nowhere..*)

adpirz 8y ago

In case you don't want to register and are curious about some metadata: Videos: 100K video clips: Size:1.8TB Info: The GPS/IMU information recorded along with the videos: Size: 3.9GB Images It has two subfolders. 1) 100K labeled key frame images extracted from the videos at 10th second 2) 10K key frames for full-frame semantic segmentation.: Size: 6.5GB Labels: Annotations of road objects, lanes, and drivable areas in JSON format. Details at Github repo.: Size: 147MB Drivable Maps: Segmentation maps of Drivable areas.: Size: 661MB Segmentation: Full-frame semantic segmentation maps. The corresponding images are in the same folder.: Size: 1.2GB

sytelus 8y ago
While this is largest public annotated dataset, it's still only from 2 or 3 cities if I'm reading this right.
- Isamu 8y ago
  4 regions - see page 7 of the paper. New York, Berkeley, San Francisco, and "Bay Area" https://arxiv.org/pdf/1805.04687.pdf https://arxiv.org/pdf/1805.04687.pdf

quickben 8y ago

The website doesn't work in Firefox. "FAQ: The download buttons do not work". "The website is fully supported by Chrome now" The current state of the internet, has reached a very low point. One would expect more from berkley.edu. EDIT: I find the downvotes preposterous. Are we somehow supposed to expect requiring a proprietary browser to simply download a file from an educational institution now?

ironjunkie 8y ago
Since when is Chrome a reference? If you want something not proprietary, why don't you use Firefox?
- quickben 8y ago
  That's what I accessed it with actually. It didn't work in Firefox. I clarified my post above.
- MBCook 8y ago
  It’s in DOWNLOAD button. How does it not work in EVERY browser? A link would. A form submission would. Ultra simple JavaScript would. It’s not a question of “why doesn’t chrome work“ but more a question of “how is this even an issue“.
  - ironjunkie 8y ago
    In this case I agree with you. I was just pointing out that we should be careful with Chrome being used as a "reference".
    - r1nkgrl 8y ago
      FWIW Chrome does have the largest market share of any desktop internet browser. That said, I understand what you are getting at and agree.
    - MBCook 8y ago
      That part I totally agree with. I’m really tired of people declaring other browsers “broken“ because they don’t implement the future-of-the-minute that chrome has already added.

trillic 8y ago

License: """ Copyright ©2018. The Regents of the University of California (Regents). All Rights Reserved. Permission to use, copy, modify, and distribute this software and its documentation for educational, research, and not-for-profit purposes, without fee and without a signed licensing agreement; and permission use, copy, modify and distribute this software for commercial purposes (such rights not subject to transfer) to BDD member and its affiliates, is hereby granted, provided that the above copyright notice, this paragraph and the following two paragraphs appear in all copies, modifications, and distributions. Contact The Office of Technology Licensing, UC Berkeley, 2150 Shattuck Avenue, Suite 510, Berkeley, CA 94720-1620, (510) 643-7201, otl@berkeley.edu, http://ipira.berkeley.edu/industry-info http://ipira.berkeley.edu/industry-info for commercial licensing opportunities. IN NO EVENT SHALL REGENTS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF REGENTS HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. REGENTS SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE AND ACCOMPANYING DOCUMENTATION, IF ANY, PROVIDED HEREUNDER IS PROVIDED "AS IS". REGENTS HAS NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. """

paxy 8y ago
BDD = Berkeley Deep Drive (https://deepdrive.berkeley.edu/ https://deepdrive.berkeley.edu/)
rectang 8y ago
If I'm reading that right, it's not an open source license. It has a field-of-use restriction because only Berkeley Deep Dive members can use it for commercial purposes. EDIT: The title of this HN topic is wrong. It's not what's in the source and it to be changed. (I'm relieved that it's just a submitter summarizing incorrectly and that Berkeley Deep Dive was not responsible for this mistake.)
- mehrdadn 8y ago
  > If I'm reading that right, it's not an open source license. It has a field-of-use restriction How do usage terms dictate whether the source is open?

faitswulff 8y ago

Tangent - does anyone know if Waymo is user-generated driving data from Navigation mode on Google Maps for their self-driving data? Or if that would even be feasible or useful?

henryfjordan 8y ago
GPS + IMU data from phones might be useful but neither is particularly accurate compared to video data. Maybe the IMU data would be useful to make the car feel more "human"

dominias 8y ago

Interesting to see China winning the competition

mycall 8y ago
Next up, public bus footage, if all the ducks become aligned.

Animats 8y ago

This is nice, but it assumes that grinding on lots of successful driving video is a valid path to automatic driving.

taneq 8y ago
Presumably lots of successful driving video is at least a prerequisite to validating an automatic driving system, even if you're not using it for training?

itchyjunk 8y ago

I am curious, does training the AI on other driving data sets help? What I means is, not just sedan data set. Trucks, buses, maybe 2 wheelers? Would this help the model generalize more and make better prediction of how other vehicles work or would it just add noise?

cr4zy 8y ago
It helps a lot in my experience. So in simulation I tried this with imitation learning by training on a hood camera, a camera at the height of what a semi-truck hood would be, and another camera offset to the left 1.5m along with steering and throttle for labels. I also added random noise to the position (less than a meter), rotation (less than degree), fov (less than a degree), capture height (< 1%), and capture width (< 1%). The result was a 3x higher average score on a driving benchmark where the score was meters driven minus seconds taken, second-meters of lane deviation, and seconds where acceleration surpassed 0.5g forces (to measure comfort). The dataset, training code, and sim are at deepdrive.io - a different entity with the same name :)

sanxiyn 8y ago

Is there a similar dataset for lidar? My understanding was that lidar is more important than video. Has it changed?

rck 8y ago

The license seems a bit odd. It refers to software, but I can't see any software in any of the downloads - just data. Clearly the license is intended to cover the contents of the downloads, but the wording seems wrong then. I'm not a lawyer, so maybe someone with more expertise could chime in...

bytematic 8y ago

This made me wonder, how will driver-less cars interpret roads that have very faded, non-existent, or unique lane markings? I imagine rural, park, construction, and weather like snow/dust can really obscure things.

jamesblonde 8y ago

I am downloading this dataset. It will take about 40 days, is my guess....

nerdponx 8y ago
How would one go about creating a torrent for it? Or uploading to IPFS?
- web007 8y ago
  I already emailed the creator a couple weeks back to request / offer a torrent, but haven't heard anything back. The problem here is that both of your suggestions involve a 2-step process: 1. Download the file 2. Create a torrent from it, or upload it to IPFS Since step 1 is already a 2TB download, getting to either version of step 2 is untenable. I agree with one of the other posters in this thread, the default for something like this should be torrent since you get both distribution and checksumming for free. It would also be nice if it wasn't a 2TB zip file, which then has to be unzipped onto another 2TB of storage for practical use.
  - jamesblonde 8y ago
    Subject to licensing, we intend to make the dataset available (along with loads of other big datasets for ML) using a bit-torrent like program called Dela for the Hops Hadoop platform. Maybe in 3 weeks or so, it will all be released - with this dataset. Dela integrates with HDFS/S3/GCS backends, and it supports NAT traversal, and a delay-based congestion control over UDP - good for high bandwidth/high latency networks. See http://www.hops.io http://www.hops.io and our paper - https://ieeexplore.ieee.org/document/7980225/ https://ieeexplore.ieee.org/document/7980225/

syntaxing 8y ago

Before I register to download the data, is there a smaller dataset for you to play with on the portal? I've been itching to do something fun after taking my SDC from Udacity. Nut1.8TB is way more than I can handle right now. Can someone upload a portion of this? (<10GB)?

syntaxing 8y ago

I'm confused...are the datasets free (as in for profit and nonprofit) to use and the pretrained models (mentioned in the paper) are under the UC license?

supergirl 8y ago

good luck downloading them with 10KB/s.

Uberphallus 8y ago

Does it include accidents and incidents with correct maneuvers? That's the training set hard to come by.