4 ms·

Once Dataset: One Million Scenes for Autonomous Driving

93 points by lnyan 5y ago

villgax 5y ago
We should also get a simulator for self driving car manufacturers, that way you can evaluate them on accidents that happen & see how each one would react, also comparing to the average driver in the same situation for baselines
- ansible 5y ago
  People have been doing something like that with GTA-V. If there is another engine that provides good rendering and a better physics model for cars, that would be even better.
  - villgax 5y ago
    I think what you meant is probably for training self-driving, what I meant is for testing scenarios with whatever is built, complete with all the inputs it needs etc but yeah some engine which allows manufacturers to extend for their. A high-tech term for this is hardware-in-the-loop
Animats 5y ago
These are stills? That's not too useful. You don't get to see how things came out, and you don't get a later close-up of things you previously saw at distance. If you wanted to train, say, a distant traffic light recognizer, you could do that by training one to recognize close-ups of traffic lights. Then tag close traffic lights in video with it, track the tagged object back to when it first appeared, and that becomes a new tagged training case for distant traffic lights. Same for other objects.
timdaub 5y ago
Shameless plug, I run a website that lists data sets by their markets' performance: https://rugpullindex.com https://rugpullindex.com Damn it would be cool if I could rank this data set too...
- galuggus 5y ago
  People sell datasets on eth? How does that work? What kind of data is being sold and who's buying?
  - timdaub 5y ago
    How does it work? See https://oceanprotocol.com https://oceanprotocol.com What kind of data? Check on https://rugpullindex.com https://rugpullindex.com for a summary ("Data Set" column) Who is buying? There's speculation. I don't think there's any real consumption yet.
    - littlestymaar 5y ago
      > There's speculation. I don't think there's any real consumption yet. Blockchain tech in a nutshell.
      - timdaub 5y ago
        From the HN Guidelines in section "In comments": > Don't be snarky.
        meepmorp 5y ago
        A bit further down: > Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. Maybe that's reasonable formulation of their opinion, and you shouldn't assume it's snark.
        mumblemumble 5y ago
        I think that the strongest plausible interpretation of this whole thread is that AI is officially on the downward trend of the hype curve. We're now so bored with the subject that, in the HN comments section for an autonomous driving data set, we'd mostly rather just argue about blockchain. Speaking as a jaded professional data scientist, I couldn't be more pleased about this development.
Taniwha 5y ago
Doesn't say which side of the road it's for driving on, nor which country's road rules are in effect (though , since it's from Huawei I'm guessing it's RHS/China)
- mumblemumble 5y ago
  In the example images, cars are driving on the right and signage has Chinese text.
galuggus 5y ago
Is there value in dashcam footage? Or does a dataset require input from other sensors to be valuable?
- ansible 5y ago
  If it is annotated, it is quite useful for various recognition tasks, alone, or combined with the other sensor data.
mumblemumble 5y ago
> 15k fully annotated scenes with 5 classes (Car, Bus, Truck, Pedestrian, Cyclist) It strikes me that this isn't nearly enough classes. Other important ones that I can think of offhand include deer (other animals too - recognizing the species helps with anticipating how they might move), playground ball, shopping cart, fallen limb, bicycle flag... I could possibly go on all morning. Even looking at the list has me thinking that treating this as a simple classification subtask is possibly going about it in the wrong way. At least in my city, you need to be able to make finer distinctions, such as between a pedestrian stepping out into the street to see if a bus is coming or look for a taxicab, a panhandler standing on the centerline, or a pedestrian stepping out into the street to cross it. This is a legitimate safety concern for city driving - if a self-driving car hits the brakes every time it sees someone looking for a bus, it's only a matter of time until someone rear-ends it. And, while it should definitely give that panhandler some space, stopping in the middle of a busy intersection to let them clear the road when they have no intention of doing so is going to excessively obstruct traffic, and possibly also create a further safety hazard as cars behind it maneuver to try and get around. I'm not sure figuring these things out is really a classification problem in the first place. It seems like it's more of a, for lack of a better term, "inferring intent" problem. Part of what I think might make that distinct is that classification assumes you're working with a categorical variable - a finite set of distinct classes. Intentions that are relevant to the driver tend to clump into a few distinct categories, but they're perhaps better modeled as fuzzy regions in a continuous vector space than as a categorical variable. Which isn't to say that datasets like this are useless. Progress is progress, obviously. But it seems possible that, in the long view, the most important thing that all of this progress might have accomplished thus far is helping us to get out of the Dunning-Kruger phase of our understanding of the problem.
web007 5y ago
I wish academics and practitioners would talk more often or have better interfaces to prevent things like "our dataset is published on Google Drive". Every one of the files is now rate-limited (blocked) for 24 hours because Drive is not designed to be used this way. If this was published as one or more torrents it would be ideal: all of those who have downloaded the full set would be able to contribute bandwidth to help newcomers, and could spare the source from having to serve multiple TBs of content directly. Uploading to Archive.org and/or the Chinese equivalent would be even better, since they provide mirroring plus torrents plus direct downloads automatically and for free.

93 points by lnyan 5y ago

We should also get a simulator for self driving car manufacturers, that way you can evaluate them on accidents that happen & see how each one would react, also comparing to the average driver in the same situation for baselines

ansible 5y ago
People have been doing something like that with GTA-V. If there is another engine that provides good rendering and a better physics model for cars, that would be even better.
- villgax 5y ago
  I think what you meant is probably for training self-driving, what I meant is for testing scenarios with whatever is built, complete with all the inputs it needs etc but yeah some engine which allows manufacturers to extend for their. A high-tech term for this is hardware-in-the-loop

Animats 5y ago

These are stills? That's not too useful. You don't get to see how things came out, and you don't get a later close-up of things you previously saw at distance. If you wanted to train, say, a distant traffic light recognizer, you could do that by training one to recognize close-ups of traffic lights. Then tag close traffic lights in video with it, track the tagged object back to when it first appeared, and that becomes a new tagged training case for distant traffic lights. Same for other objects.

timdaub 5y ago

Shameless plug, I run a website that lists data sets by their markets' performance: https://rugpullindex.com https://rugpullindex.com Damn it would be cool if I could rank this data set too...

galuggus 5y ago
People sell datasets on eth? How does that work? What kind of data is being sold and who's buying?
- timdaub 5y ago
  How does it work? See https://oceanprotocol.com https://oceanprotocol.com What kind of data? Check on https://rugpullindex.com https://rugpullindex.com for a summary ("Data Set" column) Who is buying? There's speculation. I don't think there's any real consumption yet.
  - littlestymaar 5y ago
    > There's speculation. I don't think there's any real consumption yet. Blockchain tech in a nutshell.
    - timdaub 5y ago
      From the HN Guidelines in section "In comments": > Don't be snarky.
      - meepmorp 5y ago
        A bit further down: > Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. Maybe that's reasonable formulation of their opinion, and you shouldn't assume it's snark.
        mumblemumble 5y ago
        I think that the strongest plausible interpretation of this whole thread is that AI is officially on the downward trend of the hype curve. We're now so bored with the subject that, in the HN comments section for an autonomous driving data set, we'd mostly rather just argue about blockchain. Speaking as a jaded professional data scientist, I couldn't be more pleased about this development.

Taniwha 5y ago

Doesn't say which side of the road it's for driving on, nor which country's road rules are in effect (though , since it's from Huawei I'm guessing it's RHS/China)

mumblemumble 5y ago
In the example images, cars are driving on the right and signage has Chinese text.

galuggus 5y ago

Is there value in dashcam footage? Or does a dataset require input from other sensors to be valuable?

ansible 5y ago
If it is annotated, it is quite useful for various recognition tasks, alone, or combined with the other sensor data.

mumblemumble 5y ago

> 15k fully annotated scenes with 5 classes (Car, Bus, Truck, Pedestrian, Cyclist) It strikes me that this isn't nearly enough classes. Other important ones that I can think of offhand include deer (other animals too - recognizing the species helps with anticipating how they might move), playground ball, shopping cart, fallen limb, bicycle flag... I could possibly go on all morning. Even looking at the list has me thinking that treating this as a simple classification subtask is possibly going about it in the wrong way. At least in my city, you need to be able to make finer distinctions, such as between a pedestrian stepping out into the street to see if a bus is coming or look for a taxicab, a panhandler standing on the centerline, or a pedestrian stepping out into the street to cross it. This is a legitimate safety concern for city driving - if a self-driving car hits the brakes every time it sees someone looking for a bus, it's only a matter of time until someone rear-ends it. And, while it should definitely give that panhandler some space, stopping in the middle of a busy intersection to let them clear the road when they have no intention of doing so is going to excessively obstruct traffic, and possibly also create a further safety hazard as cars behind it maneuver to try and get around. I'm not sure figuring these things out is really a classification problem in the first place. It seems like it's more of a, for lack of a better term, "inferring intent" problem. Part of what I think might make that distinct is that classification assumes you're working with a categorical variable - a finite set of distinct classes. Intentions that are relevant to the driver tend to clump into a few distinct categories, but they're perhaps better modeled as fuzzy regions in a continuous vector space than as a categorical variable. Which isn't to say that datasets like this are useless. Progress is progress, obviously. But it seems possible that, in the long view, the most important thing that all of this progress might have accomplished thus far is helping us to get out of the Dunning-Kruger phase of our understanding of the problem.

web007 5y ago

I wish academics and practitioners would talk more often or have better interfaces to prevent things like "our dataset is published on Google Drive". Every one of the files is now rate-limited (blocked) for 24 hours because Drive is not designed to be used this way. If this was published as one or more torrents it would be ideal: all of those who have downloaded the full set would be able to contribute bandwidth to help newcomers, and could spare the source from having to serve multiple TBs of content directly. Uploading to Archive.org and/or the Chinese equivalent would be even better, since they provide mirroring plus torrents plus direct downloads automatically and for free.