4 ms·

1 hour of video is a lot of data. The bigger question is how well is it annotated? And how "dense" is the driving action?

foota 3y ago
Don't most of the competitors have like hundreds of thousands to millions of hours of driving?
- SomewhatLikely 3y ago
  With lidar point clouds corresponding to every frame...
  - brucethemoose2 3y ago
    ...Yeah, that is more like it.
m00x 3y ago
This looks like self-supervised training to generate new videos, so it's unlikely to be annotated/labeled.
fnordpiglet 3y ago
1 hour of video is a lot of bytes. But 1 hour of driving doesn’t contain much of the total experiential set possible when driving. Generative AI only really can interpolate between things it has been trained on, it can’t extrapolate. This feels like an infinitesimal amount of driving data. That said, a POC doesn’t have to be production worthy. In the hands of a major automobile company perhaps this tech is profoundly powerful. Or perhaps it’s a single model in an ensemble. Regardless it’s going to be interesting where this goes.
- XorNot 3y ago
  Generative AI can absolutely extrapolate, that's the whole reason it works. The whole point of machine learning is to derive the underlying rules relating your input data. Extrapolation is just extending where you follow those "curves" beyond bounds of known data points.
  - fnordpiglet 3y ago
    Oh but it can’t, but with a sufficiently complex vector space it can seem like it can. What seems like extrapolation is an interpolation in the semantic vector space, particularly in the transformer / attention model. This is a key difference between human intelligence and current AI, it’s not able to “create” and see beyond what it’s been trained on. Any approximation of that is simply indicative of a very complete training set, and it is sufficiently powerful enough to fool people with its expectation based inference - but when you dig into the details of cutting edge stuff you’re an expert in and ask it conceptual questions that extend beyond the semantic corpus embedded in its vector space, it will hallucinate, or if well fine tuned admit lack of knowledge, because the best it can do is interpolate within its own semantic vector space. But listen, I’m a big buyer of generative AI, what it does is incredible. But it’s useful to not ascribe more power to a tool than the math allows. And there are very few machine learning algorithms that do extrapolation at all with any precision. Generally they project an expectation,often of some complex highly dimensional non linear system, which is amazing, but when they are confronted with a novel input pattern they are thrown off. The issue is they’re at their core probabilistic systems, and if the data experiences a regime change that’s unexpected the model will misbehave and output garbage.
    - sillysaurusx 3y ago
      By this metric, a large number of humans can’t extrapolate either. In fact if you imagine your first paragraph were written about humans, it lines up pretty well.
      - uoaei 3y ago
        Don't be a dick, you know what is meant.
        sillysaurusx 3y ago
        Huh? I disagree with the premise, and explained why. Reading over your comments for the last few days, you seem consistently aggressive. If you need to vent to someone about something, you can DM me. Happy to just listen.
- candiodari 3y ago
  But ... all this goes for humans too! Is the argument that we should just outlaw driving alltogether, all possible forms? One famous example of that is how to react correctly when the car starts to slip due to speed, braking, or driving on water, mud, sand, snow or ice. I think everyone knows people's reflexes are to floor the brakes, and start wildly turning the steering wheel, which only results in total loss of control over the vehicle. Is anyone demanding drivers learn to correctly handle cars or other vehicles under these circumnstances? There are only very minimal efforts, because it is completely impractical to teach many humans better driving practices. So we just accept the flaws ... and the constant stream of victims this generates. Reality: Yes, a driving AI is not ready for all possible situations. It just isn't. It will never be. Is that a problem? Also reality: Humans drive drunk. Humans drive while under the influence of drugs. Humans drive trucks near kids when they're so tired they can't keep their head lifted up. And roads are full of dead cats, squirrels, mice, ... Also also reality: AI driving software can, after an accident, be taught to handle the situation that caused the accident, and the result of this learning process can then be uploaded to all instances of this software. Humans will keep making the same mistakes, with the same consequences, over and over and over again. Perhaps there is very slow improvement (mostly by modifying roads), but it takes decades. Practical view: I have driven around in Mountain View next to self-driving cars. One thing's for sure: self-driving cars behave much better than human drivers. Including me. It's very irritating how good they behave on the road. If the roads have many self-driving cars, I'm pretty damn sure it'll result in much fewer accidents and lower transit times. Never mind that self-driving cars of course solve the parking problem. I don't get why people hate them. And I hate this goalpost moving where AIs are compared to multiple top-performing humans, that you see everywhere. Of course, there are now cases where AIs have actually beaten groups of top-performing humans (translation, chess, Go, robot control, ...)
  - moe_sc 3y ago
    Your arguments don't really fit, what was previously said. There was nothing said against driving AI in general, just that 4700h of videos seems low. I also get that humans are pretty bad drivers, but isn't that exactly why we shouldn't use them as the baseline for AIs to compare to? We are now at a point where we can set high standards for AI, so we get a best possible result, because while it isn't feasible to have everyone learn driving over a couple of years, a good AI has to be trained once and can be used by many, so we have the time. And sure, it can be updated, but should we really trust companies to keep innovating once they are already allowed to have the AI in use? The incentive to do so is far bigger if they have to do so before they got any money out of it.
plaguuuuuu 3y ago
1 hour of driving isn't a lot of driving, or a lot of anything really. There are heaps of individual things that happen maybe once or twice in a lifetime that we're already somewhat equipped to deal with based on our other lived experiences - for example, seeing a ball bouncing towards the road and a child or dog chasing after it, I'm already starting to brake before they even approach the edge of the sidewalk. Or seeing a fast-moving wheel bouncing down the road - I know from watching dashcam footage on youtube that you'd want to keep WELL clear of a 30+kg obstacle with a whole bunch of angular momentum, because it will fuck up anything in its path. Good luck to an AI to figure out what's going on, or what the path of a single fast-rolling wheel is going to look like.
- OJFord 3y ago
  1) how many hours does a pilot need on a 'type' to fly unsupervised? 2) it seems kind of a meaningless unit here, because nobody said it was real world real-time driving hours? And even if it is, if they were logged with the intention of finding 'interesting' scenarios vs. just A-B motorway driving would make an enormous difference.
  - krisoft 3y ago
    > how many hours does a pilot need on a 'type' to fly unsupervised? Excellent question. And the answer is between 45 hours and a few hundred million years. Hear me out. 45 hours is because that is the minimum amount of flying you need to get a private pilot licence: https://www.takeflightaviation.co.uk/ppl-guide.html https://www.takeflightaviation.co.uk/ppl-guide.html (Ignoring here ultra lights and paragliders where you sometimes don't need a licence in some jurisdictions.) So that is the straightforward answer maybe you are looking for. Then we might say that all pilots are required to be at least 17 years old to obtain a private pilot licence. So that is 148920 hours of pre-training in preceiving objects and movement, and coordinating one's actions with perception. Then one might also say that one requires to be a human, and that comes with hundreds of millions of years of pre-training where all of our ancestors were evolutionary selected to be good at perceiving and moving. (At least good enough to survive until they could propagate their genes.) Now this answer might come of as flippant, and maybe it is. What I'm trying to say is that it is hard to compare "training hours" directly between computers and humans. And it is hard because of these two things which humans have "pretraining by lived experience" and "pretraining by evolution".
kombine 3y ago
This method is unsupervised and does not train on annotated data. They use a similar approach to GPT: autoregressive sequence modelling.

foota 3y ago

Don't most of the competitors have like hundreds of thousands to millions of hours of driving?

SomewhatLikely 3y ago
With lidar point clouds corresponding to every frame...
- brucethemoose2 3y ago
  ...Yeah, that is more like it.

m00x 3y ago

This looks like self-supervised training to generate new videos, so it's unlikely to be annotated/labeled.

fnordpiglet 3y ago

1 hour of video is a lot of bytes. But 1 hour of driving doesn’t contain much of the total experiential set possible when driving. Generative AI only really can interpolate between things it has been trained on, it can’t extrapolate. This feels like an infinitesimal amount of driving data. That said, a POC doesn’t have to be production worthy. In the hands of a major automobile company perhaps this tech is profoundly powerful. Or perhaps it’s a single model in an ensemble. Regardless it’s going to be interesting where this goes.

XorNot 3y ago
Generative AI can absolutely extrapolate, that's the whole reason it works. The whole point of machine learning is to derive the underlying rules relating your input data. Extrapolation is just extending where you follow those "curves" beyond bounds of known data points.
- fnordpiglet 3y ago
  Oh but it can’t, but with a sufficiently complex vector space it can seem like it can. What seems like extrapolation is an interpolation in the semantic vector space, particularly in the transformer / attention model. This is a key difference between human intelligence and current AI, it’s not able to “create” and see beyond what it’s been trained on. Any approximation of that is simply indicative of a very complete training set, and it is sufficiently powerful enough to fool people with its expectation based inference - but when you dig into the details of cutting edge stuff you’re an expert in and ask it conceptual questions that extend beyond the semantic corpus embedded in its vector space, it will hallucinate, or if well fine tuned admit lack of knowledge, because the best it can do is interpolate within its own semantic vector space. But listen, I’m a big buyer of generative AI, what it does is incredible. But it’s useful to not ascribe more power to a tool than the math allows. And there are very few machine learning algorithms that do extrapolation at all with any precision. Generally they project an expectation,often of some complex highly dimensional non linear system, which is amazing, but when they are confronted with a novel input pattern they are thrown off. The issue is they’re at their core probabilistic systems, and if the data experiences a regime change that’s unexpected the model will misbehave and output garbage.
  - sillysaurusx 3y ago
    By this metric, a large number of humans can’t extrapolate either. In fact if you imagine your first paragraph were written about humans, it lines up pretty well.
    - uoaei 3y ago
      Don't be a dick, you know what is meant.
      - sillysaurusx 3y ago
        Huh? I disagree with the premise, and explained why. Reading over your comments for the last few days, you seem consistently aggressive. If you need to vent to someone about something, you can DM me. Happy to just listen.
candiodari 3y ago
But ... all this goes for humans too! Is the argument that we should just outlaw driving alltogether, all possible forms? One famous example of that is how to react correctly when the car starts to slip due to speed, braking, or driving on water, mud, sand, snow or ice. I think everyone knows people's reflexes are to floor the brakes, and start wildly turning the steering wheel, which only results in total loss of control over the vehicle. Is anyone demanding drivers learn to correctly handle cars or other vehicles under these circumnstances? There are only very minimal efforts, because it is completely impractical to teach many humans better driving practices. So we just accept the flaws ... and the constant stream of victims this generates. Reality: Yes, a driving AI is not ready for all possible situations. It just isn't. It will never be. Is that a problem? Also reality: Humans drive drunk. Humans drive while under the influence of drugs. Humans drive trucks near kids when they're so tired they can't keep their head lifted up. And roads are full of dead cats, squirrels, mice, ... Also also reality: AI driving software can, after an accident, be taught to handle the situation that caused the accident, and the result of this learning process can then be uploaded to all instances of this software. Humans will keep making the same mistakes, with the same consequences, over and over and over again. Perhaps there is very slow improvement (mostly by modifying roads), but it takes decades. Practical view: I have driven around in Mountain View next to self-driving cars. One thing's for sure: self-driving cars behave much better than human drivers. Including me. It's very irritating how good they behave on the road. If the roads have many self-driving cars, I'm pretty damn sure it'll result in much fewer accidents and lower transit times. Never mind that self-driving cars of course solve the parking problem. I don't get why people hate them. And I hate this goalpost moving where AIs are compared to multiple top-performing humans, that you see everywhere. Of course, there are now cases where AIs have actually beaten groups of top-performing humans (translation, chess, Go, robot control, ...)
- moe_sc 3y ago
  Your arguments don't really fit, what was previously said. There was nothing said against driving AI in general, just that 4700h of videos seems low. I also get that humans are pretty bad drivers, but isn't that exactly why we shouldn't use them as the baseline for AIs to compare to? We are now at a point where we can set high standards for AI, so we get a best possible result, because while it isn't feasible to have everyone learn driving over a couple of years, a good AI has to be trained once and can be used by many, so we have the time. And sure, it can be updated, but should we really trust companies to keep innovating once they are already allowed to have the AI in use? The incentive to do so is far bigger if they have to do so before they got any money out of it.

plaguuuuuu 3y ago

1 hour of driving isn't a lot of driving, or a lot of anything really. There are heaps of individual things that happen maybe once or twice in a lifetime that we're already somewhat equipped to deal with based on our other lived experiences - for example, seeing a ball bouncing towards the road and a child or dog chasing after it, I'm already starting to brake before they even approach the edge of the sidewalk. Or seeing a fast-moving wheel bouncing down the road - I know from watching dashcam footage on youtube that you'd want to keep WELL clear of a 30+kg obstacle with a whole bunch of angular momentum, because it will fuck up anything in its path. Good luck to an AI to figure out what's going on, or what the path of a single fast-rolling wheel is going to look like.

OJFord 3y ago
1) how many hours does a pilot need on a 'type' to fly unsupervised? 2) it seems kind of a meaningless unit here, because nobody said it was real world real-time driving hours? And even if it is, if they were logged with the intention of finding 'interesting' scenarios vs. just A-B motorway driving would make an enormous difference.
- krisoft 3y ago
  > how many hours does a pilot need on a 'type' to fly unsupervised? Excellent question. And the answer is between 45 hours and a few hundred million years. Hear me out. 45 hours is because that is the minimum amount of flying you need to get a private pilot licence: https://www.takeflightaviation.co.uk/ppl-guide.html https://www.takeflightaviation.co.uk/ppl-guide.html (Ignoring here ultra lights and paragliders where you sometimes don't need a licence in some jurisdictions.) So that is the straightforward answer maybe you are looking for. Then we might say that all pilots are required to be at least 17 years old to obtain a private pilot licence. So that is 148920 hours of pre-training in preceiving objects and movement, and coordinating one's actions with perception. Then one might also say that one requires to be a human, and that comes with hundreds of millions of years of pre-training where all of our ancestors were evolutionary selected to be good at perceiving and moving. (At least good enough to survive until they could propagate their genes.) Now this answer might come of as flippant, and maybe it is. What I'm trying to say is that it is hard to compare "training hours" directly between computers and humans. And it is hard because of these two things which humans have "pretraining by lived experience" and "pretraining by evolution".

kombine 3y ago

This method is unsupervised and does not train on annotated data. They use a similar approach to GPT: autoregressive sequence modelling.