4 ms·

> GAIA-1’s training dataset consists of 4,700 hours of proprietary driving data That doesn't seem like very much

brucethemoose2 3y ago
1 hour of video is a lot of data. The bigger question is how well is it annotated? And how "dense" is the driving action?
- foota 3y ago
  Don't most of the competitors have like hundreds of thousands to millions of hours of driving?
  - SomewhatLikely 3y ago
    With lidar point clouds corresponding to every frame...
    - brucethemoose2 3y ago
      ...Yeah, that is more like it.
- m00x 3y ago
  This looks like self-supervised training to generate new videos, so it's unlikely to be annotated/labeled.
- fnordpiglet 3y ago
  1 hour of video is a lot of bytes. But 1 hour of driving doesn’t contain much of the total experiential set possible when driving. Generative AI only really can interpolate between things it has been trained on, it can’t extrapolate. This feels like an infinitesimal amount of driving data. That said, a POC doesn’t have to be production worthy. In the hands of a major automobile company perhaps this tech is profoundly powerful. Or perhaps it’s a single model in an ensemble. Regardless it’s going to be interesting where this goes.
  - XorNot 3y ago
    Generative AI can absolutely extrapolate, that's the whole reason it works. The whole point of machine learning is to derive the underlying rules relating your input data. Extrapolation is just extending where you follow those "curves" beyond bounds of known data points.
    - fnordpiglet 3y ago
      Oh but it can’t, but with a sufficiently complex vector space it can seem like it can. What seems like extrapolation is an interpolation in the semantic vector space, particularly in the transformer / attention model. This is a key difference between human intelligence and current AI, it’s not able to “create” and see beyond what it’s been trained on. Any approximation of that is simply indicative of a very complete training set, and it is sufficiently powerful enough to fool people with its expectation based inference - but when you dig into the details of cutting edge stuff you’re an expert in and ask it conceptual questions that extend beyond the semantic corpus embedded in its vector space, it will hallucinate, or if well fine tuned admit lack of knowledge, because the best it can do is interpolate within its own semantic vector space. But listen, I’m a big buyer of generative AI, what it does is incredible. But it’s useful to not ascribe more power to a tool than the math allows. And there are very few machine learning algorithms that do extrapolation at all with any precision. Generally they project an expectation,often of some complex highly dimensional non linear system, which is amazing, but when they are confronted with a novel input pattern they are thrown off. The issue is they’re at their core probabilistic systems, and if the data experiences a regime change that’s unexpected the model will misbehave and output garbage.
      - sillysaurusx 3y ago
        By this metric, a large number of humans can’t extrapolate either. In fact if you imagine your first paragraph were written about humans, it lines up pretty well.
        uoaei 3y ago
- plaguuuuuu 3y ago
  1 hour of driving isn't a lot of driving, or a lot of anything really. There are heaps of individual things that happen maybe once or twice in a lifetime that we're already somewhat equipped to deal with based on our other lived experiences - for example, seeing a ball bouncing towards the road and a child or dog chasing after it, I'm already starting to brake before they even approach the edge of the sidewalk. Or seeing a fast-moving wheel bouncing down the road - I know from watching dashcam footage on youtube that you'd want to keep WELL clear of a 30+kg obstacle with a whole bunch of angular momentum, because it will fuck up anything in its path. Good luck to an AI to figure out what's going on, or what the path of a single fast-rolling wheel is going to look like.
  - OJFord 3y ago
    1) how many hours does a pilot need on a 'type' to fly unsupervised? 2) it seems kind of a meaningless unit here, because nobody said it was real world real-time driving hours? And even if it is, if they were logged with the intention of finding 'interesting' scenarios vs. just A-B motorway driving would make an enormous difference.
    - krisoft 3y ago
      > how many hours does a pilot need on a 'type' to fly unsupervised? Excellent question. And the answer is between 45 hours and a few hundred million years. Hear me out. 45 hours is because that is the minimum amount of flying you need to get a private pilot licence: https://www.takeflightaviation.co.uk/ppl-guide.html https://www.takeflightaviation.co.uk/ppl-guide.html (Ignoring here ultra lights and paragliders where you sometimes don't need a licence in some jurisdictions.) So that is the straightforward answer maybe you are looking for. Then we might say that all pilots are required to be at least 17 years old to obtain a private pilot licence. So that is 148920 hours of pre-training in preceiving objects and movement, and coordinating one's actions with perception. Then one might also say that one requires to be a human, and that comes with hundreds of millions of years of pre-training where all of our ancestors were evolutionary selected to be good at perceiving and moving. (At least good enough to survive until they could propagate their genes.) Now this answer might come of as flippant, and maybe it is. What I'm trying to say is that it is hard to compare "training hours" directly between computers and humans. And it is hard because of these two things which humans have "pretraining by lived experience" and "pretraining by evolution".
- kombine 3y ago
  This method is unsupervised and does not train on annotated data. They use a similar approach to GPT: autoregressive sequence modelling.
jojobas 3y ago
Where I live, protein drivers are allowed to drive solo after merely 120 hours of supervised driving, few places are more stringent that this. 4700 hours of driving is super-much.
- Loocid 3y ago
  "Protein drivers" can learn on a much smaller set of data than AI can.
  - jojobas 3y ago
    Maybe. Maybe they incorporate a much larger corpus of knowledge gathered outside driving into everyday and emergency driving, and maybe AI drivers should too.
- melling 3y ago
  Humans also have 16 years of learning… wait do we really need to cover this again?
- jacquesm 3y ago
  They're allowed to drive solo after passing an exam.

1 hour of video is a lot of data. The bigger question is how well is it annotated? And how "dense" is the driving action?

foota 3y ago
Don't most of the competitors have like hundreds of thousands to millions of hours of driving?
- SomewhatLikely 3y ago
  With lidar point clouds corresponding to every frame...
  - brucethemoose2 3y ago
    ...Yeah, that is more like it.
m00x 3y ago
This looks like self-supervised training to generate new videos, so it's unlikely to be annotated/labeled.
fnordpiglet 3y ago
1 hour of video is a lot of bytes. But 1 hour of driving doesn’t contain much of the total experiential set possible when driving. Generative AI only really can interpolate between things it has been trained on, it can’t extrapolate. This feels like an infinitesimal amount of driving data. That said, a POC doesn’t have to be production worthy. In the hands of a major automobile company perhaps this tech is profoundly powerful. Or perhaps it’s a single model in an ensemble. Regardless it’s going to be interesting where this goes.
- XorNot 3y ago
  Generative AI can absolutely extrapolate, that's the whole reason it works. The whole point of machine learning is to derive the underlying rules relating your input data. Extrapolation is just extending where you follow those "curves" beyond bounds of known data points.
  - fnordpiglet 3y ago
    Oh but it can’t, but with a sufficiently complex vector space it can seem like it can. What seems like extrapolation is an interpolation in the semantic vector space, particularly in the transformer / attention model. This is a key difference between human intelligence and current AI, it’s not able to “create” and see beyond what it’s been trained on. Any approximation of that is simply indicative of a very complete training set, and it is sufficiently powerful enough to fool people with its expectation based inference - but when you dig into the details of cutting edge stuff you’re an expert in and ask it conceptual questions that extend beyond the semantic corpus embedded in its vector space, it will hallucinate, or if well fine tuned admit lack of knowledge, because the best it can do is interpolate within its own semantic vector space. But listen, I’m a big buyer of generative AI, what it does is incredible. But it’s useful to not ascribe more power to a tool than the math allows. And there are very few machine learning algorithms that do extrapolation at all with any precision. Generally they project an expectation,often of some complex highly dimensional non linear system, which is amazing, but when they are confronted with a novel input pattern they are thrown off. The issue is they’re at their core probabilistic systems, and if the data experiences a regime change that’s unexpected the model will misbehave and output garbage.
    - sillysaurusx 3y ago
      By this metric, a large number of humans can’t extrapolate either. In fact if you imagine your first paragraph were written about humans, it lines up pretty well.
      - uoaei 3y ago
plaguuuuuu 3y ago
1 hour of driving isn't a lot of driving, or a lot of anything really. There are heaps of individual things that happen maybe once or twice in a lifetime that we're already somewhat equipped to deal with based on our other lived experiences - for example, seeing a ball bouncing towards the road and a child or dog chasing after it, I'm already starting to brake before they even approach the edge of the sidewalk. Or seeing a fast-moving wheel bouncing down the road - I know from watching dashcam footage on youtube that you'd want to keep WELL clear of a 30+kg obstacle with a whole bunch of angular momentum, because it will fuck up anything in its path. Good luck to an AI to figure out what's going on, or what the path of a single fast-rolling wheel is going to look like.
- OJFord 3y ago
  1) how many hours does a pilot need on a 'type' to fly unsupervised? 2) it seems kind of a meaningless unit here, because nobody said it was real world real-time driving hours? And even if it is, if they were logged with the intention of finding 'interesting' scenarios vs. just A-B motorway driving would make an enormous difference.
  - krisoft 3y ago
    > how many hours does a pilot need on a 'type' to fly unsupervised? Excellent question. And the answer is between 45 hours and a few hundred million years. Hear me out. 45 hours is because that is the minimum amount of flying you need to get a private pilot licence: https://www.takeflightaviation.co.uk/ppl-guide.html https://www.takeflightaviation.co.uk/ppl-guide.html (Ignoring here ultra lights and paragliders where you sometimes don't need a licence in some jurisdictions.) So that is the straightforward answer maybe you are looking for. Then we might say that all pilots are required to be at least 17 years old to obtain a private pilot licence. So that is 148920 hours of pre-training in preceiving objects and movement, and coordinating one's actions with perception. Then one might also say that one requires to be a human, and that comes with hundreds of millions of years of pre-training where all of our ancestors were evolutionary selected to be good at perceiving and moving. (At least good enough to survive until they could propagate their genes.) Now this answer might come of as flippant, and maybe it is. What I'm trying to say is that it is hard to compare "training hours" directly between computers and humans. And it is hard because of these two things which humans have "pretraining by lived experience" and "pretraining by evolution".
kombine 3y ago
This method is unsupervised and does not train on annotated data. They use a similar approach to GPT: autoregressive sequence modelling.

jojobas 3y ago

Where I live, protein drivers are allowed to drive solo after merely 120 hours of supervised driving, few places are more stringent that this. 4700 hours of driving is super-much.

Loocid 3y ago
"Protein drivers" can learn on a much smaller set of data than AI can.
- jojobas 3y ago
  Maybe. Maybe they incorporate a much larger corpus of knowledge gathered outside driving into everyday and emergency driving, and maybe AI drivers should too.
melling 3y ago
Humans also have 16 years of learning… wait do we really need to cover this again?
jacquesm 3y ago
They're allowed to drive solo after passing an exam.