3 ms·
Gaia-1 a 9B parameter generative world model for autonomous driving
- nkrebs13 3y ago> GAIA-1’s training dataset consists of 4,700 hours of proprietary driving data That doesn't seem like very much
- brucethemoose2 3y ago1 hour of video is a lot of data. The bigger question is how well is it annotated? And how "dense" is the driving action?
- foota 3y agoDon't most of the competitors have like hundreds of thousands to millions of hours of driving?
- SomewhatLikely 3y agoWith lidar point clouds corresponding to every frame...
- brucethemoose2 3y ago...Yeah, that is more like it.
- m00x 3y agoThis looks like self-supervised training to generate new videos, so it's unlikely to be annotated/labeled.
- fnordpiglet 3y ago1 hour of video is a lot of bytes. But 1 hour of driving doesn’t contain much of the total experiential set possible when driving. Generative AI only really can interpolate between things it has been trained on, it can’t extrapolate. This feels like an infinitesimal amount of driving data. That said, a POC doesn’t have to be production worthy. In the hands of a major automobile company perhaps this tech is profoundly powerful. Or perhaps it’s a single model in an ensemble. Regardless it’s going to be interesting where this goes.
- XorNot 3y agoGenerative AI can absolutely extrapolate, that's the whole reason it works. The whole point of machine learning is to derive the underlying rules relating your input data. Extrapolation is just extending where you follow those "curves" beyond bounds of known data points.
- fnordpiglet 3y agoOh but it can’t, but with a sufficiently complex vector space it can seem like it can. What seems like extrapolation is an interpolation in the semantic vector space, particularly in the transformer / attention model. This is a key difference between human intelligence and current AI, it’s not able to “create” and see beyond what it’s been trained on. Any approximation of that is simply indicative of a very complete training set, and it is sufficiently powerful enough to fool people with its expectation based inference - but when you dig into the details of cutting edge stuff you’re an expert in and ask it conceptual questions that extend beyond the semantic corpus embedded in its vector space, it will hallucinate, or if well fine tuned admit lack of knowledge, because the best it can do is interpolate within its own semantic vector space. But listen, I’m a big buyer of generative AI, what it does is incredible. But it’s useful to not ascribe more power to a tool than the math allows. And there are very few machine learning algorithms that do extrapolation at all with any precision. Generally they project an expectation,often of some complex highly dimensional non linear system, which is amazing, but when they are confronted with a novel input pattern they are thrown off. The issue is they’re at their core probabilistic systems, and if the data experiences a regime change that’s unexpected the model will misbehave and output garbage.
- sillysaurusx 3y agoBy this metric, a large number of humans can’t extrapolate either. In fact if you imagine your first paragraph were written about humans, it lines up pretty well.
- uoaei 3y ago
- plaguuuuuu 3y ago1 hour of driving isn't a lot of driving, or a lot of anything really. There are heaps of individual things that happen maybe once or twice in a lifetime that we're already somewhat equipped to deal with based on our other lived experiences - for example, seeing a ball bouncing towards the road and a child or dog chasing after it, I'm already starting to brake before they even approach the edge of the sidewalk. Or seeing a fast-moving wheel bouncing down the road - I know from watching dashcam footage on youtube that you'd want to keep WELL clear of a 30+kg obstacle with a whole bunch of angular momentum, because it will fuck up anything in its path. Good luck to an AI to figure out what's going on, or what the path of a single fast-rolling wheel is going to look like.
- OJFord 3y ago1) how many hours does a pilot need on a 'type' to fly unsupervised? 2) it seems kind of a meaningless unit here, because nobody said it was real world real-time driving hours? And even if it is, if they were logged with the intention of finding 'interesting' scenarios vs. just A-B motorway driving would make an enormous difference.
- krisoft 3y ago> how many hours does a pilot need on a 'type' to fly unsupervised? Excellent question. And the answer is between 45 hours and a few hundred million years. Hear me out. 45 hours is because that is the minimum amount of flying you need to get a private pilot licence: https://www.takeflightaviation.co.uk/ppl-guide.html https://www.takeflightaviation.co.uk/ppl-guide.html (Ignoring here ultra lights and paragliders where you sometimes don't need a licence in some jurisdictions.) So that is the straightforward answer maybe you are looking for. Then we might say that all pilots are required to be at least 17 years old to obtain a private pilot licence. So that is 148920 hours of pre-training in preceiving objects and movement, and coordinating one's actions with perception. Then one might also say that one requires to be a human, and that comes with hundreds of millions of years of pre-training where all of our ancestors were evolutionary selected to be good at perceiving and moving. (At least good enough to survive until they could propagate their genes.) Now this answer might come of as flippant, and maybe it is. What I'm trying to say is that it is hard to compare "training hours" directly between computers and humans. And it is hard because of these two things which humans have "pretraining by lived experience" and "pretraining by evolution".
- kombine 3y agoThis method is unsupervised and does not train on annotated data. They use a similar approach to GPT: autoregressive sequence modelling.
- jojobas 3y agoWhere I live, protein drivers are allowed to drive solo after merely 120 hours of supervised driving, few places are more stringent that this. 4700 hours of driving is super-much.
- Loocid 3y ago"Protein drivers" can learn on a much smaller set of data than AI can.
- jojobas 3y agoMaybe. Maybe they incorporate a much larger corpus of knowledge gathered outside driving into everyday and emergency driving, and maybe AI drivers should too.
- melling 3y agoHumans also have 16 years of learning… wait do we really need to cover this again?
- jacquesm 3y agoThey're allowed to drive solo after passing an exam.
- deleted 3y ago[deleted]
- mzs 3y agoYeah that's not snow on the intro vid. Snow falls, sticks, and covers.
- londons_explore 3y agoIt's UK snow. The UK doesn't get much snow - London only sees snow for like 1 day some years.
- ben_w 3y agoAnd it's newsworthy when there's even this much, such is the standard for a "White Christmas".
- londons_explore 3y agoWith global warming heading the way it is, we may in our lifetimes see the last ever snow in London. I doubt already that we'll ever see the Thames freeze over again - it last happened in 1963, and used to happen about once per decade till then.
- nealabq 3y agoIs this how we finally get full self-driving? Maybe have enough compute in the car to take care of mundane cruising and emergency braking, checking with a centralized/shared expert-driver AI in exceptional circumstances. Maybe the expert-driver AI contacts an even-more-expert AI in even-more-exceptional circumstances. Etc. AIs all the way up. Imagine the implications.
- inductive_magic 3y ago>Imagine the implications. > exceptional circumstance appears > no network > implications
- otabdeveloper4 3y agoWe will never have self-driving. Any attempt to sell this technology is a pump-and-dump scam. We have had fully autonomous airplanes and trains for decades now. The technology has been shelved because of legal reasons, not because it doesn't work. Self-driving technology will only last until the first child gets run over by a self-driving car, at which point the technology will be banned. Lawyers will need a fall guy to take the blame, and since tech bro corporations have a EULA that absolves them of all responsibility that will be the end of self-driving.
- jeroenhd 3y agoIf companies take responsibility for their technology, the technology is viable. Various low speed self driving vehicles are already deployed all around the world. Low speed to reduce inevitable injury, which keeps risk to the company behind them down. The problem with self driving technology is that companies like Tesla want to keep you liable for the decisions their AI makes. That's why you have to sit behind the wheel and still keep an eye out. Here in the Netherlands there were several tests with self driving trains, but the systems didn't prove satisfactory yet. The systems are being worked on, though. Putting passengers in the hands of automated systems requires a lot of care and responsibility, and a few companies are willing to take that risk (for plenty of money, of course). As for planes, autopilot is used very often. Automated landing is mandatory in some runways to keep time between planes down, though pilots don't seem to like the aggressive touchdown these systems provide. We'll never have fully autonomous planes because someone needs to be around when the plane runs into trouble; computers won't land you on the Hudson.
- jackblemming 3y agoWho is their target audience for this? Places like Tesla and co already have more than enough real training data and their own simulators.
- zoklet-enjoyer 3y agohttps://www.comma.ai/openpilot https://www.comma.ai/openpilot DIY kits for upgrading cars
- reliablereason 3y agoComma ai already has this type of thing. https://youtu.be/EqQNZXqzFSI?si=23Dm88Raesce4bij https://youtu.be/EqQNZXqzFSI?si=23Dm88Raesce4bij [Learning a Driving Simulator | Yassine Yousfi | COMMA_CON 2023]
- potatoman22 3y agoProbably other large car manufacturers
- Chirono 3y agoWayve is developing their own autonomous vehicle system, so I think they are their own target audience here.
- sampo 3y ago> generate realistic driving videos Does it only generate video? Or does it tell you, with coordinates and geometry, where the nearby cars actually are located?
- potatoman22 3y agoJust video. Other techniques are better for getting the geometries.
- DanHulton 3y agoAnd what happens when your car hallucinates at highway speeds?
- Waterluvian 3y agoProbably the same thing as when I hallucinate at highway speeds.
- m00x 3y agoSo, typical Monday morning.
- bamboozled 3y agoClearly, you’re not focusing on the upside, naughty you.
- zelicious 3y ago[flagged]
- xcodevn 3y agoSo this is what it looks like inside a car's dreams.
- throw310822 3y agoDo autopilots dream of electric cars?
- isoprophlex 3y agoImpressive temporal stability in those videos!
- kaitocross 3y agoSeems like a good method to generate Dashcam footage for fun