5 ms·

I think this is a critical part of cracking self-driving cars - "scene understanding", or more accurately, identification AND MODELLING of all traffic participa

by ddalex 3y ago

I think this is a critical part of cracking self-driving cars - "scene understanding", or more accurately, identification AND MODELLING of all traffic participants. A tiger chasing a deer will identify and model what the deer will do, what the terrain is, and take appropriate decisions on how to best pursue; a good driver will do the same - e.g. understanding that a round object bouncing in the street of a residential neighborhood is a ball, and where a ball comes, a child will follow. I also think that this feedback loop of "Model the world, take action, observe, update model" is a key component of consciousness (whereas we model ourselves!) and key to AGI. I would not be surprised that AGI comes from a self-driving car development model.

nunez 3y ago
I'm guessing this is one of the things that Tesla's Dojo supercomputer combined with the billions of miles collected from their fleet is trying to solve
- ddalex 3y ago
  I'm guessing not; what NNs do is pattern matching. Basically they try to "remember" what should be done at in a given situation, but does not have continuous representation and modelling of other objects; not even "object permanence" - when a car disappears from their world, it's gone.
  - peterfirefly 3y ago
    Tesla's FSD does actually have a degree of object permanence. It would be a lot of worse if it didn't.
moffkalast 3y ago
Maybe it's not reliable enough as-is, but seems to me that this "something odd is going on, better slow down or stop" requirement might be entirely solvable with anomaly detection. When the data is not as it normally is, take extra care. It's not that complicated and AGI is hardly required.
- semiquaver 3y ago
  The problem is “something odd is going on” requires precisely the “scene understanding” that is currently absent from all AVs. The reality is that nearly all sensory input is novel in some way and a large part of the work the human brain does is quietly filtering out a massive amount of irrelevant input in a way that is computationally infeasible with current technology.
bryanlarsen 3y ago
OTOH, this is not what people do in emergency situation; the modelling loop is just too slow. In emergencies people need to rely on reactions which are built through training and practice. A good modeller can often avoid the emergency situation (i.e. your ball example), but once in an emergency situation it's a reaction situation.
- mnw21cam 3y ago
  The difficult part of the UK driving test is doing exactly that modelling loop. I agree with you. It's not about responding to an emergency - yes the loop would be too slow for that. It's about foreseeing the emergency before it happens. Looking ahead further down the road for things that could turn into hazards that should be prepared for ahead of time. It's credited with reducing the accident rate, though I'm not entirely sure how they worked that out.
  - HPsquared 3y ago
    For those outside the UK, this is called the Hazard Perception Test https://www.gov.uk/theory-test/hazard-perception-test https://www.gov.uk/theory-test/hazard-perception-test
- sottol 3y ago
  Imo our reflexes differ based on context. But I'm not sure if it's the reflexive system chosing one of N options that are always available or the planner that "activates" just a few options based on the current context.

by ddalex 3y ago

nunez 3y ago

I'm guessing this is one of the things that Tesla's Dojo supercomputer combined with the billions of miles collected from their fleet is trying to solve

ddalex 3y ago
I'm guessing not; what NNs do is pattern matching. Basically they try to "remember" what should be done at in a given situation, but does not have continuous representation and modelling of other objects; not even "object permanence" - when a car disappears from their world, it's gone.
- peterfirefly 3y ago
  Tesla's FSD does actually have a degree of object permanence. It would be a lot of worse if it didn't.

moffkalast 3y ago

Maybe it's not reliable enough as-is, but seems to me that this "something odd is going on, better slow down or stop" requirement might be entirely solvable with anomaly detection. When the data is not as it normally is, take extra care. It's not that complicated and AGI is hardly required.

semiquaver 3y ago
The problem is “something odd is going on” requires precisely the “scene understanding” that is currently absent from all AVs. The reality is that nearly all sensory input is novel in some way and a large part of the work the human brain does is quietly filtering out a massive amount of irrelevant input in a way that is computationally infeasible with current technology.

bryanlarsen 3y ago

OTOH, this is not what people do in emergency situation; the modelling loop is just too slow. In emergencies people need to rely on reactions which are built through training and practice. A good modeller can often avoid the emergency situation (i.e. your ball example), but once in an emergency situation it's a reaction situation.

mnw21cam 3y ago
The difficult part of the UK driving test is doing exactly that modelling loop. I agree with you. It's not about responding to an emergency - yes the loop would be too slow for that. It's about foreseeing the emergency before it happens. Looking ahead further down the road for things that could turn into hazards that should be prepared for ahead of time. It's credited with reducing the accident rate, though I'm not entirely sure how they worked that out.
- HPsquared 3y ago
  For those outside the UK, this is called the Hazard Perception Test https://www.gov.uk/theory-test/hazard-perception-test https://www.gov.uk/theory-test/hazard-perception-test
sottol 3y ago
Imo our reflexes differ based on context. But I'm not sure if it's the reflexive system chosing one of N options that are always available or the planner that "activates" just a few options based on the current context.