4 ms·

> And then we have comma, which controls your vehicle on a non-realtime system... Lol. This is what I found extremely shocking and is the first thing that I no

> And then we have comma, which controls your vehicle on a non-realtime system... Lol. This is what I found extremely shocking and is the first thing that I noticed. I don't think anyone serious about embedded design would touch it. They are trying to ignore decades of experience in real time systems and build some hacky stuff and make it open source.

traverseda 7y ago
I don't understand how you'd do hard real-time when one of the major inputs, the neural-networks, aren't deterministic? In a hard real-time system, what happens if your neural-networks are late telling you where the lanes are? In reality the comma-AI isn't controlling the car, a number of peripherals on the canbus are controlling the car. All those peripherals are hard-real time, and the comma-AI is bridging the non-deterministic neural networks and those hard-real-time peripherals. At some point you need to deal with the non-deterministic neural networks, and bridge them into a hard real-time control system. People complaining that part of the system isn't real-time just seems like they haven't thought about it very much. Of course the UI and neural networks aren't going to be hard real-time.
- etaioinshrdlu 7y ago
  Neural networks are very much real-time. They generally consist of a fixed series of math operations of fixed size. Only the weights and inputs change. The neural net will never be late. It just might be inaccurate.
  - traverseda 7y ago
    Huh, I suppose they aren't inherently incompatible with real-time constraints. It looks like it's pretty challenging to do in practice though. I'm still somewhat confused as to how multi-object recognition works in a real-time way, naively I'd presume that recognizing n+1 objects in a frame would take longer than n objects. You might be able to more practically get real-time neural networks using something like YOLO (you only look once)? Either way it seems to me like hard real-time neural networks are a very challenging problem.

traverseda 7y ago

I don't understand how you'd do hard real-time when one of the major inputs, the neural-networks, aren't deterministic? In a hard real-time system, what happens if your neural-networks are late telling you where the lanes are? In reality the comma-AI isn't controlling the car, a number of peripherals on the canbus are controlling the car. All those peripherals are hard-real time, and the comma-AI is bridging the non-deterministic neural networks and those hard-real-time peripherals. At some point you need to deal with the non-deterministic neural networks, and bridge them into a hard real-time control system. People complaining that part of the system isn't real-time just seems like they haven't thought about it very much. Of course the UI and neural networks aren't going to be hard real-time.

etaioinshrdlu 7y ago
Neural networks are very much real-time. They generally consist of a fixed series of math operations of fixed size. Only the weights and inputs change. The neural net will never be late. It just might be inaccurate.
- traverseda 7y ago
  Huh, I suppose they aren't inherently incompatible with real-time constraints. It looks like it's pretty challenging to do in practice though. I'm still somewhat confused as to how multi-object recognition works in a real-time way, naively I'd presume that recognizing n+1 objects in a frame would take longer than n objects. You might be able to more practically get real-time neural networks using something like YOLO (you only look once)? Either way it seems to me like hard real-time neural networks are a very challenging problem.