3 ms·

At this point I think just having the composability without the compactness would be a major breakthrough. Heck, even just figuring out how to efficiently harne

by m0zg 5y ago

At this point I think just having the composability without the compactness would be a major breakthrough. Heck, even just figuring out how to efficiently harness multimodal inputs or better handle inputs that vary over time (e.g. video) would be a breakthrough as well. Case in point: nearly all academic models for object detection are borderline useless in practice because they work on individual frames, rather than on video, and as a result they aren't very robust and they lose objects frame to frame for no good reason. There are tracking models, but they are far less researched than single-frame ones, and much more complex. In essence, it's like that drunk who was looking for his car keys under a street lamp rather than where he lost them because it's easier to search for them there. People don't realize how little of this surface we've scratched so far, and how awesome this stuff will get once we properly scratch it all over.

joe_the_user 5y ago
At this point I think just having the composability without the compactness would be a major breakthrough. I'd claim that practical composibility without compactness impossible. Being huge, like a deep neural net, makes composability intractable. If your "elements" are akin to deep nets, composition will be very, very hard to calculate and so impossible to train.
- m0zg 5y ago
  By "composability" I mean you'd be able to train models separately and then perhaps slightly refine the composed model as a whole on the final task(s). And models would be using each other's outputs, sort of like today you can take an ImageNet backbone and put an object detection head on top, except at a higher level. I.e. an optical (+IR) multi-camera object tracking model, fused with lidar and radar, feeding into a path planning model. As of today, I have just described a billion dollars worth of modeling at least. That needs to be simpler and cheaper. Currently stuff like this has to be trained end to end, which, as you've pointed out, is very difficult to do, to the point where there are only a handful of companies that can convincingly pull it off (Tesla, Google, OpenAI, _maybe_ Facebook), and even they are arguably at a local minimum this approach allows. And this local minimum won't get us to "real" AI, which I think was the point of this rather rambling article.
captaindiego 5y ago
On the topic of handling multi-modal detections over time, this is often termed "object tracking" in academic literature. If you wanna dig deeper, check out this youtube channel: https://www.youtube.com/channel/UCa2-fpj6AV8T6JK1uTRuFpw/featured https://www.youtube.com/channel/UCa2-fpj6AV8T6JK1uTRuFpw/fea... there are also a couple universities with classes on this topic with lecture notes online. Often solutions in this space are not ML based, focusing on probabilistic approaches for associating and segmenting data from ML-based single frame detections. This sort of pipeline is a standard feature of many autonomous vehicle stacks.
- m0zg 5y ago
  I work in this field, I could create my own "youtube channel" about it, if I didn't have better things to do. This doesn't change the fact that the field is relatively nascent, and a lot of people are researching stuff that doesn't really matter all that much, at the expense of the stuff that does matter.

by m0zg 5y ago

joe_the_user 5y ago

At this point I think just having the composability without the compactness would be a major breakthrough. I'd claim that practical composibility without compactness impossible. Being huge, like a deep neural net, makes composability intractable. If your "elements" are akin to deep nets, composition will be very, very hard to calculate and so impossible to train.

m0zg 5y ago
By "composability" I mean you'd be able to train models separately and then perhaps slightly refine the composed model as a whole on the final task(s). And models would be using each other's outputs, sort of like today you can take an ImageNet backbone and put an object detection head on top, except at a higher level. I.e. an optical (+IR) multi-camera object tracking model, fused with lidar and radar, feeding into a path planning model. As of today, I have just described a billion dollars worth of modeling at least. That needs to be simpler and cheaper. Currently stuff like this has to be trained end to end, which, as you've pointed out, is very difficult to do, to the point where there are only a handful of companies that can convincingly pull it off (Tesla, Google, OpenAI, _maybe_ Facebook), and even they are arguably at a local minimum this approach allows. And this local minimum won't get us to "real" AI, which I think was the point of this rather rambling article.

captaindiego 5y ago

On the topic of handling multi-modal detections over time, this is often termed "object tracking" in academic literature. If you wanna dig deeper, check out this youtube channel: https://www.youtube.com/channel/UCa2-fpj6AV8T6JK1uTRuFpw/featured https://www.youtube.com/channel/UCa2-fpj6AV8T6JK1uTRuFpw/fea... there are also a couple universities with classes on this topic with lecture notes online. Often solutions in this space are not ML based, focusing on probabilistic approaches for associating and segmenting data from ML-based single frame detections. This sort of pipeline is a standard feature of many autonomous vehicle stacks.

m0zg 5y ago
I work in this field, I could create my own "youtube channel" about it, if I didn't have better things to do. This doesn't change the fact that the field is relatively nascent, and a lot of people are researching stuff that doesn't really matter all that much, at the expense of the stuff that does matter.