3 ms·

Not true. You can code your prior knowledge into the model, and make do with way less x/y pairs. Data is a substitute for doing engineering by hand. If you do

Not true. You can code your prior knowledge into the model, and make do with way less x/y pairs. Data is a substitute for doing engineering by hand. If you do the latter, you require less data.

Quarrelsome 8y ago
I don't see how it makes my statement untrue. The most effective ML techniques are more effective the more x/y pairs you have. Sure, you can get a leg up if you partially define F yourself but then you're no longer being as hands off as before. In practice the WHOLE of F() will likely be a blend of machine learning as well as heuristics as a product of various layers that are one or the other.
- gugagore 8y ago
  Not the poster you're replying to, but it's just that you said "that's not how it works", and then suggested that the only way to get F is through data, even if it means generating synthetic data. So that's wrong and untrue, as you recognize in this latest comment by saying you can partially define F. To make a concrete example, the convolution operator, the number of layers, their connectivity--all of that partially defines F. Finding a structure is what allows you to learn from the x/y pairs instead of just storing them in a look-up table. It makes sense, then, that finding a better structure may allow you to learn from fewer x/y pairs. This is Apple's purported approach. It also means that if you have a huge number of x/y pairs, you might benefit from (or get away with) using a simpler structure. This is Google's approach in the domains where they have a lot of data. It is true that an important lesson has been "when designers build structure into machine learning systems, they often build in the wrong things, and they end up making commitments that the data cannot undo". But there's still a lot of structure that we put into machine learning system, and we're still learning about what good structures are for different applications, and different regimes of data.
kthejoker2 8y ago
Traffic is a chaos system, you can't engineer the edge cases. At best you have heuristics.
- gugagore 8y ago
  I don't think "traffic is chaotic" is the issue with driverless cars. We'd be satisfied with a car that solved everything except for global (i.e. non-local) traffic flow issues. That said, a dynamical system may exhibit chaos for one setting of the parameters, and may not exhibit chaos for other settings of the parameters. It may be true that the equations governing traffic flow have the potential to exhibit chaos, and do exhibit chaos when 1. humans drive, or 2. the drivers have only local sensing. But that doesn't mean it would still be true for driverless cars. But again, that's not the issue at play. We wouldn't fault an driverless car for getting stuck in traffic. It's more like it has to work when it's snowing, when you can't see the lane markers, when a kid in a trashbag costume jumps into the street, etc. Those are the edge cases that we care about right now.

Not true. You can code your prior knowledge into the model, and make do with way less x/y pairs. Data is a substitute for doing engineering by hand. If you do the latter, you require less data.

Quarrelsome 8y ago

I don't see how it makes my statement untrue. The most effective ML techniques are more effective the more x/y pairs you have. Sure, you can get a leg up if you partially define F yourself but then you're no longer being as hands off as before. In practice the WHOLE of F() will likely be a blend of machine learning as well as heuristics as a product of various layers that are one or the other.

gugagore 8y ago
Not the poster you're replying to, but it's just that you said "that's not how it works", and then suggested that the only way to get F is through data, even if it means generating synthetic data. So that's wrong and untrue, as you recognize in this latest comment by saying you can partially define F. To make a concrete example, the convolution operator, the number of layers, their connectivity--all of that partially defines F. Finding a structure is what allows you to learn from the x/y pairs instead of just storing them in a look-up table. It makes sense, then, that finding a better structure may allow you to learn from fewer x/y pairs. This is Apple's purported approach. It also means that if you have a huge number of x/y pairs, you might benefit from (or get away with) using a simpler structure. This is Google's approach in the domains where they have a lot of data. It is true that an important lesson has been "when designers build structure into machine learning systems, they often build in the wrong things, and they end up making commitments that the data cannot undo". But there's still a lot of structure that we put into machine learning system, and we're still learning about what good structures are for different applications, and different regimes of data.

kthejoker2 8y ago

Traffic is a chaos system, you can't engineer the edge cases. At best you have heuristics.

gugagore 8y ago
I don't think "traffic is chaotic" is the issue with driverless cars. We'd be satisfied with a car that solved everything except for global (i.e. non-local) traffic flow issues. That said, a dynamical system may exhibit chaos for one setting of the parameters, and may not exhibit chaos for other settings of the parameters. It may be true that the equations governing traffic flow have the potential to exhibit chaos, and do exhibit chaos when 1. humans drive, or 2. the drivers have only local sensing. But that doesn't mean it would still be true for driverless cars. But again, that's not the issue at play. We wouldn't fault an driverless car for getting stuck in traffic. It's more like it has to work when it's snowing, when you can't see the lane markers, when a kid in a trashbag costume jumps into the street, etc. Those are the edge cases that we care about right now.