6 ms·

The scary thing about non-ad hoc techniques is that deep net is a "black box" -- you really don't know how pathologies occurred nor do you know how to fix them.

The scary thing about non-ad hoc techniques is that deep net is a "black box" -- you really don't know how pathologies occurred nor do you know how to fix them. Not only that, there are _inherent_ pathologies associated with using deep nets in the first place.

makecheck 10y ago
It doesn’t have to be all or nothing. For instance, the network might only be used to decide among a series of actions, and those actions can still have limits (such as “car cannot travel faster than X” or “Y cannot change more than 3 times per minute”, or whatever). There is still an abundance of attention put into safety, as usual for the auto industry. It isn’t just a brain hooked up to an engine that is allowed to run rampant.
- apk17 10y ago
  Depending on the context, 'turn the wheel left two degrees' can be a non-rampant, a very rampant, or a rampage-compensating action.
  - marcosdumay 10y ago
    "Keep the car in a straight line" or "turn right, 10°" are the kinds of operation that will always be more performant, take less engineering, and more reliable to create as an old-fashioned simple math function than by using machine learning. One would expect the output of the neural nets to be encoded in those.
    - Hydraulix989 10y ago
      It's the exceptional cases that get you: 1. "Keep the car in a straight line" => The road is closed or there is a detour, or the lane markers aren't visible (or incorrectly marked like on the Palo Alto stretch of the 101 where there are two sets of lane markers; one new (the "actual" markers) and another old but barely faded and still very visible. 2. "Turn right, 10 degree" => With what steering radius? What if the road is bumpy or sloped and the driver has to compensate with the wheel? etc. These corner cases are everyday occurrences and lend a necessity for something far more sophisticated than a mere ad hoc model to represent and control human-level driving behaviors with a finite set of rudimentary actions. Actuator inputs like steering and pedals are almost always represented using continuous values rather than discrete sets for these very reasons.
lucidrains 10y ago
Fun thought experiment. An alien race landed on earth and demands to play a game of Go. We only get to play one game with them. If they win, our planet is destroyed. Who would you trust to play for the human race if this scenario happened tomorrow? Lee Sedol or AlphaGo? Remember that we do not completely understand how AlphaGo reasons, it is still a black box to us.
- tempestn 10y ago
  Lee Sedol is also a black box, no? Also, one shot for the survival of humanity is very different from billions of iterations of driving situations every day for the foreseeable future. A complete understanding of the system is much more valuable when you have the opportunity to iterate. I'm not necessarily arguing for one approach; just saying that the analogy doesn't really apply to the case at hand.
  - lucidrains 10y ago
    Yea the analogy doesn't hold. I was simply probing at the author's criticism (or fear) of deep learning as a flawed approach given the resulting model is pretty opaque. To be honest, I would have felt the same way two years ago. Seeing all these recent advances in the field has slowly changed my mentality.
  - cscurmudgeon 10y ago
    > Lee Sedol is also a black box, no? Last time I checked we can talk to Lee Seedol and ask him to explain things. We can ask him questions. We can have an intelligent conversation with him.
    - nradov 10y ago
      Human explanations for their decisions are often rationalizations after the fact. The explanations don't necessarily accurately represent how the decisions were actually made. Most decisions are made subconsciously based on intuition and emotion. So that intelligent conversion might not have any real significance.
matthewmarkus 10y ago
DB: So, control theory is model-based and concerned with worst case. Machine learning is data based and concerned with average case. Is there a middle ground? BR: I think there is! And I think there's an exciting opportunity here to understand how to combine robust control and reinforcement learning. Being able to build systems from data alone simplifies the engineering process, and has had several recent promising results. Guaranteeing that these systems won't behave catastrophically will enable us to actually deploy machine learning systems in a variety of applications with major impacts on our lives. It might enable safe autonomous vehicles that can navigate complex terrains. Or could assist us in diagnostics and treatments in health care. There are a lot of exciting possibilities, and that's why I'm excited about how to find a bridge between these two viewpoints. https://www.oreilly.com/ideas/machine-learning-in-the-wild https://www.oreilly.com/ideas/machine-learning-in-the-wild
deepnotderp 10y ago
Definitely, quite scary, keeps me up at night. At least I know EXACTLY what my taxi driver is thinking.... /s
foglerek 10y ago
As do humans (see the various cognitive biases). The question is: does the AI's black box result in a safer outcome?
- Hydraulix989 10y ago
  And the answer is an obvious no, not even close, with the current state-of-the-art in deep nets. I could forgive an opaque model if it actually performed well.

makecheck 10y ago

It doesn’t have to be all or nothing. For instance, the network might only be used to decide among a series of actions, and those actions can still have limits (such as “car cannot travel faster than X” or “Y cannot change more than 3 times per minute”, or whatever). There is still an abundance of attention put into safety, as usual for the auto industry. It isn’t just a brain hooked up to an engine that is allowed to run rampant.

apk17 10y ago
Depending on the context, 'turn the wheel left two degrees' can be a non-rampant, a very rampant, or a rampage-compensating action.
- marcosdumay 10y ago
  "Keep the car in a straight line" or "turn right, 10°" are the kinds of operation that will always be more performant, take less engineering, and more reliable to create as an old-fashioned simple math function than by using machine learning. One would expect the output of the neural nets to be encoded in those.
  - Hydraulix989 10y ago
    It's the exceptional cases that get you: 1. "Keep the car in a straight line" => The road is closed or there is a detour, or the lane markers aren't visible (or incorrectly marked like on the Palo Alto stretch of the 101 where there are two sets of lane markers; one new (the "actual" markers) and another old but barely faded and still very visible. 2. "Turn right, 10 degree" => With what steering radius? What if the road is bumpy or sloped and the driver has to compensate with the wheel? etc. These corner cases are everyday occurrences and lend a necessity for something far more sophisticated than a mere ad hoc model to represent and control human-level driving behaviors with a finite set of rudimentary actions. Actuator inputs like steering and pedals are almost always represented using continuous values rather than discrete sets for these very reasons.

lucidrains 10y ago

Fun thought experiment. An alien race landed on earth and demands to play a game of Go. We only get to play one game with them. If they win, our planet is destroyed. Who would you trust to play for the human race if this scenario happened tomorrow? Lee Sedol or AlphaGo? Remember that we do not completely understand how AlphaGo reasons, it is still a black box to us.

tempestn 10y ago
Lee Sedol is also a black box, no? Also, one shot for the survival of humanity is very different from billions of iterations of driving situations every day for the foreseeable future. A complete understanding of the system is much more valuable when you have the opportunity to iterate. I'm not necessarily arguing for one approach; just saying that the analogy doesn't really apply to the case at hand.
- lucidrains 10y ago
  Yea the analogy doesn't hold. I was simply probing at the author's criticism (or fear) of deep learning as a flawed approach given the resulting model is pretty opaque. To be honest, I would have felt the same way two years ago. Seeing all these recent advances in the field has slowly changed my mentality.
- cscurmudgeon 10y ago
  > Lee Sedol is also a black box, no? Last time I checked we can talk to Lee Seedol and ask him to explain things. We can ask him questions. We can have an intelligent conversation with him.
  - nradov 10y ago
    Human explanations for their decisions are often rationalizations after the fact. The explanations don't necessarily accurately represent how the decisions were actually made. Most decisions are made subconsciously based on intuition and emotion. So that intelligent conversion might not have any real significance.

matthewmarkus 10y ago

DB: So, control theory is model-based and concerned with worst case. Machine learning is data based and concerned with average case. Is there a middle ground? BR: I think there is! And I think there's an exciting opportunity here to understand how to combine robust control and reinforcement learning. Being able to build systems from data alone simplifies the engineering process, and has had several recent promising results. Guaranteeing that these systems won't behave catastrophically will enable us to actually deploy machine learning systems in a variety of applications with major impacts on our lives. It might enable safe autonomous vehicles that can navigate complex terrains. Or could assist us in diagnostics and treatments in health care. There are a lot of exciting possibilities, and that's why I'm excited about how to find a bridge between these two viewpoints. https://www.oreilly.com/ideas/machine-learning-in-the-wild https://www.oreilly.com/ideas/machine-learning-in-the-wild

deepnotderp 10y ago

Definitely, quite scary, keeps me up at night. At least I know EXACTLY what my taxi driver is thinking.... /s

foglerek 10y ago

As do humans (see the various cognitive biases). The question is: does the AI's black box result in a safer outcome?

Hydraulix989 10y ago
And the answer is an obvious no, not even close, with the current state-of-the-art in deep nets. I could forgive an opaque model if it actually performed well.