7 ms·

I think a valid concern is that ML methods are being applied in critical, real-life scenarios without some practitioners being aware of flaws (bias, adversarial

I think a valid concern is that ML methods are being applied in critical, real-life scenarios without some practitioners being aware of flaws (bias, adversarial attacks, privacy issues) and without any theoretic safetynet that helps them reason about how these systems will behave. James Mickens discussed this recently in a keynote: https://www.usenix.org/conference/usenixsecurity18/presentation/mickens https://www.usenix.org/conference/usenixsecurity18/presentat... Maybe the only way to make steady progress here is to blaze ahead and rely on empirical evidence, whilst the theory is inevitably fleshed out. That's often the counter argument - that we do not know how the brain works, but rely on it nonetheless.

andrewflnr 8y ago
We don't rely on the brain though, at least not on any single one. Any system that relies on human brains alone without cross-checking or, ideally, much simpler automatic systems, will eventually malfunction terribly. A large organization never wants to rely on a single person's judgment for anything, a programmer wants automated systems checking their work, etc.
- dontreact 8y ago
  We rely on a single human to drive a car. In medicine, machine learning systems will work alongside other brains. What are the instances you’re imagining where a group of brains running an important system are replaced by a single machine learning algorithm running in isolation?
  - andrewflnr 8y ago
    Yeah, and around 800 people a day die in auto accidents in the US. Bringing up cars bolsters my point, which is that "we rely on the brain without understanding it and that works out fine" is not a good argument. So let's be careful. I'm mystified as to why that seems to be controversial.
    - dontreact 8y ago
      Are you saying that we should require 2 drivers per car? It seems like at the moment the risk/reward ratio of allowing people to drive a car after getting a license seems to work out well enough. 1 death per 100 million miles driven, and falling.
      - andrewflnr 8y ago
        No. I'm saying > that "we rely on the brain without understanding it and that works out fine" is not a good argument. It wasn't my idea to bring cars into it particularly.

andrewflnr 8y ago

We don't rely on the brain though, at least not on any single one. Any system that relies on human brains alone without cross-checking or, ideally, much simpler automatic systems, will eventually malfunction terribly. A large organization never wants to rely on a single person's judgment for anything, a programmer wants automated systems checking their work, etc.

dontreact 8y ago
We rely on a single human to drive a car. In medicine, machine learning systems will work alongside other brains. What are the instances you’re imagining where a group of brains running an important system are replaced by a single machine learning algorithm running in isolation?
- andrewflnr 8y ago
  Yeah, and around 800 people a day die in auto accidents in the US. Bringing up cars bolsters my point, which is that "we rely on the brain without understanding it and that works out fine" is not a good argument. So let's be careful. I'm mystified as to why that seems to be controversial.
  - dontreact 8y ago
    Are you saying that we should require 2 drivers per car? It seems like at the moment the risk/reward ratio of allowing people to drive a car after getting a license seems to work out well enough. 1 death per 100 million miles driven, and falling.
    - andrewflnr 8y ago
      No. I'm saying > that "we rely on the brain without understanding it and that works out fine" is not a good argument. It wasn't my idea to bring cars into it particularly.