4 ms·

I love this quote at the end of the the google cached result - "we should not throw away the oven because the temperature could be set wrong and burn the pizza.

I love this quote at the end of the the google cached result - "we should not throw away the oven because the temperature could be set wrong and burn the pizza."

confounded 8y ago
I don’t remember us being asked if we wanted to live in the oven, but here we are anyway.
bvc35 8y ago
When the oven will burn you if used incorrectly, and Amazon gives police departments (who don't know how ovens work) the controls over the oven regardless of their provable technical ability and with no oversight, we should definitely throw away the oven.
- joshgel 8y ago
  Maybe yes, maybe no. But realistically, with the hype out there about AI solving all problems, do we think its not going to get used? Seems like more productive conversation could be had over how to use, than blanket dismissals. The proverbial 400-lbs man in his basement can code this up relatively easily with existing tools, to near state-of-the-art results. So the guy that works for the FBI will too. Let's come up with appropriate, targeted safety mechanisms so the oven doesn't light the whole building on fire.
- white-flame 8y ago
  Or more precisely, take the flaky oven away from people who misuse it in high stakes situations.
- visarga 8y ago
  Not only that you can't uninvent a technology that can run on cheap hardware with open source software and which can be reimplemented from scratch in a week from arxiv papers, but this is a clear case of improper use, where the solution is to set better norms.
stochastic_monk 8y ago
Good luck getting law enforcement to properly interpret machine learning results. Look at how they still use disproven methods like bite marks to unjustly imprison innocent people.
- noway421 8y ago
  The product in question is Rekognition, which matches photos. It would be more verifiable by naked human eye than bite marks? As long as the results are not taken as judgements, but only auxiliary tool, it would not be a concern. The article indeed argues that any positive match should be vetted by a human.
  - mcny 8y ago
    > The article indeed argues that any positive match should be vetted by a human. I have said before and I will say again: after the disaster that was the Amazon Fire Phone I have zero trust in any Amazon.com employee to pick up that any time phone to sound the alarm or if they do, I don't expect Amazon to "do the right thing". The reason I mention this is because I don't think this PhD fellow speaks for the the sales "engineers" at Amazon.com. The sales people will say what they have to say to make a sale. Lets face it: we programmers just work at a company, the business actually runs the show. I have zero confidence that the customer will understand that "any positive match should be vetted by a human". What does vetting mean anyway? No police has ever stopped me for matching the profile of a suspect, ever. However, I know people who say it happens (or at least used to happen) to them at least twice a year. Is a blanket ban on law enforcement from using facial recognition possible? Or at least restrict it to federal agencies (a ban on all law enforcement that is not a federal agency)?
    - noway421 8y ago
      That's actually a very good point, sales people will try to exaggerate the capabilities of the technology to the dangerous levels... Well said.
sgt101 8y ago
How about "we should not throw away the medical scanner because it can be set wrong and it burned this child"? Or, don't use the oven on setting 3 because if you do your house will burn down?

I love this quote at the end of the the google cached result - "we should not throw away the oven because the temperature could be set wrong and burn the pizza."

confounded 8y ago

I don’t remember us being asked if we wanted to live in the oven, but here we are anyway.

bvc35 8y ago

When the oven will burn you if used incorrectly, and Amazon gives police departments (who don't know how ovens work) the controls over the oven regardless of their provable technical ability and with no oversight, we should definitely throw away the oven.

joshgel 8y ago
Maybe yes, maybe no. But realistically, with the hype out there about AI solving all problems, do we think its not going to get used? Seems like more productive conversation could be had over how to use, than blanket dismissals. The proverbial 400-lbs man in his basement can code this up relatively easily with existing tools, to near state-of-the-art results. So the guy that works for the FBI will too. Let's come up with appropriate, targeted safety mechanisms so the oven doesn't light the whole building on fire.
white-flame 8y ago
Or more precisely, take the flaky oven away from people who misuse it in high stakes situations.
visarga 8y ago
Not only that you can't uninvent a technology that can run on cheap hardware with open source software and which can be reimplemented from scratch in a week from arxiv papers, but this is a clear case of improper use, where the solution is to set better norms.

stochastic_monk 8y ago

Good luck getting law enforcement to properly interpret machine learning results. Look at how they still use disproven methods like bite marks to unjustly imprison innocent people.

noway421 8y ago
The product in question is Rekognition, which matches photos. It would be more verifiable by naked human eye than bite marks? As long as the results are not taken as judgements, but only auxiliary tool, it would not be a concern. The article indeed argues that any positive match should be vetted by a human.
- mcny 8y ago
  > The article indeed argues that any positive match should be vetted by a human. I have said before and I will say again: after the disaster that was the Amazon Fire Phone I have zero trust in any Amazon.com employee to pick up that any time phone to sound the alarm or if they do, I don't expect Amazon to "do the right thing". The reason I mention this is because I don't think this PhD fellow speaks for the the sales "engineers" at Amazon.com. The sales people will say what they have to say to make a sale. Lets face it: we programmers just work at a company, the business actually runs the show. I have zero confidence that the customer will understand that "any positive match should be vetted by a human". What does vetting mean anyway? No police has ever stopped me for matching the profile of a suspect, ever. However, I know people who say it happens (or at least used to happen) to them at least twice a year. Is a blanket ban on law enforcement from using facial recognition possible? Or at least restrict it to federal agencies (a ban on all law enforcement that is not a federal agency)?
  - noway421 8y ago
    That's actually a very good point, sales people will try to exaggerate the capabilities of the technology to the dangerous levels... Well said.

sgt101 8y ago

How about "we should not throw away the medical scanner because it can be set wrong and it burned this child"? Or, don't use the oven on setting 3 because if you do your house will burn down?