3 ms·

> Rekognition performed so poorly compared to our prototype (which was not state of the art or anything, just good enough for our company’s bespoke conpliance p

> Rekognition performed so poorly compared to our prototype (which was not state of the art or anything, just good enough for our company’s bespoke conpliance problem). It was just so bad. Overall intersection-over-union scores on our training & acceptance data was really poor, and perhaps worse, the latency of the requests to Rekognition was abysmal and completely unusable for us, especially for images that had a large number of identifiable faces (Rekognition service calls isually timed out at 5 seconds and returned whatever amount of faces it could process). Criticizing the technology's performance is tangential to the emergence of the technology on public streets, and its adoption by local governments for surveillance. It does not matter what performance it has now; I argue and predict it will improve, and then your statement becomes moot. The issue is not how well it performs. The issue is allowing it to happen, and without trust from the people. My fear is: government wants surveillance, and businesses pursue revenue, so there is no incentive among those two to question their actions.

mlthoughts2018 8y ago
I agree with you generally. But I differ in the sense that my default belief is that these technologies are over hyped and cannot provide the scale of performance or detail of prediction that could make them effective as oppressive surveillance tools, at least not for a long time. Mostly they are “surveillance theater” akin to useless (but costly) TSA surveillance. The way it interacts with society is by being vague and scary, not by actually enabling truly scary scales of surveillance. Another way to look at it is that the government buys these contracts and services and rarely holds the provider accountable, perhaps especially in fields like machine learning where people can more easily make political arguments out of the debate over whether it’s working or not. Since agencies are happy to drop big money even when the technology does not work then providers like Rekognition actually do not have incentives to invest in making the scale of accurate surveillance larger. Why would they when they’ll make more money yanking an engineer off of a material project like that and slotting them to work on some braindead TED talk sort of demoware that gets them the next big contract? I worry about the spread of these surveillance contracts more as a taxpayer seeing government money wasted that could help people through much simpler means than I do as a concerned citizen who sees the danger in Orwellian surveillance.
- cryoshon 8y ago
  >my default belief is that these technologies are over hyped and cannot provide the scale of performance or detail of prediction that could make them effective as oppressive surveillance tools, at least not for a long time. you've never heard of the chinese police state's facial surveillance which they're using to throw countless uighurs in jail for stuff like jaywalking, i suppose? >as a concerned citizen who sees the danger in Orwellian surveillance. you don't see the danger because your paycheck relies on you not seeing it. the rest of us are not so willfully blind.
  - mlthoughts2018 8y ago
    But I do see danger. Both in the misapplication of this unproven tech, and in the opportunity cost of wasting public funds on it.
    - cryoshon 8y ago
      then why work on it when it's bound for abuse either way?
      - mlthoughts2018 8y ago
        It seems misguided to dismiss the technology itself, rather than scrutinizing specific uses. It reminds me of David Deutsch's book The Beginning of Infinity, which has an extended anecdote about a scientist who deeply objected to the development of color TVs because "people didn't need them" and he could only think of examples where bad actors would use them. Fast forward several decades, and color TVs are used for all sorts of important outcomes, like life-saving medical imaging technology, or sharing video of an important life moment with someone who is too sick to travel. You could imagine similar things for face recognition. An Alzheimer's patient who has a large digital photo library of family members and events, and would like some information retrieval tool to help search for a specific memory, family member, etc. Or someone who takes stock photographs and wants to photoshop bystanders out of the photos so that nobody's image rights are infringed when submitting the stock photo for publication-- might want face detection to aid in the person-removal task. I agree completely with scrutinizing motives and refusing to work on tech when the goal is unethical. But I don't agree this allows us to point at a generic, amoral technology in and of itself and say it's conceptually unethical for existing. Maybe there would be some extreme or contrived cases when I would agree to that, but even this face recognition stuff is very far from it.
TuringNYC 8y ago
Unfortunately the "cost" of poor performance is sometimes pushed to the public, or worse, subsets of the public. Consider the million-name-long restricted fly list of the last decade -- they were nice juicy profitable projects for corporations but the cost was borne by a subset of mostly innocent people sometimes lost all ability to fly. If the costs are compartmentalized enough, sometimes the general public doesnt care (see: NYC's Stop and Frisk policy.)
- fixermark 8y ago
  Agreed. I don't see much benefit to pushing back against the technologies themselves; they're here, they're easy to apply, and we're well past the point where trying to ban them will just push them underground. Misapplication of the technologies and erosion of fundamentals of evidence-based action are a real problem. Contrast the California police using genealogy datasets to narrow the realm of potential "John Doe" serial killers. As court evidence? Shouldn't be admissible. As a tool to focus the search to a subset of the population? Extremely good idea. Similarly, the face-recognition camera can tell the authorities where to turn their eyes but should never be the be-all and end-all of adjudication; they're too easy to fool (as Maryland discovered when it started auto-ticketing cars that blew through red lights at camera-tagged intersections, only to find a wave of high-schoolers thinking it was hilarious to tape a photo of an unliked teacher's license plate across their own and then blow through an empty intersection at 80 MPH at 2AM).
fixermark 8y ago
It does, however, suggest that whether Amazon's doing it today is irrelevant, if any org with the resources of mlthoughts2018's org could cobble together a system that performs sufficiently to their needs with open-source solutions and some locally-grown or publicly-sourced training data. This barn is devoid of horses and is getting chilly from all the airflow.

mlthoughts2018 8y ago

I agree with you generally. But I differ in the sense that my default belief is that these technologies are over hyped and cannot provide the scale of performance or detail of prediction that could make them effective as oppressive surveillance tools, at least not for a long time. Mostly they are “surveillance theater” akin to useless (but costly) TSA surveillance. The way it interacts with society is by being vague and scary, not by actually enabling truly scary scales of surveillance. Another way to look at it is that the government buys these contracts and services and rarely holds the provider accountable, perhaps especially in fields like machine learning where people can more easily make political arguments out of the debate over whether it’s working or not. Since agencies are happy to drop big money even when the technology does not work then providers like Rekognition actually do not have incentives to invest in making the scale of accurate surveillance larger. Why would they when they’ll make more money yanking an engineer off of a material project like that and slotting them to work on some braindead TED talk sort of demoware that gets them the next big contract? I worry about the spread of these surveillance contracts more as a taxpayer seeing government money wasted that could help people through much simpler means than I do as a concerned citizen who sees the danger in Orwellian surveillance.

cryoshon 8y ago
>my default belief is that these technologies are over hyped and cannot provide the scale of performance or detail of prediction that could make them effective as oppressive surveillance tools, at least not for a long time. you've never heard of the chinese police state's facial surveillance which they're using to throw countless uighurs in jail for stuff like jaywalking, i suppose? >as a concerned citizen who sees the danger in Orwellian surveillance. you don't see the danger because your paycheck relies on you not seeing it. the rest of us are not so willfully blind.
- mlthoughts2018 8y ago
  But I do see danger. Both in the misapplication of this unproven tech, and in the opportunity cost of wasting public funds on it.
  - cryoshon 8y ago
    then why work on it when it's bound for abuse either way?
    - mlthoughts2018 8y ago
      It seems misguided to dismiss the technology itself, rather than scrutinizing specific uses. It reminds me of David Deutsch's book The Beginning of Infinity, which has an extended anecdote about a scientist who deeply objected to the development of color TVs because "people didn't need them" and he could only think of examples where bad actors would use them. Fast forward several decades, and color TVs are used for all sorts of important outcomes, like life-saving medical imaging technology, or sharing video of an important life moment with someone who is too sick to travel. You could imagine similar things for face recognition. An Alzheimer's patient who has a large digital photo library of family members and events, and would like some information retrieval tool to help search for a specific memory, family member, etc. Or someone who takes stock photographs and wants to photoshop bystanders out of the photos so that nobody's image rights are infringed when submitting the stock photo for publication-- might want face detection to aid in the person-removal task. I agree completely with scrutinizing motives and refusing to work on tech when the goal is unethical. But I don't agree this allows us to point at a generic, amoral technology in and of itself and say it's conceptually unethical for existing. Maybe there would be some extreme or contrived cases when I would agree to that, but even this face recognition stuff is very far from it.

TuringNYC 8y ago

Unfortunately the "cost" of poor performance is sometimes pushed to the public, or worse, subsets of the public. Consider the million-name-long restricted fly list of the last decade -- they were nice juicy profitable projects for corporations but the cost was borne by a subset of mostly innocent people sometimes lost all ability to fly. If the costs are compartmentalized enough, sometimes the general public doesnt care (see: NYC's Stop and Frisk policy.)

fixermark 8y ago
Agreed. I don't see much benefit to pushing back against the technologies themselves; they're here, they're easy to apply, and we're well past the point where trying to ban them will just push them underground. Misapplication of the technologies and erosion of fundamentals of evidence-based action are a real problem. Contrast the California police using genealogy datasets to narrow the realm of potential "John Doe" serial killers. As court evidence? Shouldn't be admissible. As a tool to focus the search to a subset of the population? Extremely good idea. Similarly, the face-recognition camera can tell the authorities where to turn their eyes but should never be the be-all and end-all of adjudication; they're too easy to fool (as Maryland discovered when it started auto-ticketing cars that blew through red lights at camera-tagged intersections, only to find a wave of high-schoolers thinking it was hilarious to tape a photo of an unliked teacher's license plate across their own and then blow through an empty intersection at 80 MPH at 2AM).

fixermark 8y ago

It does, however, suggest that whether Amazon's doing it today is irrelevant, if any org with the resources of mlthoughts2018's org could cobble together a system that performs sufficiently to their needs with open-source solutions and some locally-grown or publicly-sourced training data. This barn is devoid of horses and is getting chilly from all the airflow.