4 ms·

It is not the technology that is dangerous but the scale and manner of its deployment. Patented, opaque, closed source, championed by mega-corporations closely

It is not the technology that is dangerous but the scale and manner of its deployment. Patented, opaque, closed source, championed by mega-corporations closely allied with powerful governments, entirely beyond the analysis or oversight of the vast majority of citizens, completely connected realtime to various other systems, automated, eternally slurping up data for indefinite storage... these are the real concerning factors. After all, a good old security guard also does 'face recognition' of everyone passing by, but with a human there are obvious limits and a person to confront, charge or circumvent. With ubiquitous, tiny robots all over the place, society will get sharply stratified into the 'watched' and the 'watchers' and the latter will get access to nearly limitless data and power. That's not good.

bilbo0s 8y ago
>society will get sharply stratified into the 'watched' and the 'watchers' and the latter will get access to nearly limitless data and power... Don't kid yourself, even the 'watchers' will be 'watched'. The people bringing all these systems together will not be your local yocal Barney Fifes. They will be professional and highly intelligent technical teams whose members have likely studied little other than security and intelligence gathering for their entire professional lives. These are not the type of teams that carelessly leave obvious security holes like that laying around. Taken as a whole, the technologies coming out today will enable surveillance regimes the likes of which even the UK and China never previously would have dared to dream up.
- makotech222 8y ago
  Yeah, because the CEOs, Politicians, etc... are definitely the people walking around in public spaces or delivering packages or buying stuff from stores.
- bsenftner 8y ago
  Yeah, except it's not like that. It is much closer to the random collection of anybodies found at any company, just with a tech bent. The idea of these uber specialist is myth. I work in security, at high levels. It's just normal people.
- jolmg 8y ago
  I think you misunderstood who the watchers are supposed to be. Santosh83 is probably referring to the high risk of the establishment of a police state.
- maxxxxx 8y ago
  "Taken as a whole, the technologies coming out today will enable surveillance regimes the likes of which even the UK and China never previously would have dared to dream up. " thay's really important. Hitler, Stalin, the Stasi and many others would have been very pleased to have the systems we are building right now. Once a full country surveillance system gets in the hand of a ruthless dictator it will be very hard to get rid of that dictator.
- gaze 8y ago
  Why do you think this? Who are these elites that never leave security holes in things? The snowden leaks show they don't work for the NSA. They don't work in consumer electronics... the deeply flawed copy protection schemes in most video game consoles show that. They don't work for the FSB. What gives you the impression that these people even exist? People seem to not understand the asymmetry in difficulty between offence and defense. For something to be broken, there just needs to be one fuckup, or a few tiny fuckups that can be linked together. Smart people are not infallible. Everything is exploited eventually. The only foolproof thing to do is not to build the infrastructure in the first place. But yes, I agree with your last point. There's no privacy anymore.
- btbuildem 8y ago
  And who watches these specialized, intelligent watchers? Another, "higher caste" of more specialized, more intelligent watchers? The more likely scenario is that this line of work would likely attract creepy antisocial perverts who like to watch, and love to exert authority -- essentially the people we would like to keep as far as possible from this kind of technology.
- _37go 8y ago
  ring's entire technical team is based in the ukraine because the labor is cheaper and the data moderators that tag the videos for facial recognition and video processing make $3.00 - Ring isn't a sophisticated technology or even a secure product. For example, there is no two factor authentication and it's not encrypted end to end. Surveillance capitalism is a rush to the bottom to make money, not protect society, leaving gaping holes and cutting corners on quality (and fair pay) for profits.
JohnFen 8y ago
"After all, a good old security guard also does 'face recognition' of everyone passing by" Yes, but the essential difference is that the good old security guard isn't photographing everyone passing by and putting those pictures into a database.
- setr 8y ago
  Which is only a problem because its... entirely beyond the analysis or oversight of the vast majority of citizens, completely connected realtime to various other systems, automated, eternally slurping up data for indefinite storage.. It wouldn’t be as concerning if it was only being stored in a local sqlite db.
- deleted 8y ago
  [deleted]
- snowwrestler 8y ago
  I mean, he or she is collecting images with their eyes and storing them in their brain. We're fine with that because it's only local storage--the police can't just plug a network cord into your security guard and suck out all the images and timestamps every hour. I think facial recognition door systems are a fine idea if they can be implemented like a doorman: local storage only, carefully trained on local data. With current technology that would be impossible or extremely expensive, though.
  - JohnFen 8y ago
    "I think facial recognition door systems are a fine idea if they can be implemented like a doorman" I agree, with the addition that all actual processing is done locally, that would be fine. But that's not what this patent is talking about.
toss1 8y ago
This. The difference between individual human watchers doing surveillance, face recognition, etc. and at-scale watchers doing the same thing is key. The difference is no less than the old corner stores every few blocks vs Amazon's automated warehouses & shipping. An ordinary drive with a bit of speeding here and there everyday goes from perhaps a ticket or two every decade, to 6 tickets every trip. A meaningless wrong turn, I'm lost, let's look around and try to figure out where I am, turns from a non-event to a Suspicious Activity Report... Just because the laws are word-for-word unchanged, the scaling of enforcement technology transforms the identical laws into a completely different society and experience (just as other scaling technology transforms our experience in a positive way). This phenomena, as unintended consequences ("just trying to make the neighborhood safer...", but scaled 10^7), is perhaps as dangerous as Bostrom's AI warnings.
- lambdadmitry 8y ago
  > An ordinary drive with a bit of speeding here and there everyday goes from perhaps a ticket or two every decade, to 6 tickets every trip. Can it be that the best way of avoiding that is avoiding speeding? It's interesting that you are lumping together objectively dangerous, even if prevalent in the US, public behaviour (deliberate speeding) with honest mistakes (making a wrong turn).
  - toss1 8y ago
    Yes, an obvious almost tautological suggestion, which often does not apply, and can actually increase the danger of situations even while technically reducing your chances of a ticket. I regularly drive on highways with relatively dense traffic all going 10-20mph above the speed limit; in these conditions, it is objectively MORE dangerous to attempt to go the speed limit. I've seen the whole pack drive right by a state cop at 80+, and he'll do nothing because just to turn on his lightshow will likely cause a major wreck. Outside of residential/school/congested areas with non-vehicular traffic & obstacles, it is NOT any numerical speed that causes issues, but the differential in speeds. Going too much faster -or too much slower- than surrounding traffic is what will cause accident situations (obviously, up to the limits of conditions, car preparation, driver skill & attention, etc.) The Autobahns are regulated as an excellent example of this. In rural areas very high speeds are permitted, and lane discipline is rigorously enforced. You'll get busted way harder for driving slow in the left lane than for driving 300kph. And, in the more congested zones, the speed limits come down. And the safety records exceed those of the 65MPH US highways. Speed does not kill, but rather driver ignorance does. A safe fast driver is better than an unsafe slow driver any day. And most US drivers are never taught even the basics of how to actually drive. I know otherwise very intelligent people who balked at taking their teens to driver safety & skills classes taught by SCCA and others (which objectively reduce accident rates) because they thought that teaching these skills (car control, balance, braking & turning at the limits, etc.) would give them wrong ideas...
chaostheory 8y ago
> a good old security guard also does 'face recognition' of everyone passing by, but with a human Not many people can afford their own good human security guard. Not everyone lives in areas where the police department is properly funded either.
- Spivak 8y ago
  That's not really the point the parent is making. The problem isn't using an unattended system performing the function of a security guard -- because nobody's arguing that security guards are a tantamount to a surveillance state. It's that a single company operating all of these autonomous security guards gives them, and anyone with power to compel them, a level of viability which simply didn't exist before.

bilbo0s 8y ago

>society will get sharply stratified into the 'watched' and the 'watchers' and the latter will get access to nearly limitless data and power... Don't kid yourself, even the 'watchers' will be 'watched'. The people bringing all these systems together will not be your local yocal Barney Fifes. They will be professional and highly intelligent technical teams whose members have likely studied little other than security and intelligence gathering for their entire professional lives. These are not the type of teams that carelessly leave obvious security holes like that laying around. Taken as a whole, the technologies coming out today will enable surveillance regimes the likes of which even the UK and China never previously would have dared to dream up.

makotech222 8y ago
Yeah, because the CEOs, Politicians, etc... are definitely the people walking around in public spaces or delivering packages or buying stuff from stores.
bsenftner 8y ago
Yeah, except it's not like that. It is much closer to the random collection of anybodies found at any company, just with a tech bent. The idea of these uber specialist is myth. I work in security, at high levels. It's just normal people.
jolmg 8y ago
I think you misunderstood who the watchers are supposed to be. Santosh83 is probably referring to the high risk of the establishment of a police state.
maxxxxx 8y ago
"Taken as a whole, the technologies coming out today will enable surveillance regimes the likes of which even the UK and China never previously would have dared to dream up. " thay's really important. Hitler, Stalin, the Stasi and many others would have been very pleased to have the systems we are building right now. Once a full country surveillance system gets in the hand of a ruthless dictator it will be very hard to get rid of that dictator.
gaze 8y ago
Why do you think this? Who are these elites that never leave security holes in things? The snowden leaks show they don't work for the NSA. They don't work in consumer electronics... the deeply flawed copy protection schemes in most video game consoles show that. They don't work for the FSB. What gives you the impression that these people even exist? People seem to not understand the asymmetry in difficulty between offence and defense. For something to be broken, there just needs to be one fuckup, or a few tiny fuckups that can be linked together. Smart people are not infallible. Everything is exploited eventually. The only foolproof thing to do is not to build the infrastructure in the first place. But yes, I agree with your last point. There's no privacy anymore.
btbuildem 8y ago
And who watches these specialized, intelligent watchers? Another, "higher caste" of more specialized, more intelligent watchers? The more likely scenario is that this line of work would likely attract creepy antisocial perverts who like to watch, and love to exert authority -- essentially the people we would like to keep as far as possible from this kind of technology.
_37go 8y ago
ring's entire technical team is based in the ukraine because the labor is cheaper and the data moderators that tag the videos for facial recognition and video processing make $3.00 - Ring isn't a sophisticated technology or even a secure product. For example, there is no two factor authentication and it's not encrypted end to end. Surveillance capitalism is a rush to the bottom to make money, not protect society, leaving gaping holes and cutting corners on quality (and fair pay) for profits.

JohnFen 8y ago

"After all, a good old security guard also does 'face recognition' of everyone passing by" Yes, but the essential difference is that the good old security guard isn't photographing everyone passing by and putting those pictures into a database.

setr 8y ago
Which is only a problem because its... entirely beyond the analysis or oversight of the vast majority of citizens, completely connected realtime to various other systems, automated, eternally slurping up data for indefinite storage.. It wouldn’t be as concerning if it was only being stored in a local sqlite db.
deleted 8y ago
[deleted]
snowwrestler 8y ago
I mean, he or she is collecting images with their eyes and storing them in their brain. We're fine with that because it's only local storage--the police can't just plug a network cord into your security guard and suck out all the images and timestamps every hour. I think facial recognition door systems are a fine idea if they can be implemented like a doorman: local storage only, carefully trained on local data. With current technology that would be impossible or extremely expensive, though.
- JohnFen 8y ago
  "I think facial recognition door systems are a fine idea if they can be implemented like a doorman" I agree, with the addition that all actual processing is done locally, that would be fine. But that's not what this patent is talking about.

toss1 8y ago

This. The difference between individual human watchers doing surveillance, face recognition, etc. and at-scale watchers doing the same thing is key. The difference is no less than the old corner stores every few blocks vs Amazon's automated warehouses & shipping. An ordinary drive with a bit of speeding here and there everyday goes from perhaps a ticket or two every decade, to 6 tickets every trip. A meaningless wrong turn, I'm lost, let's look around and try to figure out where I am, turns from a non-event to a Suspicious Activity Report... Just because the laws are word-for-word unchanged, the scaling of enforcement technology transforms the identical laws into a completely different society and experience (just as other scaling technology transforms our experience in a positive way). This phenomena, as unintended consequences ("just trying to make the neighborhood safer...", but scaled 10^7), is perhaps as dangerous as Bostrom's AI warnings.

lambdadmitry 8y ago
> An ordinary drive with a bit of speeding here and there everyday goes from perhaps a ticket or two every decade, to 6 tickets every trip. Can it be that the best way of avoiding that is avoiding speeding? It's interesting that you are lumping together objectively dangerous, even if prevalent in the US, public behaviour (deliberate speeding) with honest mistakes (making a wrong turn).
- toss1 8y ago
  Yes, an obvious almost tautological suggestion, which often does not apply, and can actually increase the danger of situations even while technically reducing your chances of a ticket. I regularly drive on highways with relatively dense traffic all going 10-20mph above the speed limit; in these conditions, it is objectively MORE dangerous to attempt to go the speed limit. I've seen the whole pack drive right by a state cop at 80+, and he'll do nothing because just to turn on his lightshow will likely cause a major wreck. Outside of residential/school/congested areas with non-vehicular traffic & obstacles, it is NOT any numerical speed that causes issues, but the differential in speeds. Going too much faster -or too much slower- than surrounding traffic is what will cause accident situations (obviously, up to the limits of conditions, car preparation, driver skill & attention, etc.) The Autobahns are regulated as an excellent example of this. In rural areas very high speeds are permitted, and lane discipline is rigorously enforced. You'll get busted way harder for driving slow in the left lane than for driving 300kph. And, in the more congested zones, the speed limits come down. And the safety records exceed those of the 65MPH US highways. Speed does not kill, but rather driver ignorance does. A safe fast driver is better than an unsafe slow driver any day. And most US drivers are never taught even the basics of how to actually drive. I know otherwise very intelligent people who balked at taking their teens to driver safety & skills classes taught by SCCA and others (which objectively reduce accident rates) because they thought that teaching these skills (car control, balance, braking & turning at the limits, etc.) would give them wrong ideas...

chaostheory 8y ago

> a good old security guard also does 'face recognition' of everyone passing by, but with a human Not many people can afford their own good human security guard. Not everyone lives in areas where the police department is properly funded either.

Spivak 8y ago
That's not really the point the parent is making. The problem isn't using an unattended system performing the function of a security guard -- because nobody's arguing that security guards are a tantamount to a surveillance state. It's that a single company operating all of these autonomous security guards gives them, and anyone with power to compel them, a level of viability which simply didn't exist before.