21 ms·
AI Mistakes Bus-Side Ad for Famous CEO, Charges Her With Jaywalking
- userbinator 8y agoIt almost sounds like a headline from The Onion, but it's the reality in an increasingly dystopian society. That's really scary.
- reaperducer 8y agoAnd once it's "proven to work" in China, it will be sold to every other country, and start falsely accusing people from Brussels to Boston to Bangkok. Save more money on traffic cops. Spend more money on prisons.
- rorykoehler 8y agoThey're trying to role out this type of shit in Singapore now (Chinese firms providing the tech).
- starbeast 8y agoThere's been a memo going around from the higher-ups in Central Services that anyone named 'Harry Buttle' should currently take extreme care, due to minor bugs at Information Retrieval.
- maxxxxx 8y agoAdd to that a justice system also mostly automated and you are quickly caught in a Kafkaesque nightmare you can't get out if something goes wrong. Imagine the justice system run like google support. Even if something is clearly wrong you can't clear it up if you are not a famous person with a big Twitter following.
- icebraining 8y agoOn the other hand, a justice system which has most of the menial tasks automated might have more resources to do the important tasks well. Similarly, Google could have a mostly automated system detecting violations and such, and also a good human support system with power to override the decisions of the former; the two are not really incompatible.
- maxxxxx 8y agoI know an Android dev who had his Play account shut down for an unjustified reason. It took him a very long time to reactivate it. Especially the little guy doesn't get much attention from humans. That was a few years ago. Maybe it's better now.
- icebraining 8y agoI'm not saying Google's support is good. I'm saying the automation is not necessarily the culprit for its ills.
- tjoff 8y agoI'd say that the automation enables it to be an order of magnitude worse. And it will be worse for any entity that doesn't really care about it.
- ams6110 8y agoIt's probably worse for a few, and better for a lot. Maybe the Google automation is shutting down a hundred nefarious developers for every one legitimate developer. Without the automation all would be active; with the automation one guy gets screwed and it's hard for him to navigate his way back, but thousands of users are better off
- maxxxxx 8y agoMost justice systems are built on the assumption that it's better to let 100 guilty people go than convicting one innocent person. Once we accept that it's OK to convict innocent people as long as the averages are fine then we are heading into serious trouble.
- james_s_tayler 8y agoI just finished reading The Secret Barrister and it already seems to be the case, at least in the UK, that if something is wrong you can't really clear it up.
- olivermarks 8y agohttps://en.wikipedia.org/wiki/Stories_of_the_Law_and_How_It%27s_Broken https://en.wikipedia.org/wiki/Stories_of_the_Law_and_How_It%... Thank you, I will read this
- ryanmarsh 8y agoThis is something ordinary people who haven’t run afoul of the justice system cannot understand. It must be witnessed first hand to be comprehended. The justice system is a big boulder at the top of a mountain. Once an arrest or accusation sets in it motion, unless you are powerful and or wealthy, it will do what boulders do, it can do no different. It is not cognizant, it just is. It takes immense resources to stop or avoid this boulder. Resources far exceeding even the well-to-do often times. The system is there to win cases and incarcerate and my god does it do just that. Innocent or not, the system moves humans. Do not pray the jury hears your side, pray you never get so much as accused. Look up the conviction rates for district attorneys across the country. You tell me, does that look justice?
- ams6110 8y agoI'd expect conviction rates to be pretty high. It means they are only bringing charges when they have strong evidence. If conviction rates were low I'd be asking why that prosecutor was wasting his time on cases he can't win.
- ryanmarsh 8y agoIt means they are only bringing charges when they have strong evidence We would hope so. It could also mean far worse things are happening. We’ve all seen documentaries and news shows of people wrongly convicted. Why do we think the only incidences are the ones reported. I’ve watched detectives lie on the stand when a guilty verdict was already in hand. Why do that? How do they treat the innocent if that’s how they act when the facts of the case are enough to convict?
- eloff 8y agoThis sounds like a lot of people's experiences with Equifax and Transunion. They make a mistake and it can take you a decade and a lot of money and effort to clear your name.
- baddox 8y agoI don’t know how to compare the US justice system to Google support, but the former is already notoriously difficult, expensive, inefficient, and inaccessible to large groups of people.
- redthrowaway 8y agoIf you're big enough, Google provides excellent support. I was once responsible for a product that generated $30k/month in adsense revenue. It got banned due to adult content and my attempts to clean everything up weren't entirely successful. I ended up on chat with someone from adsense and they essentially held my hand for an hour as we found and removed every bit of porn in the product.
- lostgame 8y agoOkay, I think I can do one worse. Imagine if the justice system was like Apple's App Store acceptance/rejection system. Sorry for putting that in your head, y'all.
- slyall 8y agoA classic short story along this line is: Computers Don't Argue by Gordon R. Dickson A man has a complaint with his book club, but a series of errors escalate things alarmingly. The Wikipedia article has the whole plot: https://en.wikipedia.org/wiki/Computers_Don%27t_Argue https://en.wikipedia.org/wiki/Computers_Don%27t_Argue
- ryacko 8y agoOr have the actual story: https://www.atariarchives.org/bcc2/showpage.php?page=133 https://www.atariarchives.org/bcc2/showpage.php?page=133
- jammygit 8y agoI feel weird about reading sci fi for free online. I feel like its unnatural and that it should come from used bookstores like god intended. I miss used bookstores, most of the big ones in my city have closed down
- 8bitsrule 8y agoAccording to one online source, 'jammygit', you bought several Phil Dick novels at one of those used bookstores. Maybe that's why you feel 'weird'. I also regret the loss of so many used bookstores ... one of my favorite haunts. We can only try to visit the survivors much more often.
- modzu 8y agoPKD's electric sheep was one of the first things i read "online" -- via BBS baby
- 8bitsrule 8y agoOh my. I've read that story in the distant past. It was merely hilarious at that time. Today I was grimacing.
- delbel 8y agoThis is literally the plot to the 1971 George Lucas sci-fi film THX 1138, down to the detail: "set in a dystopian future in which the populace is controlled through android police ... ."
- brailsafe 8y agoGoogle has support?
- 013a 8y agoThe purpose of computers is to enhance the ability of humans who have a job to accomplish, not do their job for them. If we don't figure this out as a society, we're doomed.
- CogitoCogito 8y agoAlso the operators of a computer system should always be held responsible for any errors of that system. Mistakes by the system are your* mistakes. Any decisions made by the system must be explained by you. All costs rectifying those mistakes must be bared by you. *Obviously "you" means your organization, company, etc. and not necessarily some random office clerk.
- sgustard 8y ago"I predict that within fifteen years, AI will be able to replace 40 to 50 percent of jobs in the US." - Kai-Fu Lee
- deleted 8y ago[deleted]
- dmix 8y agoThe justice system is hardly automated. It's one of the least automated systems (ie, employing technology to reduce inefficiencies) of all industry 'niches' in the western world. Law enforcement might be using technology to dole out punishment/fines more efficiently, but the actual justice system that civilians have to deal with after getting fined or charged (ie, the world of lawyers, judges, and courts) is extremely backwards in their use of humans, paper, fax machines, etc. Not using even basic technology. The justice system is famous for the extreme slowness for the amount of human-time required in processing of even the most basic small details. By eliminating the amount of human-time (by a hired lawyer or oneself) that each person has to engage the systems (for example by missing work or paying with money instead of time) would do a lot of help the poor better deal with the justice system - and get better and fairer treatment by the system.
- jammygit 8y agoon a tangent... "caught in a Kafkaesque nightmare" I've been meaning to read kafka one day. Is there a title in specific you would recommend to start with?
- maxxxxx 8y agoI don't know how good the English translations are but I would look at "the Trial" or "Metamorphosis". This is a good short story: https://genius.com/Franz-kafka-before-the-law-annotated https://genius.com/Franz-kafka-before-the-law-annotated
- crazygringo 8y agoThat's absolutely hilarious. But at least it was easily fixed. But to anyone complaining about how this is dystopian: compare it to human police officers who can racially profile and exhibit their prejudices to disproportionately target segments of the population, and which there is absolutely zero easy fix for, especially when we consider how prevalent unconscious bias is. At least an automated system can be analyzed for bias and, when it's found, corrected. There's a lot to be said (in terms of justice) for fair and uniform enforcement of a law, assuming of course there is due process for appeals (and that the law is democratically legitimate in the first place).
- reaperducer 8y agocompare it to human police officers who can racially profile and exhibit their prejudices It's already been demonstrated that computers can racially profile people. Everything from consumer cameras that tell the users that people in the pictures don't have their eyes open, but in reality are Asian; to chatbots that turn into Nazi bullhorns. You can at least train a bad cop to be better. Or fire them. How do you fire an AI into which an authoritarian government poured billions of dollars, years of work, and reams of press releases? AI was made by people, and because of that will always be flawed. It's only in science fiction books that computers and algorithms are perfect and clean and rational.
- dymk 8y agoIt's far easier to iterate on an AI and deploy updates statewide than it is to somehow weed out the individual biases of each individual cop.
- BonsaiDen 8y agoThat however goes both ways, making it also far easier to install a dictatorship etc. because you don't have to replace tens of thousands of police officers with regime loyal ones. We're walking on razors edge here, it only takes one "election gone wrong" and we might find ourselves in a nightmare with no escape.
- SN76477 8y agoImagine that she wasnt a famous CEO and lost her job and spent the rest of her life struggling because of a silly mistake.
- reaperducer 8y agoIt's China. So she's just have her social credit score repeatedly dinged every time the bus ran a red light until she wasn't allowed to travel anymore, wasn't allowed to work anymore, wasn't allowed to have a bank account, or allowed out of her apartment. Bureaucracy is so much more efficient when everything is automated, and we're all tabulated. We can just let the computers disappear troublesome people. If a few people are disenfranchised for no fault of their own, that's OK. They're just "edge cases." /s
- tlrobinson 8y agoBasically the Black Mirror “Nosedive” episode.
- gsich 8y agoAn even more dystopian view can be seen in the episode "White Christmas" at the end. I don't know if you have seen it, so I won't spoil it here.
- pteredactyl 8y agoThis is exactly why the role of the state, in general, needs a much closer examination.
- loeg 8y agoIt's dystopian China. It's already basically several Black Mirror episodes put together. It doesn't generalize.
- mytherin 8y agoThe exact same things are done constantly by big companies (Google, Apple, etc.). The only difference is you have much less recourse when it happens there. At least the government gives you rights you can fight for in court.
- sharemywin 8y agothat's why you need a good appeals process.
- quickthrower2 8y agoYep. Pay the fine or hire a lawyer to defend you in court. If you lose pay court costs and bigger fine.
- LanceH 8y agoAnd if you win, you pay the lawyer, court fees and administrative costs. The last ticket I had for speeding was rougly $90. The speeding fine was only $30 of that -- the other $60 was not recoverable with a win in court.
- KaoruAoiShiho 8y agoThat this is possible is concerning, it means the system is not smart enough. Might open the possibility of trolling by holding up cardboard cutouts of people and running across the street. Of course that's easier said than done but still...
- Zarath 8y agoHilarious, but unfortunately, incidents like these are used to "prove" how incompetent and useless these technologies are, all the while pushing forward their use.
- netsharc 8y agoPull Request: Make arrest system more bug-resilient. Unit test: drop bug while typewriter types out "Tuttle", make sure it doesn't type "Buttle".
- lisper 8y agoIronically, the better the AI, the worse this problem is going to get. The error rate will never be zero, but the closer it gets to zero, the more the AI's judgement will be presumed to be correct by the legal system. So if you happen to be one of those unlucky souls who draws the false-positive card in an era where those are rare, you could be totally hosed.
- red75prime 8y agoAt least you can hope that upcoming updates can fix it.
- EGreg 8y agoI always thought this movie premise was entirely contrived but now I am not so sure: https://m.youtube.com/watch?v=TVvGEj84y98 https://m.youtube.com/watch?v=TVvGEj84y98
- madrox 8y agoBecause our current justice system never hands out false positive cards today. The real question should be if this is better than the false positive rate we have today, though that’s probably unknowable. At least when AI fails, it fails spectacularly.
- lisper 8y agoNo. Sometimes AI fails spectacularly, and sometimes it fails banally (not to be confused with benignly). But when it fails spectacularly it makes the papers. When it fails banally you never hear about it. Today society has a healthy skepticism of AI because it's new and untried. But some day that will change. AI will get better, and the spectacular failures in particular will be reigned in. Nonetheless, it will still fail some times. Society may forget that if we don't have the spectacular failures to remind us. > our current justice system never hands out false positive cards today. Of course it does. The difference is that we know that humans are fallible. At the moment we still know this about machines too. My concern is what happens when the machines start to appear infallible, but still aren't.
- danso 8y agoThis system seems dumber and simpler than I had imagined it would be. I had guessed the face recognition system would be linked to each citizen's cellphone (i.e. GPS coordinates). How else could it be so certain of any given person's identity, in densely and highly populated areas?
- guitarbill 8y agoOn the other hand, would be pretty easy to fool then by leaving your phone at home.
- danso 8y agoThat would leave the average citizen at a huge inconvenience, given how much of modern life is tied to smartphone usage. Doesn’t seem worth it just so you can go on a jaywalking/red-light-running spree.
- guitarbill 8y agoI've managed to survive a few hours without my smartphone. I now also live in an American city made for cars, not humans, so "jaywalking" is basically required to make any sane progress. So for grocery shopping, it'd take that trade-off.
- pbhjpbhj 8y agoIf you leave your phone at home the automated number plate reader you pass on the way out of your village/suburb/estate stops and locks your electric vehicle and you wait inside for 3 hours. A police tow truck takes you to court. Two days later you're releases by the [AI] judges clerk ... you decide to always carry your government ID enabled mobile device in future ...
- mtgx 8y agoAll existing AI (and I mean all, even AlphaGo Zero, etc) is overhyped. With all of these systems the devil is always in the details and it's never as good as presented in a blog post or video. Often, even its creators don't realize about all of its shortcomings.
- raleigh_user 8y agoHeres further proof the justice system is screwed. Yesterday I was stopped while driving for not having current tags. Registration was renewed but I didn't have the sticker yet. Cop looked at my paperwork said all good be back in 5. Come to find out, I haven't had a valid license in 5 years. There was a mistake in court 5 years ago when I paid a fine (speeding ticket). I have a receipt from court saying all settled/money received but court says they never got money. So, after talking to a lawyer yesterday I will be paying another $250 for lawyer fees + court costs to get out of the mistake made by the court in another state 5 years ago. I am lucky. I can pay that. It sucks but some people would be out food money if they suddenly had to pay $500 to drive again. But, I did nothing wrong (except for speeding 6 years ago which I paid the fine and haven't had any issues since) and am stuck paying for the courts mistake.
- lotsofpulp 8y agoHold the government employees liable (monetarily and criminally if they cause a loss of rights) for their mistakes and the problem will fix itself in quick order.
- raleigh_user 8y agoThat might help. I am no expert on govt stuff so I will defer to those who are. But it does suck I don't have an ID right now. You never notice how often you use it (travel, bars, events) until you don't have one!
- joecool1029 8y agoPassport card is something I recommend. When traveling in states out of the area, some don't like my license as primary ID. There's a second privacy benefit too, the passport card does not have your address on it for bars that scan licenses and keep the addresses in database.
- allannienhuis 8y agoIf employees were personally liable for costs of mistakes, no one would do the work - too much personal risk involved. Also, the degree of blaming anyone-but-me would be off the charts, crippling the organization. Having some type of accountability is important - incompetent people should not be allowed to keep their job, and institutions should have some incentive to improve, but not sure putting all the burden on individuals is the right answer. What if you had to pay the cost of every bug you created? I'm sure a lot of us developers would be digging ditches instead. :)
- gboliknow 8y agoI am a web developer and I need a jobs.I am expert in frontend and could do backend easily ....with frameworks like jqeury,bootstrap and laravel
- uptownfunk 8y agoWe love any link that has machine learning in the title and quickly vote it to the top. This is the future guys, who knows what’s next.
- newname2018 8y agoIt's good that some famous person is getting hit with this.maybe it will create doubt within the court system that we all may benefit from.
- infinity0 8y ago> The police said they have upgraded their tech to avoid issues like this in the future. if (person_id == "Dong Mingzhu") { // TODO: generalise this return false }
- themodelplumber 8y agoHow many of us would be surprised to learn that not a single government official can now be auto-charged with jaywalking? There's generalization accomplished in minutes.
- gsich 8y agoMy thoughts too. That is the "beauty" of AI. I don't think they will retrain the model, as you probably don't know what caused this in the first place. The "TODO" will be replaced with a db query.
- sgzfx 8y agowhich soon becomes if (person_score > 10) ...
- anticensor 8y agono, it would become if (person_score == 100)
- ravenstine 8y agoSomeone could just do that every time that problem occurs; commit a new if-statement, close the ticket, and upper management says "Look at how much work this guy gets done! Now there's our star programmer."
- bah_zillionaire 8y ago> A billion lines of code??? Incredible!!!
- deleted 8y ago[deleted]
- max_sendfeld 8y agoI once read about a proposed 1$ fee for mistakes like that. For individuals, it's just one dollar. For companies and especially the state, it can cost 1000$ and more in processing fees altogether.
- Joakal 8y agoI once read that companies can clearly claim increased costs and thus increased fines. Alternative is that government loses the provider.
- aw3c2 8y agoWhat is the trustworthiness of this website?
- dongping 8y agoCaixin is a reputable magazine on financial news in China, its chinese PMI (Purchasing Managers' Indexes） are widely cited. [1] https://www.scmp.com/business/china-business/article/1385987/new-boss-and-backing-pipeline-hu-shulis-caixin-media-group https://www.scmp.com/business/china-business/article/1385987...
- Evidlo 8y agoOthers have pointed out that it doesn't seem possible to identify a single face out of millions with enough confidence to issue a fine with current technology.
- tempodox 8y agoMaybe calling a machine that has just demonstrated its profound stupidity an “Artificial Intelligence” is part of the problem. This crappy tech is overhyped and overrated.
- tmkc13 8y agoFunny what happens when you run before you can walk, but with any such technology, there would be false positives even when this gets improved significantly. At such a nascent stage, this is bound to happen.
- arghwhat 8y agoYes, but even hidden, it's probably still going to appear quite significant, and I do believe that at some point, the fine size becomes nonsensical. But then again, I do not think that who you are or what you have should change the magnitude of the crime you commit. I don't think it is sensible that I should get greater fines because I chose to bust my ass and suffer from stress to gain more for my family, whereas I could get a much smaller fine if I just lived a more laid back life. Granted, it's supposed to differentiate between the rich and poor, but in doing so it also differentiates between the hardworking and the lazy, punishing productivity and rewarding laziness.
- monadgonad 8y agoOh, give it a break. People don't choose to be poor.
- cle 8y agoSome do, some don't. Blanket statements in either direction are incorrect and unproductive.
- arghwhat 8y agoI did not say that people chose to be poor. However, it does not seem fair to be punished for working harder than required.
- askvictor 8y agoThis is assuming wealth/income is cause by hard work; while that's true for some people, many are rich because their parents are rich (would be curious to see stats around this)
- arghwhat 8y agoI cannot say much about the stats, but I only really know of rich-by-work rather than rich-by-luck. But I do not think we should design a system to cater for outliers with immense wealth. We should cater for the general population, and I believe that for non-Dubai prince wealth, it is likely that it likely came through work.
- jacquesm 8y agoGood thing they didn't fine her for speeding at the same time. I find it more than a little bit implausible that facial recognition accuracy is good enough to pluck a face out of a billion people and accurately identify a single individual to the point that a fine would be automatically issued, especially without corroborating evidence that the person fined was even there.
- chromanoid 8y ago> Face recognition algorithm classifies bus-side ad as person and triggers an automated regulatory fininig process. FTFY
- debt 8y agoEvery time I see the acronym “AI” I cringe a little bit. It’s literally not AI! It’s very quick statistical work; which works for many circumstances but definitely is not AI. Especially when it doesn’t work.
- jaimex2 8y ago"debuted crime-fighting facial recognition technology to much fanfare over the past year" People wanted this in China or is this a state run news source?
- aaron695 8y agoBe careful no-one actually links to the real info on HN. We wouldn't want to stray to far from Fox. (The fact it does say the persons name publically is scary, the fact they got it wrong is technically interesting) https://www.abacusnews.com/digital-life/facial-recognition-camera-catches-top-businesswoman-jaywalking-because-her-face-was-bus/article/2174508 https://www.abacusnews.com/digital-life/facial-recognition-c...
- Buge 8y agoStrange, the title says jaywalking, but the body says running a red light.
- usaar333 8y agoI'm seriously confused by what went down. First off, why are the police even reporting this? Secondly, what can the system even do? I'd believe a system could detect a person jaywalking. Identifying them is a huge problem though. running facial recognition against a billion person corpus seems impossible (in fact what even is your match data coming from?). (?). Wouldn't you need to narrow it down somehow, perhaps using phone GPS records?
- kkarakk 8y agochinese corp claims it can complete matching the whole population in 1 second but it's china so who knows what it's actually doing http://www.atimes.com/article/surveillance-system-can-scan-chinese-population-in-1-second/ http://www.atimes.com/article/surveillance-system-can-scan-c...
- usaar333 8y agoYa, I'm dubious there. accuracy rate "up to" 99.8%? What does "up to" mean? This article paints a much more realistic view of what face rec can do, which jives with my 10 year dated experience in this field: https://www.businessinsider.com/china-facial-recognition-limitations-2018-7 https://www.businessinsider.com/china-facial-recognition-lim... Impressive for sure, but nowhere at the ability of scanning the entire country (which in addition to no algorithm being capable of doing that, you also have the problem that you lack data to even match against the entire country)