14 ms·
AI Camera Ruins Soccer Game After Mistaking Referee's Bald Head for Ball
- skunkworker 6y agoYouTube video for those interested, music overlay warning though. https://youtu.be/9zoJP2FkpgU https://youtu.be/9zoJP2FkpgU
- woliveirajr 6y agoSometimes the ball was behind some player and the referee's head was reflecting the sun light and was the only visible "ball".
- Buttons840 6y agoI, for one, had a good laugh at the music.
- mam2 6y agoThis is hilarious. Science is made better mistake after mistake. Better in a football game than in a shuttle, or a law enforcement AI.
- st1x7 6y agoOnce the algorithm is used commercially and in production I wouldn't necessarily call it science anymore.
- qzw 6y agoAnd it's totally okay! Because nobody died, we all get a nice laugh, and the AI will be tweaked to make it better. It's the kind of move-fast-and-break-things that I can get behind.
- deleted 6y ago[deleted]
- ramoz 6y agoI don't like this mindset. AI is greenfield for everyone and the consequences seem greater than any modern innovation. e.g. Tesla deaths.
- ncallaway 6y agoI think the parent was saying it's context dependent. AI for controlling a football camera, sure, move fast and break things. Why not. AI for controlling a moving motor vehicle, maybe we should be extremely cautious about our testing and use of AI.
- bartread 6y agoI totally get your perspective... but you're clearly not a fan of either team involved.
- fastball 6y agoIf someone's day is ruined because the AI camera was following a bald ref, they should probably take a step back and re-evaluate what's actually important in life.
- Fomite 6y ago"I was looking forward to watching my favorite local team as an outlet for the massive stress of living through a global pandemic, and instead it kept zooming to the ref's head" is, at the very least, reasonably a massive let down.
- mc32 6y agoProbably just the training data not including enough bald people...
- viraptor 6y agoI'm sure the training data also didn't include this person http://1.bp.blogspot.com/_XU9x8G7khv0/SklFgSPQOdI/AAAAAAAAGiE/o_-Q5LI3hnc/s400/ball-buzz.jpg http://1.bp.blogspot.com/_XU9x8G7khv0/SklFgSPQOdI/AAAAAAAAGi... And it's just bound to happen at some point.
- devmunchies 6y agoOr it misclassified the training data too and they didn’t catch it. It’s not like AI engineers manually run all varieties of footage against the trained model to check accuracy.
- raverbashing 6y agoThis is not about training data I think the ML specialists often forget there's a gap between pattern matching and solving a problem The problem here is: locating a ball and moving the camera If you just try to find a ball on static image by static image you're going to have a bad time. Even worse if you try to find only the ball but without the play contexts (kicking, flying, being held, etc) Balls "teleport" but not always Yes, a ball can be on top of a player, but not all the time So, no, if you didn't see the ball go there, it didn't suddenly appear in the middle of the field (but then again, with some exceptions).
- papaf 6y agoIf you just try to find a ball on static image by static image you're going to have a bad time. This isn't the point you were making, but as an aside, I think converting video into static images is throwing away useful information. Most video codecs work by encoding the changes between video frames which includes movement information. It seems hard, but worthwhile, to use this information for object recognition. I just did a search and saw that these techniques go by the name "compressed domain analysis" [1]. [1] https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=compressed+domain+analysis&btnG= https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=comp...
- loeg 6y agoNext time they'll have the bald ref wear a hat.
- meekrohprocess 6y agoOr they'll paint their head like a soccer ball to make sure that the updated AI works properly. These sorts of failures demand stress-tests. What if a future team has a soccer-ball-headed mascot?
- inopinatus 6y agoWe can certainly foresee a future, one the uncharitable might call dystopian, in which all the humans have been barcoded with purpose & capability metadata so that AIs can process them more efficiently.
- phreack 6y agoIt's already begun to gestate, some external things are found to be easier to change than fixing software https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates https://www.theverge.com/2020/8/6/21355674/human-genes-renam...
- cmroanirgo 6y agoIt's ironic, especially when the obvious engineering solution would be to add tracking paint/sigils on the ball... and doubly ironic when the engineering solution would be unlikely to be accepted.
- oarsinsync 6y agoOnly ironic to software developers. The engineers who produced the ball understand that changing the properties of the ball in that way would reflect in the aerodynamic properties of the ball, which gets fixed at the start of competition. It’s not hard to fix your software that you created. It’s much harder to get global consensus on a change that wasn’t required before your buggy code was released into the wild.
- ddevault 6y agoIt's not that great. Remember that a real person used to do this work, and they've been replaced by a buggy robot in the name of cost savings. That's one fewer job on the market. That real person would also have been able to do things like direct the camera to follow whatever was most important - even if it wasn't happening next to the ball.
- chmod775 6y agoPeople need to realize that we're quickly heading towards a world that won't need as much human labor as we have available. And that's a good thing. There's only so many bullshit paper-pushing jobs and useless over-budget infrastructure projects we can create to make up for the lack of work. A better way to deal with too much labor on the market would be setting government-mandated maximum work-hours, so as to spread the amount of available work evenly among the entire populace. Secondly - for those countries where this isn't a reality yet - school and university should be free and you should receive government aid while attending them, to allow those out of a job to learn new skills or just pursue their interests.
- kzrdude 6y agoIt's only a good thing if we can turn around our societies to handle this new reality and where we can keep our political and enconomical freedoms, including of course a chance at a decent living
- Viliam1234 6y agoA four-day workweek would be a nice start.
- AnIdiotOnTheNet 6y agoThat'd be better for us, sure, but I think it is more likely that those at the top will will simply find a way to reduce the excess labor class.
- giovannibonetti 6y ago
- rriepe 6y agoIs it, though? These are soccer fans we're talking about. An AI misreporting a loss for a win or a win for a loss could completely change the tone of the post-game riot.
- suyash 6y agoYou wouldn't say that is the camera was assisting with a serious surgery or firing a military rocket, it's not okay, we need to make sure the bar is set very high and there needs to be penalties/ fines for mistakes if we are going to jump on the fully autonomous bandwagon.
- tdeck 6y agoThose applications already have higher standards for general software reliability, to say nothing of AI. Obviously the bar is different for different applications.
- bobthepanda 6y agoThat sounds wildly out of proportion for a soccer match where you can just recheck the footage
- cmroanirgo 6y agoConsidering that at least a cameraman has lost his job recently, there's always going to be a social cost to moving to pure tech. It's never just one thing (ie. It's not just a soccer match)
- ARandomerDude 6y agoNo rockets, no surgery here. Just a game. It's probably ok to lower the budget by an order of magnitude and open a ticket in Jira after the match.
- jader201 6y ago> You wouldn't say that is the camera was assisting with a serious surgery or firing a military rocket No you wouldn’t. But it’s not. “Move fast, break things” is perfectly ok for certain applications, such as recreational AI. Nobody said it’s ok for medical or military applications. You can live in a world where the bar for quality control varies depending on the application. Experimentation will never happen if we set medical/military grade expectations across the board.
- matthewfelgate 6y agoI hope pundits will get replaced by AI next... I mean how would it be for a AI to take in a score and output "they played well"...
- Areading314 6y agoYou could probably replicate entire sports broadcasts with a few GANs
- m00x 6y agoWhy would you use a GAN for this?
- kens 6y agoThe company StatSheet (now Automated Insights) has been using algorithms to produce sports journalism and business reports since 2007. It basically takes the score and background information and turns it into a paragraph or two. Earnings reports stories are similar. The results are better than you'd expect. https://techcrunch.com/2010/11/12/automated-news-sports-statsheet/ https://techcrunch.com/2010/11/12/automated-news-sports-stat... https://en.wikipedia.org/wiki/Automated_Insights https://en.wikipedia.org/wiki/Automated_Insights
- gentleman11 6y agoThose articles are so annoying to stumble on. They rank very highly in search results because the google crawlers like them but they are obviously written by some lame algorithm and are a waste of time to skim. I hate those types of article so much
- schemescape 6y ago> I hope pundits will get replaced by AI next... “Did you see that ludicrous display last night? ...”
- iso1631 6y ago
- jml7c5 6y agoI wonder what went wrong here. Perhaps it wasn't trained for these lighting conditions? I can't tell if they use multiple cameras around the stadium or not.
- XCSme 6y agoThe thing about AI is that you never truly know what it actually looks looks for. This reminds me of the tanks neural net story: https://www.gwern.net/Tanks https://www.gwern.net/Tanks
- ArnoVW 6y agoIt used to be that we had to chose between models that were 'powerful' (complex neural networks) and those that were 'interpretable' (linear régression, trees). In the last years a lot of research has come out. We have since then developed ways of analysing the inner workings of neural networks. You can understand how each feature drives the predicted result by using SHAP https://github.com/slundberg/shap https://github.com/slundberg/shap You can analyse the individual layers and gain an understanding of how each layer encodes the input https://ai.googleblog.com/2017/11/interpreting-deep-neural-networks-with.html https://ai.googleblog.com/2017/11/interpreting-deep-neural-n... https://www.kdnuggets.com/2019/07/google-technique-understand-neural-networks-thinking.html https://www.kdnuggets.com/2019/07/google-technique-understan... Do we 'know' in an absolute way why a network thinks something? Not yet. But neither do we know that for humans. We just have a huge amount of experience with their failure modes and we work around them (see aviation).
- XCSme 6y agoThat's true, I know that you can view what "excites" specific neurons. I still think the is a problem: this only shows whenenever you give an image what it uses from that image to classify the object, but when a new, unseen image is given it's really hard to tell beforehand what the network will actually look like and which pixels would be considered. Yes, you could look at the features and see which patterns appear there, but for large networks those are thousands or millions, so it's still hard to tell what the result will be, unless you actually use the new image as an input for the network.
- mwcampbell 6y agoReminds me of an incident I read about in Hackers by Steven Levy, where a ball-catching robot in the MIT AI Lab (in the 60s or 70s) mistook Prof. Marvin Minsky's bald head for a ping-pong ball.
- PaulKeeble 6y agoI am not seeing an obvious scenario that makes it switch. I thought maybe it was when the ball "joined" with a player but it does it when its in motion too. That is just completely broken. We often hear that the AI is more accurate than humans in some image matching, but it also clearly makes mistakes that no human would make. Well here is a great example of such a mistake which a human wouldn't be making.
- grp000 6y agoits probably finding bounding boxes based on highest probability %. So, it's a shame the refs head looks more like a soccer ball than the soccer ball!
- hda2 6y ago.. to the AI. The fundamental problem is that no training set can ever be really complete. Humans have general intelligence to deal with this. ML does not.
- novia 6y agothe biggest problem is that ML/AI has no conception of time, context, continuity, or cause and effect.
- NateEag 6y agoWhich is why some of us avoid calling these systems "AI". Yes, the term has become accepted, but it's a horribly irresponsible label because it massively misleads laypeople about how these systems work and how trustworthy they are.
- gentleman11 6y agoWhat’s a better catch all term? Data modelling system? Prediction system?
- tedunangst 6y agoSince the game was ruined, they'll have to replay the match to determine a winner, right?
- cblconfederate 6y agoCamera has obviously developed a crush for said referee. We can't just ignore that AI's got feelings. It's so wrongheaded
- kps 6y agoDon't anthropomorphize computers. They don't like it.
- aaron695 6y agoWow, so this companies tech also does automated - "highlights, replays, and adverts" https://www.pixellot.tv/ https://www.pixellot.tv/
- stubish 6y agoAnd I imagine one day you can rent a camera and have your kid's little league matches presented like a division 1. (Or maybe you won't even need to rent the camera, if you use the free version supplied by Google which automatically adds personalized sponsorship visuals) (Or the Facebook version, automatically posting 'hilarious' and embarrassing snippets designed to maximize user engagement)
- joe_the_user 6y agoIt's kind of mystifying why a company would go to the trouble of having AI-based soccer camera to begin with. I can't imagine that the training and maintenance of such a system could be cheaper than the pay of a single camera person and I don't remember ordinary camera people having much trouble following the ball in soccer. Which is to say, while current ML/AI may work well for some things, for other things, it will be abandoned as a fad unless there's some fundamental improvement.
- recursivecaveat 6y agoI can imagine it being cheaper if there's a 3rd party who just leases it to you as a push-button system, and they service a bunch of teams. On the other hand it does seem incredibly penny wise and pound foolish of management. Broadcasting is like half of a pro sports team's revenue in a normal, non-lockdown year. Why you would entrust that to an AI just to eliminate a single, semi-skilled laborer is beyond me.
- LexGray 6y agoModern sports do not have just a few cameras. With the tech there can be a hundred perspectives. Soccer used to be a few cams covering the whole field. Ai should be able to predict power and angles of the kick for the most exciting framing making for better experiences.
- gentleman11 6y agoOr nonsensical drifting into the bleachers to look at an unusually round baby
- recursivecaveat 6y agoI thought about that but the video from the article seems to be only a single camera. If there's really only one running, then that seems like an odd choice in and off itself. Perhaps only one malfunctioned because it's closer to that particular lineman.
- 6y ago
- shannifin 6y agoWhile it's hilarious, I wonder why they didn't keep a human backup available just in case. Anyway, I think the solution is clear: the ref must be replaced by AI.
- undersuit 6y agoBecause they fired the human. Haha.
- babesh 6y agoAll sorts of edge cases here to handle. Its not just a bald head. What happens if there are multiple balls? What happens if the ball is redesigned with a different pattern? What happens if someone wears a shirt with a picture of a ball? Does it make sense to center the camera on the ball? Does it make sense to also show where the ball can go? I wonder how it's handled by people currently? I suppose it can be handled with more training data/labelling instead of custom coding. We are looking are replacing the camera person job with AI just as we replaced other people's jobs with programming earlier and just as we replaced other people's jobs with AI more recently.
- soylentgraham 6y agoI worked at a company doing football (and footballer) tracking for ecery premier league game for a couple of years. Every other week there was a case we hadnt covered (and fixed for the next week); - white ball swapped for orange ball - player swapped shirt, new number. (We tracked players by number on their back) - player swapped shirt to a number someone else had - player swapped shirt to no number - player shirt covered entirely in mud - player swaps shirt where number is white instead of black - white ball left on sideline (2 balls) - crowd releases 50 white balloons onto the pitch - ref swapped shirt colour - sideline advert has a filled in white O on it (moving around) There were loads more, but that was off the top of my head. I dont think we ever mistake a head for a ball though! We worked out likely balls from movement (syncing cameras with bounces)
- gnrlst 6y agoIverness Caledonian Thistle sound like an auto generated human friendly code name / url
- rkagerer 6y agoThis pretty much sums up my experience with apps whose devs thought they could make their code smarter than me.
- interestica 6y agoIt's crazy thinking about the amount of technology that went into the infamous "Fox Puck" for tracking and display (where they literally cut a puck in half to embed tech and coordinate with an array of receivers in the arena) to what can now be done by the cameras themselves. https://en.m.wikipedia.org/wiki/FoxTrax https://en.m.wikipedia.org/wiki/FoxTrax I suspect in this instance it wasn't just the bald head, but the particular near-sunset lighting that was a contributing factor.
- faeyanpiraat 6y agoIt's interesting, here is a short 10 min youtube documentary about it: https://www.youtube.com/watch?v=DCoUQy34O7I https://www.youtube.com/watch?v=DCoUQy34O7I
- manc_lad 6y agoThat was a good watch. thanks for sharing. Great example of first iteration tech being the tip of the iceberg.
- mkl95 6y agoI love it. It feels like a Monty Python sketch.
- sbassi 6y agoWhy are so cheap to replace a cameraman by an AI? They could leave the camera control from remote to someone in a low income country, would make a better job for peanuts.
- tjah1087 6y agoA few reasons why this outsourcing is more extensive than you might think: Latency: (cheap labour is usually far away from the match). This makes teleoperation harder. Bandwidth (TV ops run from a van in a car park - so getting a reliable video feed out is expensive). Logistics: you still need to hire, train and utilise that person effectively. Reliability: what if your remote operator's internet goes down?
- tjah1087 6y agoThis is a surprisingly easy mistake to make. Background: I was one of the original engineers on the Goal Line Technology system now used by the vast majority of professional leagues. Heads are surprisingly ball shaped from a a lot of camera angles, and the male balding pattern conspires to give a good impression of different football graphical designs. When we were developing the system, we thought we had ball detection pretty nailed, and we never had false positives from any of the heads of engineers on the team. Then, our boss came to test the system ... As far as our algorithms were concerned, his head was exactly a football, just as in the article! Highly embarrassing, but everyone saw the funny side. It gave us better data and inspired a few more robustness checks that ended up being crucial later on! :)
- Udik 6y agoSeems pretty silly to try to classify a ball as a round object, period. A ball is a round object that doesn't sit on someone's shoulders.
- jon-wood 6y agoUnless the camera is angled in such a way that a ball kicked up in the air obscures a player’s head.
- Udik 6y agoThis will happen, of course, but shouldn't last long. Also momentum of the tracked object should help. Btw, funny that my comment above attracted so much hatred. Didn't want to sound like a know-it-all but yes, at least in the case of the referee I guess that checking for a body would have helped. For example, in this stylised picture we can easily tell the player's head from the ball: https://www.pngitem.com/pimgs/m/80-803112_italia-italia-90-logo-png-transparent-png.png https://www.pngitem.com/pimgs/m/80-803112_italia-italia-90-l...
- Luechkt 6y agoNot sure how heads are "surprisingly ball shaped." What part of a round head surprises you exactly?
- lmilcin 6y agoI guess it is "No bald referees" going on. At least they can wear hats. If you are black and rely on face detection you are out of luck.
- robodale 6y agoMaybe the referees could wear a little red cone or something similar...so the AI camera could filter out the false positives?