6 ms·
A school locked down after AI flagged a gun. It was a clarinet
- CoastalCoder 11mo agoGood grief, imagine if it had been an oboe.
- derelicta 11mo agoJust ban music instruments altogether then! No need to fix the dang LLM prompt this way! /s
- Zobat 11mo agoThey tried to find contraband, they found a marching band!
- gosub100 11mo agoThe sleazy AI company that sold this "tech" to the school has to face the music.
- b112 11mo agoSleazy? I'll have you know, the ToS clearly says it's in beta.
- sunrunner 11mo agoNext time the contraband could be a contrabassoon.
- bryanrasmussen 11mo agoSeventy-six trombones caught the morning sun With a hundred and ten cornets right behind The AI thought about it long and hard calling up the national guard Cause of the horns of ev'ry shape and kind. --- There were copper bottom tympani in horse platoons Thundering, thundering all along the way. Double bell euphoniums and big bassoons, And Swarming SWAT Teams Goons so they say --- There were cross fires and blown tires And reporters from the local news Clarinets of ev'ry size And trumpeters who'd improvise And video games that went pew-pew! ---- on multiple edits: tried to find a layout that fit the song
- kotaKat 11mo agoThose banned band books… they might have come across One Thousand and One Vulgar Marching Band Formations!
- swarnie 11mo agoI have so, so many questions..... Why are there cameras in schools? Why are they looking for guns? Why are they using AI to do it? Could this not all be avoided by not letting kids have guns or am i missing the point?
- spicyusername 11mo agoThere are cameras in schools for the same reasons there are cameras anywhere, to try to maximize accountability. Kids are already not allowed to have guns.
- lotsofpulp 11mo agoAs a parent, I would like more cameras in schools, since students’ abuse gets ignored, and the person fighting back gets more punishment than the instigator.
- theoreticalmal 11mo agoHow would cameras solve the punishment issue? Aren’t most schools still “zero tolerance” for physical violence?
- narag 11mo agoThe post you are responding to is about punishing the victim because teachers are too lazy/cowards to punish the culprits. Cams incentivize them to do the right thing.
- theandrewbailey 11mo agohttps://archive.ph/WbOWl https://archive.ph/WbOWl
- vdupras 11mo agoNext up, the guy, probably named Buttle, will get billed for the intervention.
- saidnooneever 11mo agoweapon of brass destruction
- theflyingelvis 11mo agoAt least it wasn’t a percussive device
- Rooster61 11mo agoIf it was, they'd need to get the cops on the horn quickly
- rmunn 11mo agoThis hardly needs to be said here, but there should have been human review of the AI output before taking any drastic action. That would (I assume, though since I can't read the original article I don't know if that assumption is correct) have immediately let them know that the alleged "gun" was nothing of the sort, and avoided the massive disruption of a totally unnecessary lockdown.
- nucleardog 11mo agoNot sure how much that would help overall. Unless it's completely clear that it's not a gun, the reviewer is essentially always going to pull the alarm. The risk of a false alarm is going to be seen as minimal, while the risk of a false negative is catastrophic. False alarm makes the news for now because it's novel, we all go "What the hell, guys?" and life goes on. Nobody wants to end up sitting in front of a prosecutor, the media, etc explaining why they chose not to pull the alarm, when the AI _clearly_ identified the gun, and instead chose to let all those kids die.
- jermaustin1 11mo agoOne more instance of offloading blame to a computer system: "It's not my fault the cops shot the kid, the system said it was a gun." The only way this gets fixed is if there are consequences at every level for false positives.
- WillAdams 11mo agoOne approach for this is that the person who makes the call needs to be on-site and in the front of the situation --- similarly, a judge signing off on a No-Knock Warrant --- the judge needs to be at least be present, and should be required to walk through the building/home/apartment after the warrant is served. If it's not important/severe enough for a judge to do this, then I would argue that there's no need for the "no knock" aspect.
- brk 11mo agoThat sounds good on paper, but is really impossible to implement in any practical way. In this case, the kid was holding the clarinet like a weapon, and though we have not seen the actual video, the descriptions of it make it sound like overall resolution was poor. The alternative to the false positive here, is to not report anything that you cannot be 110% certain of, which means that you're likely to miss some true positives. Overall this situation mostly reads like everything worked as intended, and the press turned it into more than it needed to be. School shooting are a real thing, there is plenty of evidence of that. Weapons detection has become a necessary component of a school safety strategy. For many reasons, it is not practical to have personnel at the school, or within the district, act as the first-pass reviewer of AI detections of weapons.
- Vinnl 11mo agoI feel like as soon as a particular type of student learns that this is used, they'll have an excellent way to get that test that they didn't study for postponed, and even have plausible deniability that they didn't intend to lock the school down. At least for the first one or two times, after that it's back to triggering the fire alarm.
- estimator7292 11mo agoAt my high school it was bomb threats. At least two or three a year
- bluGill 11mo agoThat happened at my mom's school (back in the 1960s) until the school put all the kids on a bus and brought them to the local armory gym and made all student sit quietly (no talking) on the floor until the end of the school day. Once the bomb threat wasn't a way to get out of school on a nice day there was never another one. I'm not sure how to apply that to this situation, but it is one every school should think about when students try things.
- aleph_minus_one 11mo ago> Once the bomb threat wasn't a way to get out of school on a nice day there was never another one. If I think of my school time, I would believe even the fact that a bomb threat would be an annoyance to teachers would a be sufficient reason (of course, in the schools of the country where I live there were other methods than bomb threats to be an annoyance to teachers).
- altairprime 11mo agoAn annoyance you can’t brag about to anyone, however.
- Zigurd 11mo agoThat depends on who you think needs to learn a lesson about buying an AI from Temu.
- m4ck_ 11mo agoOn a long enough timeline, without any changes, this garbage is going to get a kid murdered by overzealous LEOs (or teachers in places where they want the carrying.)
- avs733 11mo ago“School resource officers” shoot kids (and themselves) with terrifying frequency, the kids typically unarmed and often autistic/neurodivergent. (The worlds laziest lit review) https://letmegooglethat.com/?q=school+resource+office+shoots+unarmed+child https://letmegooglethat.com/?q=school+resource+office+shoots...
- interloxia 11mo ago[dead]
- gosub100 11mo agoOr the next shooter will disguise their gun to look like a musical instrument. Edit: I can't find it now but there is a body cam video of a Colorado school shooting where this almost happened. It wasn't due to an AI mixup but the school police was armed and I believe fired at one of the bad guys, but as the local PD responded he almost got shot because of the confusion: "school shooting, respond, guy with a gun"
- bluGill 11mo agoThat has been a common trope in gangster movies for ages. A violin case can easially hide a gun (depending on the gun of course, but you get to select this). You can find craftsmen in your gang who can make a violin case that also has a gun so it can even be opened an inspected by someone who doesn't know how to find the secret gun compartment, and it isn't hard to teach someone to play some simple songs on a violin thus giving a reason to carry a violin (you don't need to be good, just be able to show you have another lesson next week). I don't know if this works as well in the real world as movies, but it is plausible.
- fooqux 11mo agoWell at least it wasn't a bag of chips this time. A clarinet is at least a step towards the correct shape. Sounds like the AI training is going well!
- GaryBluto 11mo agoAs much as I believe this is a story that needs to be told, why did the author of the article choose one of those annoying two-sentence millennial titles? Do they actually generate more publicity? A man saw a snappy two-sentence title. He was mildly annoyed.
- reaperducer 11mo agowhy did the author of the article choose one of those annoying two-sentence millennial titles? In big journalism organizations like this one, the writers don't come up with the headlines. There are other people for that.
- rwmj 11mo agoIt's not exactly the same situation, but this happened before AI. At least the clarinet owner didn't get shot & killed like this guy: https://en.wikipedia.org/wiki/Killing_of_Harry_Stanley https://en.wikipedia.org/wiki/Killing_of_Harry_Stanley On 22 September 1999, Stanley was returning home from the Alexandra Pub in South Hackney carrying, in a plastic bag, a table leg that had been repaired by his brother earlier that day. Someone had phoned the police to report "an Irishman with a gun wrapped in a bag".[2]
- ramon156 11mo ago> the prosecution evidence is insufficient to rebut the officers' assertion that they were acting in self defence Awesome, they got away with unlawfully killing a man.
- justin66 11mo agoIn protest at the suspensions, over 120 out of the 400 Metropolitan Police officers authorised to use firearms handed in their firearms authorisation cards, with Glen Smyth, a Police Federation spokesman saying, "The officers are very concerned that the tactics they are trained in, as a consequence of the verdict, are now in doubt."[10] The officers' suspensions were lifted shortly afterwards.[11] What a joke.
- Anonbrit 11mo agoMany, many people think the tactics they're trained in are exactly the problem - a few hours at most training in deescalation, little to no training in recognising or dealing with the mentally unwell, hundreds of hours of training in putting a bullet inn somebody
- Ajedi32 11mo agoThey got away with accidentally unlawfully killing a man while doing an inherently dangerous job by the book. You can blame the system for that certainly and propose changes, but I don't think it's fair to charge the officers with murder for acting in what they believed to be fully justified self defense in the normal course of their job.
- neom 11mo agohttps://archive.is/WbOWl https://archive.is/WbOWl
- IAmBroom 11mo agoLots of jokes made, but in reality this mistake could have been made by a real human watching a video feed. The student was intentionally mimicking a weapon with a long, tubular object. It's not like he was just walking down the hall with it in his hands, and suddenly SWAT TEAM! AI is often awful, but I'm giving it a pass on this one.
- bluGill 11mo agoI can't give a pass - kids do things like this often enough that you can't call the police on every incident. Sure it might be stupid for kids to do, but they do it, and since it isn't a real gun it is harmless fun until someone overreacts. The real issue here is fatigue: the more you overreact the less your seriously you are taken when something really does happen. Once you can make a mistake and be okay, but you soon you become "the little boy who cried wolf", and people don't move to safety when there is a real issue!
- mcphage 11mo ago> The student was intentionally mimicking a weapon with a long, tubular object. That's... pretty much every student with any long, tubular object. Of which there are many.
- IAmBroom 11mo agoThe article notes the kid was (1) wearing a military costume and (2) then using his clarinet to simulate a weapon. I don't know about the AI's decisions, but he would have caught the eye of a casual human observer. My point is: it was a bit more involved than just a 5yo pointing his finger and shouting "POW! I got you!".
- mcphage 11mo agoA bit more. But not much.
- BobaFloutist 11mo agoDo they have a way to uncall the alarm? Presumably between the time the kid mimicked a weapon and the police arrive, the kid stopped mimicking a weapon and it became clear that it was a clarinet?
- rascul 11mo ago> “We don’t think we made an error, nor does the school,” Alaimo said. “That was better to dispatch [police] than not dispatch.” Of course the co-founder of the company that made the error would say that.
- catlikesshrimp 11mo agoAnother false positive of a system working as intended. https://news.ycombinator.com/item?id=45684934 https://news.ycombinator.com/item?id=45684934 (the doritos gun)
- WhyOhWhyQ 11mo agoIs the gun situation hopeless in America? As far as I can tell, we just need to reduce the quantity of guns. Somehow 20% of the country takes that idea as an existential threat, even though not doing so is a material threat to real people. If it's actually a mental health crisis, then how do we solve that one? Especially with what the Republicans are doing right now?
- blibble 11mo agothe guns don't even seem to work for their supposed primary purpose uninvited and unwanted federal troops are roaming around cities against the wishes of state governors so seems the US has the worst of both worlds: unqualified morons owning assault weapons, plus the tyranny
- deleted 11mo ago[deleted]
- deleted 11mo ago[deleted]
- miltonlost 11mo agoBecause you have Republicmas idiotically saying "Guns don't kill people; people do" and then don't do anything about the people or the guns. Conservative gun owners simply accept that people must die by guns in order for them to feel safe and make us all more vulnerable because of their irrational fears. They're babies with weapons.
- stephenhuey 11mo agoThat's simplistic. Actually, to better understand the situation you must follow the money. Many 2nd Amendment supporters are reasonable, but unfortunately, over decades their casual support has been utilized by lobbyists whose goals do not necessarily align with many supporters. The challenge is to communicate that message in a way that reaches everyone.
- another_twist 11mo agoIts weird how object detection models are "AI" now. These models and their weird errors have been around for quite a while. The issue is vendors claiming that there is no chance of errors. Ideally you would have a 2 eyes system such that if AI has a tolerable false positive rate and have a human review. But of course, you cant fire people with AI so why would we do the sane thing. And of course there are policy wonks who would make gun ownership a human rights issue even though its fundamentally unsafe to have such free gun ownership.
- Ghoelian 11mo agoAccording to the article, they did have a human verify the images before sending the alert. Apparently they and the school still think they made the right call.
- mcv 11mo agoAI has also been around for quite a while. LLMs are hardly the first instance of AI we've seen, just the one that's suddenly getting all the hype. But yes, people trust it too much.
- M95D 11mo agoDon't these cops receive an image/video of the event in addition to the AI description?? Who tf. designed the system?
- snapplebobapple 11mo agoMaybe its not wrong? Clarinets have harmed me far more than guns ever have...