9 ms·
Ex-athletic director arrested for framing principal with AI-generated voice
- cbsmith 2y agoOh boy. This is the beginning, not the end.
- xhkkffbf 2y agoDon't let anyone say that the teachers aren't keeping up with technology. Certainly not the gym teachers.
- FrustratedMonky 2y agoSo it begins. If any joe blow ex-athletic director can do it, think about how wide spread this could become.
- ceejayoz 2y ago> Darien was being investigated as of December in a theft investigation that had been initiated by Eiswert. Police say Darien had authorized a $1,916 payment to the school’s junior varsity basketball coach, who was also his roommate, under the pretense that he was an assistant girls soccer coach. He was not, school officials said. Eiswert determined that Darien had submitted the payment to the school payroll system, bypassing proper procedures. Darien had been notified of the investigation, police said. > Police say the clip was received by three teachers the night before it went viral. The first was Darien; a third said she received the email and then got a call from Darien and teacher Shaena Ravenell telling her to check her email. Ravenell told police that she had forwarded the email to a student’s cell phone, “who she knew would rapidly spread the message around various social media outlets and throughout the school,” and also sent it to the media and the NAACP, police said. So, in this case, clear motive, a prominent figure, and a suspicious chain of custody gave the cops a reason to dig in a bit, but not everyone's gonna be that dumb about it. The swatters who do this to random people are gonna have a field day with this.
- TrainedMonkey 2y agoConcur OPSEC is not great, there is an easy to follow insertion vector. Imagine instead if he got a burner phone, made a new TikTok account and posted this tagging a few students.
- setgree 2y agoThat's true, but also the person allegedly used a school computer to research his crime: > Police wrote in charging documents that Darien had accessed the school’s network on multiple occasions in December and January searching for OpenAI tools, and used “Large Language Models” that practice “deep learning, which involves pulling in vast amounts of data from various sources on the internet, can recognize text inputted by the user, and produce conversational results.” They also connected Darien to an email account that had distributed the recording. One of our best defenses against crime is that criminals are often kind of dumb. Tyler Cowen makes the same point this morning regarding airport security: "a lot of criminals are simply some mix of stupid and incompetent or poor on execution" (https://marginalrevolution.com/marginalrevolution/2024/04/why-do-i-prefer-current-airport-procedures.html https://marginalrevolution.com/marginalrevolution/2024/04/wh...).
- sjducb 2y agoThe ones we catch are.
- nojvek 2y agoI wonder how many millionaires there are who have done a white collar crime once, got their money, cleaned it using some business and now living off it never to do it again. Take that secret to the grave. Not just white collar, but many people go missing never to be found. So I assume there are likely competent murders who understand enough of biology and chemistry to not leave a forensic trail and not attract too much attention.
- graemep 2y agoRichard Branson was caught evading tax - his family saved him from prosecution by paying it off. https://slate.com/business/2014/05/richard-branson-tax-fraud-how-a-youthful-indiscretion-helped-create-a-billionaire.html https://slate.com/business/2014/05/richard-branson-tax-fraud... If Harold Shipman had stopped with the first 100 murders or so he would probably never have been caught: https://www.britannica.com/biography/Harold-Shipman https://www.britannica.com/biography/Harold-Shipman
- surfpel 2y ago> Experts in detecting audio and video fakes told The Banner in March that there was overwhelming evidence the voice is AI-generated. They noted its flat tone, unusually clean background sounds and lack of consistent breathing sounds or pauses as hallmarks of AI. They also ran the audio through several different AI-detection techniques, which consistently concluded it was a fake, though they could not be 100% sure. All of these problems will be resolved. In which case most people either won’t raise the question of authenticity or won’t trust audio recordings to begin with. Elections at all levels are at risk along with accountability more generally.
- londons_explore 2y agoAmazingly, most people do seem to trust a simple screenshot of a conversation, even though it has been easy to fake those for 20+ years. A screenshot is considered far more trustworthy by non-tech people than "I spoke to Fred and would you believe it, he told me X and Y were doing Z behind the bike sheds!!!"
- IshKebab 2y agoYeah because it takes significantly more effort to fake a screenshot than to just make stuff up. That's entirely logical.
- andrewstuart2 2y agoI wouldn't call a quick search for "fake text message" which returns dozens of generators significantly more effort. Harder to do on the spot, maybe, but the effort is negligible.
- 9question1 2y agoYou underestimate the laziness of most humans. To be clear, screenshots are still not trustworthy. But the "negligible" relatively more effort could matter in practice. Cryptography sometimes seem to rely on a stronger version this too. With enough computing power, you can brute force a lot. Some authentication seems just expensive enough that only a nation-state actor would have enough resources to break it, and then rests on the assumption that the small subset of people who could put in enough effort won't care enough to do anything worth being concerned about. This is also why, say, people lock their doors even if they have windows.
- endisneigh 2y agoImagine if he was cleverer - was in an office, feigning a conversation with them, with kids who would be “witnesses”. Monitor the principal to the alibi isn’t solid or somehow supports it. Wild times.
- heavyarms 2y agoThere are lots of valid use cases for speech synthesis and text-to-speech technology, and there are like 1 or 2 valid/legal use cases for voice cloning that I can think of. Ignoring the moral and ethical questions, why would anybody devote time and resources building a company around a very niche solution... one in which your customer churn rate is partially dependent on users not ending up in prison. edit: typo
- itqwertz 2y agoWe are rapidly advancing towards the logical conclusion of the post-modernist question, “What is truth?” Mobs will always remain dumb and quick to anger, so this will not be an isolated case.
- barbariangrunge 2y agoI feel like the post modernist stance isn’t to question what constitutes truth, as the ancient Greeks and epistemology nerds attempted, but rather to posit: there is no truth. I’m not a fan of that stance, except for-fun in certain art forms; similar to how violence is fun in art but not in real life
- causal 2y agoUp next: Some enterprising startup founder's "AI-voice detector" landing people in prison because they can't keep up with ML advances but authorities trust them anyway.
- dotnet00 2y agoThere have already been cases like professors failing entire classes for plagiarism without any critical thought because an AI claimed their work was AI generated. Those were reversed because they were particularly egregious, but I can completely see individual students getting falsely accused of using AI and treated as guilty until proven innocent, going totally under the radar because single incidents don't get much public scrutiny.
- causal 2y agoConsidering the public's persistent faith in lie detectors for humans, I don't hold a lot of hope we will be wiser about lie detectors for computers.
- ryandrake 2y agoIt's going to get worse before it (unlikely) will get better. I predict within 10 years, we'll have at least one jurisdiction experimenting with AI-generated evidence, to help prosecutors get convictions. Prosecutor will push a button and computer will spit out a sworn statement that the defendant is guilty, this will be admissible, and juries will convict based on that evidence alone. Drug dogs who "hit" on command are already used as probable cause generators. Breathalyzers are basically magic boxes that produce "evidence" of a crime. It's inevitable that we're going to keep using AI and computers to automate convictions.
- causal 2y agoThis seemed implausible to me at first, then I remembered we've already had people submitting ChatGPT-hallucinated cases in court, so maybe not :(
- beaeglebeachh 2y agoWait till the kids figure out how to do it. Principal at my school tried to have me expelled, by lying saying I was inciting violence. Soon it will be easy for kids to turn the tables on these tyrannical administrators in their insular fiefdoms.
- PhasmaFelis 2y agoI'm sorry your principal was a dick, but it's weird how you're acting like "teenagers now can fraudulently incriminate anyone they want" is somehow a good thing.
- beaeglebeachh 2y agoI think it could be a good thing. When the tyrants have to face their own behavior, they'll be forced to raise the evidentiary standards high enough their own lies won't work.
- jbullock35 2y ago> When the tyrants have to face their own behavior, they'll be forced to raise the evidentiary standards high enough their own lies won't work. I don't think that tyrants work this way –- at least not in America's educational system. They're very happy to employ double standards.
- rurp 2y agoThat's not really how actual tyranny works. There is no requirement for equal treatment.
- splitstud 2y ago[dead]
- dvaun 2y agoWhile your situation is unfortunate, you can’t paint a broad stroke and assume all administrators are “tyrannical”. Most of these people are simply trying to perform their jobs.
- hdlothia 2y agoI don't think we're ready for the consequences of this technology. My parents are immigrants and my first thought was this might get someone killed or locked up in the old country. These free generators need to include some kind of audio watermark or key to indicate they are ai imitations. At least raise the barrier for this kind of action to being able to run your own llm or something.
- thraway3837 2y agoI'm really not sure why you're getting downvoted. It's almost as though HN readers are fully on the AI bandwagon and can't let anything bad be said about it. I assume this is the same crowd that also scoffs at any regulations in tech.
- educasean 2y agoYou were right the first time: You really don't understand why HN readers are downvoting the OP comment. Yet that didn't appear to have stopped you from spinning up this AI-bandwagon-riding, regulation-hating strawman to further entrench your hatred towards.
- hdlothia 2y agoIf you're a downvoter, why? I would love to hear why you disagree with my comment.
- aingisni_del 2y agoThat’s not what a straw man is (it’s two words btw). This is a known and often repeated trend in tech: make something under the guise of disruption, with no regard for safety or regulations. Complain that those things are a hindrance to progress and then spend billions and destroy lives permanently to eventually fix the problems when the regulators issue an ultimatum that professionals warned you about.
- freedomben 2y ago
- educasean 2y agoAs these synthetic voice generations become less and less obviously detectable, I fear two things will happen. 1. The obvious: There will be a lot of fake speeches floating around that spout fake news and hate-filled views. 2. The less obvious: The prevalence of hatred and sensationalized rumors will embolden those who find themselves agreeing with the extreme views seemingly endorsed by some authority figure, and will add their support and authentic voices to the mix. Perhaps this is just an extension of what has already been happening with the internet. We will find ourselves more fragmented and divided than ever, filling our chambers with literal echos of synthetic voices.
- _bkyr 2y ago3. A distrust in everything leading to more sway in public opinion. 4. Deniability in everthing that wasactually said. I think it's either Elon Musk or Donald Trump whose lawyers have argued in court/public that you can't be certain they actually said it.
- FromOmelas 2y ago3. A reversion to in-person interaction for anything important (exams, certifications, payments, loan applications, ...) Society benefited from a productivity gain by moving everything online, in a (relatively) high-trust environment. That is now becoming more expensive (due to higher % of frauds), or even infeasible. So, a drag on economic growth for years to come.
- markhaslam 2y agoHere is the audio clip in question: https://www.instagram.com/reel/C2NEEDrMo8_/ https://www.instagram.com/reel/C2NEEDrMo8_/
- inputError 2y ago[dead]
- SuperNinKenDo 2y agoDefinitely some hallmarks of AI if you know what you're looking for. But I have to admit, that is more convincing than I would have expected. A bit more manual intervention to change volume and balance and I might find itnsignificantly more difficult to be certain.
- zachmu 2y agoThe comments are full of people buying it completely
- shombaboor 2y agoWhat is the best argument why this technology exists or should exist? It's fun?Live translation is the only one I can think of, and even then the benefits of the persons 'real' voice are scant.
- grugagag 2y agoScooping up from the middle class, one class of professions at a time. But this is not just the like how the internet disrupted ‘everything’ and moved things online, this is going to take all the liveleyhoods and suck them dry filling up the technobros coffers until there’s nothing left. But there are upsides too, the potential for good applications is quite broad.
- IncreasePosts 2y ago"it's cool"?
- breakpointalpha 2y agoIf I want to do a voiceover of a ten minute video, I can type out the transcript and produce a flawless one-take audio track. This saves hours of time finding a quiet place to record, saying the lines, having a good mic, doing post-recording cleanup to remove coughs or passing airplane noise, multiple takes because I goofed a word up, etc. I don’t know if this is the best use of the tech we’ve found yet, but it’s already a huge time saver.
- nickthegreek 2y agoI create internal training videos ant my job and that’s the use case as well. Not many people want to have their voice used and it’s easier and way faster for me to have an ai produce a voice than to get someone to do it for me.
- MiguelHudnandez 2y agoIt's already actively in use to edit and clean up podcast recordings. This use case is basically identical to the malicious case, it's all about who's doing it and what their intent is. If it's fixing a word you stumbled on, or replacing an inaccurate quote with an accurate one, that's fine, but other things are problematic. Someone who is losing the ability to speak might want to have this tech so they can still have somewhat normal phone calls with their loved ones. I think the potential for abuse is pretty high with this tech but it's foolish to pretend we can keep it from being used.
- jsheard 2y agoMy condolences to all the podcasters and YouTubers who are probably going to get bombarded with extortion attempts after people use their hundreds of hours of public clean voice recordings to make a perfectly convincing voice clone, down to the tiniest quirks of pronunciation, cadence and so on. Who would have thought that would become an opsec risk.
- DontchaKnowit 2y agoHonestly? The writings been on the wall for at least a decade if not longer. This shouldnt really be surprising to anyone with a large corpus of recorded material online.
- jsheard 2y agoMaybe to people who were closely following the research, but I very much doubt it was on most peoples radar until maybe a year or so ago when ElevenLabs became available to the public and it started being used in viral memes of Trump and friends.
- DontchaKnowit 2y agoYeah I mean fair enough. Im no AI enthusiast and I knew about it a long time ago, however, I also spend a ton of time on the internet, read computer related news etc, so I spose your right. But I remember watching northernlion like 8 years ago and thinking "you could probably spoof this guys voice really easy with a nueral net"
- JdeBP 2y agoFar longer than a decade. We've known that this has been coming since the days of Max Headroom. The interesting thing is that in the Max Headroom days we thought that it would come about because of scheming corporate interests, not because of movie audiences that had pressed for ever better special effects, and fake Tom Cruises, all of these years.
- okhugres 2y ago> Who would have thought that would become an opsec risk. I did. So I’m sure many others too. They just likely met the same opposition that I did. Without defensive strategies today that allow those at risk to continue their risky behavior unencumbered then any warnings of tomorrow get drowned out by the demands of today.
- wpollock 2y ago>Eiswert determined that Darien had submitted the payment to the school payroll system, bypassing proper procedures. How did he allegedly bypass payroll procedures? An athletics director should not have passwords to the school payroll system. I wonder if social engineering was used? In any case, their security procedures need an audit!
- deleted 2y ago[deleted]
- sandspar 2y agoThe future will be stupider than we can even imagine.
- waldrews 2y agoI think the lesson here is obvious. Who among us hasn't been terrorized by PhysEd teachers, forced to run laps or climb thing or something against our will? Athletic directors must be even worse. Ban athletic directors!
- theogravity 2y agoWonder what happened to the victim, Eiswert, in the end? The damage was done, and I don't think it can ever be fully rolled back, unfortunately.
- rpjt 2y agoI wonder about this too. He's already guilty in the court of public opinion.
- Simulacra 2y agoThis will haunt that principal for years to come, the damage has already been done. Like false claims of rape, the internet will continue to keep the lie alive.