6 ms·
It was already better. I edit a podcast and have > a decade of pro audio editing experience in the film industry, and I was already using a commercial AI transc
by anigbrowl 4y ago
It was already better. I edit a podcast and have > a decade of pro audio editing experience in the film industry, and I was already using a commercial AI transcription service to render the content to text and sometimes edit it as such (outputting edited audio).
Existing (and affordable) offerings are so good that they can cope with shitty recordings off a phone speaker and maintain ~97% accuracy over hour-long conversations. I'm sure it's been an absolute godsend for law enforcement other people who need to gather poor-quality audio at scale, though much less great for the targets of repressive authority.
Having this fully open is a big deal though - now that level of transcription ability can be wrapped as an audio plugin and just used wherever. Given the parallel advances in resynthesis and understanding idiomatic speech, in a year or two I probably won't need to cut out all those uuh like um y'know by hand ever again, and every recording can be given an noise reduction bath and come out sounding like it was recorded in a room full of soft furniture.
- thfuran 4y ago>~97% accuracy over hour-long conversations. I'm sure it's been an absolute godsend for law enforcement 97% accuracy means roughly three or four errors per minute of speech. That seems potentially extremely problematic for something like law enforcement use where decisions with significant impact on people's day and/or life might be made on the basis of "evidence".
- golem14 4y agoOne would think that the few crucial bits of information gleaned are listened to manually, and the machine translation is not the only thing the judge or a jury sees.
- thfuran 4y agoYou have absolutely ruined someone's day way before they're sitting in front of a jury.
- formerly_proven 4y agoStuff like that is a very good tell that someone has zero experience with law enforcement.
- hadlock 4y agoMicrosoft announced their voice transcription technology a couple years ago and were also touting ~97-98% accuracy which was actually better than human transcription error rates. The errors are usually in part people garbling their own speech, or they move their head while talking and the microphone misses a syllable. Anything in that error bar would probably fall under "reasonable doubt"
- kyriakos 4y agoIf its anything like Microsoft teams transcription I doubt the 97%+ accuracy.
- gs17 4y agoYeah, I tried to use automated transcription for a research project and we had to do it all manually because the few errors (I would say it did pretty well given our recording quality) were often dropping words like "not", which changed the whole meaning of a sentence! It was a useful assistance during transcription, but I really hope they would verify it was correct before arresting anyone based on it.
- anigbrowl 4y agoNo it isn't. That just means 2-3% of your content needs to be double-checked by a person at the audio level, saving huge amounts of time - equally true of human transcription, in which individual words are often [UNINTELLIGEBLE]. Would you want to review this fully before going into court, absolutely - because you'd want to play the recording to a jury for emotional impact. Can you rely on it when you want to quickly read through hours of conversation and make decisions about whether to invest further resources (which might just mean another hour of listening back to the original audio)? Also absolutely. Bear in mind that a lot of these errors have little to no semantic impact, being on the same level as typos or misspellings in a written communication. Bear in mind too that if law enforcement (honest or not) is so interested in you that they're willing to record your conversations, your day is already ruined, you just don't know it yet. The change here is one of scale rather than quality.
- dotancohen 4y agoSometimes even human will disagree about what was said in a recording - I had this happen recently. I heard a specific sentence, the other person heard the exact opposite. I cannot say who was right, even after listening to the recording several times on headphones and speakers I'm as certain of my interpretation as was the other party.
- wging 4y agoDoesn't it mean 100% of your content needs to be double-checked? You can't easily identify which 2-3% of your content has errors. I'm aware that errors are more likely when the model is less confident of its predictions, but that shouldn't be enough. (edit for clarification: errors are not always something like "[UNINTELLIGIBLE]", where the system knows it doesn't know; they can also be misrecognitions that the system believes in with high confidence.)
- u8 4y agoI had to do a lot of manual transcription in Journalism school. Using a tool like Descript saved HOURS of my life. Generally it was 80% accurate, but going over an two-hour-long recording again at 3x speed while reading over the transcript, fixing errors from memory or pausing took a five hour job down to 30-40 minutes. Either way, somebody is going to have to listen to the recording. This just removes a layer of grunt work.
- selfmodruntime 4y agoI've worked with similar technology in the law enforcement space and the software is never used to make decisions. You can make out critical timestamps in conversations and a law enforcement officer will always manually confirm the software's assessments.
- JohnFen 4y agoGiven that law enforcement has made similar claims about technology use in the past that turned out to be false, I have no faith in this claim.
- selfmodruntime 4y agoIn all honesty, this is the correct mindset to have. I have limited expertise in this topic, and you should be aware that other law enforcement agencies probably do not handle this the same way.
- CTDOCodebases 4y agoI imagine a certain percentage of a given population is on a voice call at any one time. 1. Set up a computer with voice recognition software that flags certain patterns. 2. Connect computer to voice call communication network. 3. Configure computer to switch between calls every x number of seconds. Think of it like a system to generate leads for law enforcement that can be integrated with other systems to produce the best quality leads.
- NaturalPhallacy 4y agoThis is called "a fishing expedition" and is wildly unconstitutional in the US. >The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.
- jjoonathan 4y agoYes, it is wildly unconstitutional, but in practice don't the courts endorse the asinine "it's not a search unless we find something" argument from the NSA? Power always just finds a way to rationalize what it wants to do.
- kurisufag 4y agosee: Operation PRISM
- CTDOCodebases 4y agoAre you sure about that? [0] Besides I wasn't talking about the USA when I said this. I was remembering a conversation I once had with a person who worked as a technician in a telephone exchange. [0] - https://en.wikipedia.org/wiki/Jewel_v._NSA https://en.wikipedia.org/wiki/Jewel_v._NSA
- Thorentis 4y agoNot really. Imagine that they do simple keyword matching on the text. Anything that's missed (part of the 97%) the criminals get away with. Anything that matches in the 3% is then checked by a human (by listening to the audio at that time stamp). So you only need to manually check the 3%, and even then only if something you're interested in is found.
- solarmist 4y agoAny recommendations for particular services?
- anigbrowl 4y agoI use a service called sonix.ai. It's paid but I think they have a free tier or trial period, and it's not very expensive. I'm excited about this new OpenAI thing because I'd rather do it on my own hardware than send it to the cloud, but this company has earned its commercial success.
- solarmist 4y agoThat is an exciting possibility. Being able to fix bad setups and missed takes automagically. It’s always been possible, just expensive and time consuming for moderate improvements.
- deegles 4y agoThere's already software that can imitate a person's voice, so we have all the pieces already to do speech-to-text, clean up with GPT-3, and back to text-to-speech in the original person's voice. Maybe with a style transfer to keep the person's inflections etc the same?
- Karuma 4y agoI think something similar already exists. See this, for example: https://koe.ai/recast/ https://koe.ai/recast/ Although I don't know if they're using anything similar to what you suggest. Very cool idea, anyway!
- adamgordonbell 4y agoI've not found that to be the case. For technical content, I use Rev.com and provide a glossary and real humans do the transcript. Other AI transcription services get lots wrong because the context often matters. Words like "TCP/IP" or "FAT disk format" or "Big Endian" I've never found AI so far to handle well. I'm interested to test out whisper on this one. https://corecursive.com/063-apple-2001/ https://corecursive.com/063-apple-2001/
- biomcgary 4y agoSince you work on podcasts, do any open source transcription tools currently identity the speaker in the output? This would be particularly helpful for interviews.
- nico 4y agoNot sure about open source, but in general, automated transcription systems need a separate track for each different speaker. So for example, for a phone call with one person on each end, you need two separate channels (recording systems usually split them left/right on one stereo file).
- nonoesp 4y agoI'm not sure if you've tried Descript, but their ML-based "Studio Sound" filter makes bad audio sound like it was recorded and edited nicely.