6 ms·
Pro-tip: don't write the summary at all until you need it for evidence. Store the call audio at 24Kb/s Opus - that's 180KB per minute. After a year or whatever,
by dsr_ 1y ago
Pro-tip: don't write the summary at all until you need it for evidence. Store the call audio at 24Kb/s Opus - that's 180KB per minute. After a year or whatever, delete the oldest audio.
There, I've saved you more millions.
- lotsofpulp 1y agoThe summaries can help automate performance evaluation. If the employee disputes it, I imagine they pull up the audio to confirm.
- Imustaskforhelp 1y agothe amount of false positives coming from wrongful AI summaries plus having to pull up the audio to confirm is so much more hassle than not using AI and evaluating on some different metric at the first place. Seriously not kidding but the more I read these comments, the more I become horrified realizing wtf,The only reason I can think of integrating AI is because you wish to integrate AI. Nothing wrong with that, But unless proven otherwise through some benchmarks there is no way to justify AI. So its like an experiment, they use AI and if it works/ saves time, great If not, then time to roll it. But we do need to think about experiments logically and the way I am approaching it, its maybe good considering what customer service is now but man that's such a low standard that as customers we shouldn't really stand it. Call centres need to improve period. AI can't fix it. Its like man, we can do anything to save some $ for the shareholders. Only to then "invest" it proudly into AI so that they can say they have integrated AI and so they can have their valuations increased since VC's / stock market reacts differently to the sticker known as AI man.. so saying that you use AI, should be a negative indicator instead of a positive one in the market and the whole bubble is gonna come crashing down when people realize it. It physically hurts me now thinking about it once again. This loop of making humans bad for money, using that money for inferior product, using that inferior product only because you want AI sticker, because shareholders want valuation increase and the company is willing to do this all because they feel/ are rewarded for this by people who will buy anything AI related thinking its gold or maybe that more people will buy it from them at an even higher evaluation because AI sticker and so on.. Almost sounds like a pyramid.
- anoojb 1y agoAlso entrenches plausible deniability and makes legal contests way more cumbersome for plantiffs to resolve.
- deleted 1y ago[deleted]
- paulddraper 1y agoThis works unless you want to automate something with the transcripts, stats, feedback.
- Spivak 1y agoWhy wouldn't it, once you actually have that project you have the raw audio to generate the transcripts. Only spend the money at the last second when you know you need it. Edit: Tell me more how preemptively spending five figures to transcribe and summarize calls in case you might want to do some "data engineering" on it later is a sound business decision. What if the model is cheaper down the road? YAGNI.
- kenjackson 1y agoThis is the bread and butter of call centers and the companies that use them. The transcripts and summaries are used from everything from product improvement to agent assessment. This data is used continuously. Its not like they use this transcript for the one rare time someone sues because they claim an agent lied. That rarely happens.
- thfuran 1y agoA company that could save millions by not having staff write up their own call notes almost surely is already doing that.
- Spivak 1y agoAnd yet the topic of conversation is a company that did just that. The AI is just the smoke and mirrors that pushed the business to do it. Staff aren't writing their own call notes anymore. The LLM summary, almost by definition, isn't adding any additional signal to the call audio. If your data engineering pipeline works by processing LLM generated notes then it must work equally well processing the call transcript—they're the exact same data. AI finally got the business to admit that nothing of value was added by call notes and have dropped that work completely. The final step is just dropping the useless use of LLM. Just the audio transcript is way cheaper and can use existing technology.
- alooPotato 1y agoyou want to be able to search over summaries so you need to generate them right away
- krainboltgreene 1y agoPro-tip: You won't ever do that.
- alooPotato 1y agowe do
- ninininino 1y agoAdvanced organizations (think not startups, but companies that have had years of decades of profit in the public market) might have solved all the low-hanging fruit problems and have staff doing things like automated quality audits (search summaries for swearing, abusive language, etc).
- morkalork 1y agoAnd you could save a bunch of money by replacing the staff that do that with LLMs!
- krainboltgreene 1y agoI've worked at both. It is extremely rare that anyone ever does it.
- ch4s3 1y agoI would imagine OP is probably mining service call summaries to find common service issues, or at least that's what I would do.
- krainboltgreene 1y agoThat's what everyone says they'll do and then it never gets touched again.
- smohare 1y ago[dead]
- andix 1y agoImagine a follow-up call of a customer. They are referring to earlier calls and the call center agents needs to check what it was about. So they can skim/read the transcripts while talking to the customer. I guess it's really hard to listen to transcripts while you're on the phone.
- dsr_ 1y agoThat would be awesome! But in fact, customer call centers tend not to be able to even know that you called in yesterday, three days ago and last week. This is why email-ticketing call centers are vastly superior.
- ssharp 1y agoI've always guessed that they are able to tell when you called/what you called about, but they simply don't give that level of information to their frontline folks.
- Imustaskforhelp 1y agoIt might be because its in their interests to do so. It is our problem that needs fixing, so we can just wait untill either they redirect us to the right person with the right knowledge who might be one of the higher ups in the call centers. Or we just quit the call. Either way, it doesn't matter to the company. Plus points that they don't have to teach the frontline customer service more details too and it could be easier for them to onboard new people / fire old employees. Also they would have to pay less if they require very low specifications. man I remember the is 0.001 cent = 0.001 $ video /meme of verizon https://www.youtube.com/watch?v=nUpZg-Ua5ao https://www.youtube.com/watch?v=nUpZg-Ua5ao
- Jolter 1y agoPerhaps doing this suggested auto-summarizing would be what finally solves that problem?
- josefx 1y ago
- doorhammer 1y agoSentiment analysis, nuanced categorization by issue, detecting new issues, tracking trends, etc, are the bread and butter of any data team at a f500 call center. I'm not going to say every project born out of that data makes good business sense (big enough companies have fluff everywhere), but ime anyway, projects grounded to that kind of data are typically some of the most straight-forward to concretely tie to a dollar value outcome.
- la_fayette 1y agoYes that sound like important and useful use cases. However, these are solved by boring old school ML models since years...
- williamdclt 1y agoI think what they're saying is that you need the summaries to do these things
- esafak 1y agoIt's easier and simpler to use an LLM service than to maintain those ad hoc models. Many replaced their old NLP pipelines with LLMs.
- prashantsengar 1y agoThe place I work at, we replaced our old NLP pipelines with LLMs because they are easier to maintain and reach the same level of accuracy with much less work. We are not running a call centre ourselves but we are a SaaS offering the services for call centre data analysis.
- doorhammer 1y agoSo, I wouldn't be surprised if someone in charge of a QA/ops department chose LLMs over similarly effective existing ML models in part because the AI hype is hitting so hard right now. Two things _would_ surprise me, though: - That they'd integrate it into any meaningful process without having done actual analysis of the LLM based perf vs their existing tech - That they'd integrate the LLM into a core process their department is judged on knowing it was substantially worse when they could find a less impactful place to sneak it in I'm not saying those are impossible realities. I've certainly known call center senior management to make more hairbrained decisions than that, but barring more insight I personally default to assuming OP isn't among the hairbrained.
- FirmwareBurner 1y ago>Store the call audio at 24Kb/s Opus - that's 180KB per minute Why OPUS though? There's dedicated audio codecs in the VoiP/telecom industry that are specifically designed for the best size/quality for voice call encoding.
- pipo234 1y agoOpus is one of those codecs. Older codecs like g711 have better latency and steady bitrate, but they compress terribly. (Essentially just bandwidth and amplitude remapping). Opus is great for a lot of things and realtime speech over sip or webrtc is just one.
- deleted 1y ago[deleted]
- andrepd 1y agoOpus pretty much blows all those codecs out of the water, in every conceivable metric. It's actually pretty impressive that a media codec is able to universally exceed (or match) every previous one in every axis. Still, it's based on ideas from those earlier codecs of course :)
- deleted 1y ago[deleted]
- sillyfluke 1y agoYou also will have saved them all the cost of the AI summaries that are incorrect as well. The parent states: >Not only are the summaries better than those produced by our human agents... Now, since they have not mentioned what it took to actually verify that the AI summaries were in fact better than the human agents, I'm sceptical they did the necessary due dillengence. Why do I think this? Because I have actually tried to do such a verification. In order to verify that the AI summary is actually correct you have to engage in the incredibly tedious task of listening to original recording literally second by second and make sure that what is said does not conflict with the AI summary in question. Not only did the AI summary fail at this test, it failed in the first recording I tested. The AI summary stated that "Feature x was going to be in Release 3, not 4" whereas the in the recording it is stated that the feature will be in Release 4 not 3, literally the opposite of what the AI said. I'm sorry but the fact that the AI summary is nicely formatted and has not missed a major topic of conversation means fuck all if the details that are are discussed are spectacularly wrong from a decision tracking perspective, as in literally the opposite of what is stated. And I know "why" the Ai summary fucked up, because in that instance the topic of conversation was about how there was some confusion about which release that feature was going to be in, that's why the issue was a major item of the meeting agenda in the first place. Predicably, the AI failed to follow the convoluted discussion and "came to" the opposite conclusion. In short, no fucking thanks.
- roywiggins 1y agoIn the context of call centers in particular I actually can believe that a moderately inaccurate AI model could be better on average than harried humans writing a summary after the call. Could a human do better carefully working off a recording, absolutely, but that's not what needs to be compared against. It just has to be as good as a call center worker with 3-5 minutes working off their own memory of the call, not as good as the ground truth of the call. It's probably going to make weirder mistakes when it makes them though.
- trenchpilgrim 1y agoEspecially humans whose jobs are performance-graded on how quickly they can start talking to the next customer.