5 ms·

Plus they are super inaccurate. Gemini gets one of its three bullet subtly or very majorly wrong almost every time. Just a few weeks ago Gemini said we’re rolli

Plus they are super inaccurate. Gemini gets one of its three bullet subtly or very majorly wrong almost every time. Just a few weeks ago Gemini said we’re rolling out our payment setup in Russia. You know the place where we have 20+ sanctions packages on? We were talking about France in the meeting.

pjc50 5mo ago
Given how financial services can impose silent inexplicable lifetime bans for using the wrong words in the "what is this transaction for" field, I'm wondering at what point the AI automatically reports people for sanctions violation based on its mishearing.
operation_moose 5mo ago
We've found they're surprisingly good if everyone on the call is using a decent headset. The problems start when using conference room audio or someone is on their laptop mic. If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. We just went through a round of 100+ (non-sensitive) VoC interviews and they really cut down the workload of compiling all of the feedback. If the audio was a little shaky though, we pretty much had to throw away the transcripts and do them from scratch like we used to.
- user_7832 5mo ago
  > If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. Imo this is the single biggest flaw of LLMs. They're great at a lot of things, but knowing when they're wrong (or don't have enough information to actually work on) is a critical flaw. IMO there's nothing structural about why they shouldn't be able to spot this and correct themselves - I suspect it's a training issue. But presumably bots that infer context/fill in the dots rank better on what people like... at the cost of accuracy.
  - r_lee 5mo ago
    I don't think it's a training issue, it's simply that there's no inherent "I don't know" in the transformer architecture unless it's really like something completely unknown, otherwise the nearest neighbor will be chosen and that will be whatever sounds similar or is relevant, even if it might cause a problem
    - aspenmartin 5mo ago
      Not inherent in transformer architecture, we do try to ingrain a sense of uncertainty but it’s difficult not only technically but also philosophically/culturally. How confident do you want the model to be in its answer to “why did Rome fall”? Lots of tools in our toolbelts to do better uncertainty calibration but it trades off against other capabilities and actually can be rather frustrating to interact with in agentic contexts since it will constantly need input from you or otherwise be indecisive and overly cautious. It’s not technically a limitation of transformer architecture but it is more challenging to deal with than other architectures/statistical paradigms. Like you can maintain a belief state and generate conditional on this and train to ensure belief state is stable and performant. But evals reward guessing at this point, and it’s very very hard to evaluate the calibration in these open ended contexts. But we’re slowly getting there, just not nearly as fast as other capabilities.
      - fluoridation 5mo ago
        >How confident do you want the model to be in its answer to “why did Rome fall”? The confidence level can be any, as long as it's reported accurately often enough. "This is my conjecture, but", "I'm not completely sure, but", and "most historians agree that" are all perfectly valid ways to start a sentence, which LLMs never use. They state mathematical truth, general consensus, hotly debated stances, and total fabrication, with the exact same assertiveness.
        Terr_ 5mo ago
        > > Like you can maintain a belief state and generate conditional on this and train to ensure belief state is stable and performant > ways to start a sentence, which LLMs never use A huge part of the problem is we've invented a document-generator setup which exploits human cognitive illusions, and even the smartest person can't constantly override the instinctive brain-bits that "sees" fictional entities and infers the intent of a mind. That makes it weirdly-hard to discuss the setup's shortfalls or how to improve it. To wit: The machine does not possess any kind of confidence about how Rome fell. Or even whether Rome fell. It has "confidence" about which word/token will next in a "typical" document given the document-so-far has text like "How did Rome fall?" It may be straightforward to burn money training the system so that its "typical" story never has a computer-character with confident words about Roman history, but that's just papering over the underlying problem. TLDR: We can't fix the thinking-habits or beliefs inside the mind of an entity that doesn't actually exist. Changing the story-generator to contain a tee-totaling Dracula dispensing life-advice doesn't mean we "cured the disease of vampirism."
- steveBK123 5mo ago
  > The problems start when using conference room audio RTO problems
- inetknght 5mo ago
  Their quality for different language accents also significantly varies. Got a team with Indian, Chinese, Texan, British, and Australian? Your A.I.-powered translation tool is going to get 80% of your conversation wrong.
- FireBeyond 5mo ago
  My workflow uses krisp.ai for taking a transcript, and then I have a dedicated project in Claude. I feed it the transcript and ask for it to give me a summary in a specific format I define, with good front matter, etc., and it needs to spit that out in a way that I auto-import into Obsidian. But key in my prompt is asking 1) for it to flag any low confidence or context-nonsensical statements in the transcript, with the timestamp, so then I can listen to the original audio and either clarify, correct, or say "I couldn't understand that either, here's my best guess and mark it low confidence", then 2) which I see as critical: Claude also is told to create a "context" document that it maintains based on my answers, so it starts to gather ASR things like "transcript commonly hears A B and C as variants of name X", who is who, internal product and project names and context info on them. 3) Claude is told specifically to read this prior to summarizing the transcript, and to consult it as it is doing so, and to ask me on anything it's not confident on. What is then starting to get quite powerful for me is moving from full text search of my meeting notes in Obsidian (I'm a PM in a lot of meetings), but I can point Cowork to the Obsidian notes folder (because they're all Markdown) and start doing rich "querying" of it. "When did [stakeholder] first mention [feature] as a release blocker?" and it can point to the meeting. My system works well, and I've done a bit to fine tune the automation and friction reduction, and it's a bit easier to manage because I'm not generally creating summaries for broader consumption but as my second brain (I have a separate prompt that utilizes some of that "knowledge" to build those). One thing I've found helpful with this is moving the summarization itself into something with "context/memory". Krisp is capable of generating summaries but can't/doesn't review prior transcripts. Its role is just "give me the transcript as you heard it".
  - Kushalzen 5mo ago
    For the "when did X first mention Y" queries, does it surface things you'd forgotten about that turned out to matter? Or mostly just confirm what you half-remembered? Trying to figure out whether the value of the loop is rediscovery or just precise lookup.
    - FireBeyond 5mo ago
      I think it's both, for me. I struggle with recall, so it helps me remember. I'm also using it to push to Todoist, etc. I need the memory jog, and then it helps me confirm things as they come back to me, to ensure I remember correctly. (ADHD, and across many projects as a PM).
- deleted 5mo ago
  [deleted]
- insane_dreamer 5mo ago
  Recent example: - the person said 8 to 10 - LLM transcribed as 18 Granted, the person had a foreign accent and didn't enunciate very clearly. But I knew they meant 8-10 if for no other reason than 18 didn't make sense given the context. But the AI isn't smart enough, and then 18 goes into the record.
- CGMthrowaway 5mo ago
  > headset Half- vs. full duplex. Headphones is all you really need, though of course a directional mic and/or one closer to your mouth will yield a clearer audio recording as well.
- cyanydeez 5mo ago
  "This technology works as long as you're not a pleb"
- laughing_man 5mo ago
  >If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. Isn't that what people do?
  - Lammy 5mo ago
    For in-person conversations to keep the conversation flowing, sure, but any good transcription will say [unintelligible] when the scribe couldn't tell despite being able to listen over it again and again. Nixon tapes for example: https://kagi.com/search?q=site%3Anixonlibrary.gov+%22unintelligible%22&r=us&sh=10bBvH0fgFIYk3qhfvLO2Q https://kagi.com/search?q=site%3Anixonlibrary.gov+%22unintel...
- PeterStuer 5mo ago
  I have done many transcriptions of messy meeting recordings with thic euro-english accents, and a local Whisper large handled them near perfect.
elzbardico 5mo ago
Verbatim transcriptions are usually very good. Because even the ocasional "can/can't" replacement is usually obvious within the context of the full conversation. But the summarization feature is where the most ridiculous errors and omissions happens.
- therealdrag0 5mo ago
  Ok zoom the default summary template is often lacking and incoherent, but switching it to the lengthy one “Discussion” works great. I think the default only works for single topic meetings where is rare.
breppp 5mo ago
That's presumably great for legal exposure because it increases deniability
buzer 5mo ago
I wonder what kind of GDPR implications that has given the requirements around the accuracy of personal data.

pjc50 5mo ago

Given how financial services can impose silent inexplicable lifetime bans for using the wrong words in the "what is this transaction for" field, I'm wondering at what point the AI automatically reports people for sanctions violation based on its mishearing.

operation_moose 5mo ago

We've found they're surprisingly good if everyone on the call is using a decent headset. The problems start when using conference room audio or someone is on their laptop mic. If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. We just went through a round of 100+ (non-sensitive) VoC interviews and they really cut down the workload of compiling all of the feedback. If the audio was a little shaky though, we pretty much had to throw away the transcripts and do them from scratch like we used to.

user_7832 5mo ago
> If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. Imo this is the single biggest flaw of LLMs. They're great at a lot of things, but knowing when they're wrong (or don't have enough information to actually work on) is a critical flaw. IMO there's nothing structural about why they shouldn't be able to spot this and correct themselves - I suspect it's a training issue. But presumably bots that infer context/fill in the dots rank better on what people like... at the cost of accuracy.
- r_lee 5mo ago
  I don't think it's a training issue, it's simply that there's no inherent "I don't know" in the transformer architecture unless it's really like something completely unknown, otherwise the nearest neighbor will be chosen and that will be whatever sounds similar or is relevant, even if it might cause a problem
  - aspenmartin 5mo ago
    Not inherent in transformer architecture, we do try to ingrain a sense of uncertainty but it’s difficult not only technically but also philosophically/culturally. How confident do you want the model to be in its answer to “why did Rome fall”? Lots of tools in our toolbelts to do better uncertainty calibration but it trades off against other capabilities and actually can be rather frustrating to interact with in agentic contexts since it will constantly need input from you or otherwise be indecisive and overly cautious. It’s not technically a limitation of transformer architecture but it is more challenging to deal with than other architectures/statistical paradigms. Like you can maintain a belief state and generate conditional on this and train to ensure belief state is stable and performant. But evals reward guessing at this point, and it’s very very hard to evaluate the calibration in these open ended contexts. But we’re slowly getting there, just not nearly as fast as other capabilities.
    - fluoridation 5mo ago
      >How confident do you want the model to be in its answer to “why did Rome fall”? The confidence level can be any, as long as it's reported accurately often enough. "This is my conjecture, but", "I'm not completely sure, but", and "most historians agree that" are all perfectly valid ways to start a sentence, which LLMs never use. They state mathematical truth, general consensus, hotly debated stances, and total fabrication, with the exact same assertiveness.
      - Terr_ 5mo ago
        > > Like you can maintain a belief state and generate conditional on this and train to ensure belief state is stable and performant > ways to start a sentence, which LLMs never use A huge part of the problem is we've invented a document-generator setup which exploits human cognitive illusions, and even the smartest person can't constantly override the instinctive brain-bits that "sees" fictional entities and infers the intent of a mind. That makes it weirdly-hard to discuss the setup's shortfalls or how to improve it. To wit: The machine does not possess any kind of confidence about how Rome fell. Or even whether Rome fell. It has "confidence" about which word/token will next in a "typical" document given the document-so-far has text like "How did Rome fall?" It may be straightforward to burn money training the system so that its "typical" story never has a computer-character with confident words about Roman history, but that's just papering over the underlying problem. TLDR: We can't fix the thinking-habits or beliefs inside the mind of an entity that doesn't actually exist. Changing the story-generator to contain a tee-totaling Dracula dispensing life-advice doesn't mean we "cured the disease of vampirism."
steveBK123 5mo ago
> The problems start when using conference room audio RTO problems
inetknght 5mo ago
Their quality for different language accents also significantly varies. Got a team with Indian, Chinese, Texan, British, and Australian? Your A.I.-powered translation tool is going to get 80% of your conversation wrong.
FireBeyond 5mo ago
My workflow uses krisp.ai for taking a transcript, and then I have a dedicated project in Claude. I feed it the transcript and ask for it to give me a summary in a specific format I define, with good front matter, etc., and it needs to spit that out in a way that I auto-import into Obsidian. But key in my prompt is asking 1) for it to flag any low confidence or context-nonsensical statements in the transcript, with the timestamp, so then I can listen to the original audio and either clarify, correct, or say "I couldn't understand that either, here's my best guess and mark it low confidence", then 2) which I see as critical: Claude also is told to create a "context" document that it maintains based on my answers, so it starts to gather ASR things like "transcript commonly hears A B and C as variants of name X", who is who, internal product and project names and context info on them. 3) Claude is told specifically to read this prior to summarizing the transcript, and to consult it as it is doing so, and to ask me on anything it's not confident on. What is then starting to get quite powerful for me is moving from full text search of my meeting notes in Obsidian (I'm a PM in a lot of meetings), but I can point Cowork to the Obsidian notes folder (because they're all Markdown) and start doing rich "querying" of it. "When did [stakeholder] first mention [feature] as a release blocker?" and it can point to the meeting. My system works well, and I've done a bit to fine tune the automation and friction reduction, and it's a bit easier to manage because I'm not generally creating summaries for broader consumption but as my second brain (I have a separate prompt that utilizes some of that "knowledge" to build those). One thing I've found helpful with this is moving the summarization itself into something with "context/memory". Krisp is capable of generating summaries but can't/doesn't review prior transcripts. Its role is just "give me the transcript as you heard it".
- Kushalzen 5mo ago
  For the "when did X first mention Y" queries, does it surface things you'd forgotten about that turned out to matter? Or mostly just confirm what you half-remembered? Trying to figure out whether the value of the loop is rediscovery or just precise lookup.
  - FireBeyond 5mo ago
    I think it's both, for me. I struggle with recall, so it helps me remember. I'm also using it to push to Todoist, etc. I need the memory jog, and then it helps me confirm things as they come back to me, to ensure I remember correctly. (ADHD, and across many projects as a PM).
deleted 5mo ago
[deleted]
insane_dreamer 5mo ago
Recent example: - the person said 8 to 10 - LLM transcribed as 18 Granted, the person had a foreign accent and didn't enunciate very clearly. But I knew they meant 8-10 if for no other reason than 18 didn't make sense given the context. But the AI isn't smart enough, and then 18 goes into the record.
CGMthrowaway 5mo ago
> headset Half- vs. full duplex. Headphones is all you really need, though of course a directional mic and/or one closer to your mouth will yield a clearer audio recording as well.
cyanydeez 5mo ago
"This technology works as long as you're not a pleb"
laughing_man 5mo ago
>If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. Isn't that what people do?
- Lammy 5mo ago
  For in-person conversations to keep the conversation flowing, sure, but any good transcription will say [unintelligible] when the scribe couldn't tell despite being able to listen over it again and again. Nixon tapes for example: https://kagi.com/search?q=site%3Anixonlibrary.gov+%22unintelligible%22&r=us&sh=10bBvH0fgFIYk3qhfvLO2Q https://kagi.com/search?q=site%3Anixonlibrary.gov+%22unintel...
PeterStuer 5mo ago
I have done many transcriptions of messy meeting recordings with thic euro-english accents, and a local Whisper large handled them near perfect.

elzbardico 5mo ago

Verbatim transcriptions are usually very good. Because even the ocasional "can/can't" replacement is usually obvious within the context of the full conversation. But the summarization feature is where the most ridiculous errors and omissions happens.

therealdrag0 5mo ago
Ok zoom the default summary template is often lacking and incoherent, but switching it to the lengthy one “Discussion” works great. I think the default only works for single topic meetings where is rare.

breppp 5mo ago

That's presumably great for legal exposure because it increases deniability

buzer 5mo ago

I wonder what kind of GDPR implications that has given the requirements around the accuracy of personal data.