4 ms·

The craziest and most confusing behavior of Siri for me is: Sometimes you can ask a question and watch it be perfectly transcribed in real time, but then recei

The craziest and most confusing behavior of Siri for me is: Sometimes you can ask a question and watch it be perfectly transcribed in real time, but then receive a nonsensical answer from it. Ask the exact same question immediately after on the same device, transcribed exactly the same way, and get the correct answer. Where does such unpredictability come from? How can Siri transcribe the words correctly but fail to deliver the right answer?

objclxt 6y ago
> Where does such unpredictability come from? How can Siri transcribe the words correctly but fail to deliver the right answer? Voice assistants generally use both the text transcription and a bunch of contextual metadata as input. That metadata could include things like what's currently visible on the screen, your location, your recent queries, etc. So even though the underlying algorithms powering the assistant may be deterministic, the input data between two seemingly identical queries could vary quite a bit. For instance, Siri almost certainly has context around the previous questions you've asked. It would be reasonable to assume that if an assistant received two identical questions back-to-back the initial answer was wrong. In that scenario, the assistant might decide to use the a different answer (perhaps one that had a lower ranking) in an attempt to get it right.
- bgentry 6y ago
  I looked up the last screenshot I had of this. What I asked was "In 2 days remind me to call FRIEND_NAME", and Siri created a reminder that just said "call". Transcribed perfectly, wrong content in the reminder. I tried it again right after, and the reminder said "call FRIEND_NAME". I don't think there was any previous conversational context or anything like that. Hard to fathom how that could happen.
  - MereInterest 6y ago
    Maybe some overly aggressive A/B testing?

objclxt 6y ago

> Where does such unpredictability come from? How can Siri transcribe the words correctly but fail to deliver the right answer? Voice assistants generally use both the text transcription and a bunch of contextual metadata as input. That metadata could include things like what's currently visible on the screen, your location, your recent queries, etc. So even though the underlying algorithms powering the assistant may be deterministic, the input data between two seemingly identical queries could vary quite a bit. For instance, Siri almost certainly has context around the previous questions you've asked. It would be reasonable to assume that if an assistant received two identical questions back-to-back the initial answer was wrong. In that scenario, the assistant might decide to use the a different answer (perhaps one that had a lower ranking) in an attempt to get it right.

bgentry 6y ago
I looked up the last screenshot I had of this. What I asked was "In 2 days remind me to call FRIEND_NAME", and Siri created a reminder that just said "call". Transcribed perfectly, wrong content in the reminder. I tried it again right after, and the reminder said "call FRIEND_NAME". I don't think there was any previous conversational context or anything like that. Hard to fathom how that could happen.
- MereInterest 6y ago
  Maybe some overly aggressive A/B testing?