5 ms·

Whoa, whoa, whoa, on occasion, if the planets align just right, I can also get Siri to set a reminder (and at least half the time Siri gets it 80% right). LLM

Whoa, whoa, whoa, on occasion, if the planets align just right, I can also get Siri to set a reminder (and at least half the time Siri gets it 80% right). LLM Siri cannot come fast enough.

nottorp 2y ago
LLMs are there for generating/hallucinating text, not for understanding text. For natural language processing you need a different kind of neural network don't you?
- drodgers 2y ago
  What are you talking about? We've invented the Universal Translator already.
- joshstrange 2y ago
  I’m confident that LLM’s will not have hallucination problems in the type of requests that I send to Siri. I don’t ask Siri for facts (just like I don’t ask LLM’s for facts). As long as it can correctly, understand what and when I ask to be reminded about something, that would be a huge improvement for me. That and being able to map “Bedroom Fan”/“Bedroom Fan Light” to “Bedroom Fan Lights” without having to specify aliases (and even then it hearing me wrong). I’ve see Home Assistant working with LLMs and it can understand groupings that I never explicitly defined which is very nice. I can say “Turn off all overhead lights” and it will find all my overhead lights and turn them off. Siri/Alexa can’t handle those tasks currently.
- zzbzq 2y ago
  It's the other way around. The model is impeccable at "understanding text." It's a gigantic mathematical spreadsheet that quantifies meaning. The model probably "understands" better than any human ever could. Running that backwards into producing new text is where it gets hand-wavy & it becomes unclear if the generative algorithms are really progressing on the same track that humans are on, or just some parallel track that diverges or even terminates early.
  - nottorp 2y ago
    I thought it quantifies the probability that a certain word (their output) follows a given word sequence (their training corpus and the prompt)?
    - ben_w 2y ago
      Only if you wildly oversimply to the level of being misleading. The precise mechanism LLMs use for reaching their probability distributions is why they are able to pass most undergraduate level exams, whereas the Markov chain projects I made 15-20 years ago were not. Even as an intermediary, word2vec had to build a space in which the concept of "gender" exists such that "man" -> "woman" ~= "king" -> "queen".
      - nottorp 2y ago
        > Only if you wildly oversimply to the level of being misleading. Maybe I'm asking for an explanation :) Since you seem to understand the mechanism, can you do a 3 line summary please?
        ben_w 2y ago
        3 lines? That's still going to be oversimplifed to the point of being wrong, but OK. Make a bunch of neural nets to recognise every concept, the same way you would make them to recognise numbers or letters in handwiting recognition. Glue them together with more neural nets. Put another on the end to turn concepts back into words. For a less wrong but still introductory summary that still glosses over stuff, about 1.5 hours of 3blue1brown videos, #4-#8 in this playlist: https://youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&si=0EcPhIBdgIuAJQ-r https://youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_...
        nottorp 2y ago
        Thanks! ... Oh interesting. And those concepts are hand picked or generated automatically somehow? > For a less wrong but still introductory summary that still glosses over stuff, about 1.5 hours of 3blue1brown videos Sorry, my religion forbids me from watching talking heads. I'll have to live with your summary for now. Until I run into someone who condensed those 1.5 hours in text that takes at most 30 min to read...
- jonplackett 2y ago
  They are AMAZING at understanding. I think even more reliably than generating. And with context, and back and forth. Try asking ChatGPT to remember some obscure film you can only remember very hazy details about - really random stuff - I bet it will identify it for you I a few tries. You can also totally miss spell words, use messed up grammar and it has no problems at all
nessex 2y ago
If you haven't tried OpenAI's advanced voice mode, it's a mind blowing version of exactly what things like Siri really ought to become with a little more development. If that's what you mean by LLM Siri, I totally agree. Being able to chat casually with low latency, correct yourself, switch languages mid-sentence, incorporate context throughout a back-and-forth conversation etc. turns talking to these kinds of systems from a painful chore into something that can actually add value.
- skydhash 2y ago
  But does it do actual stuff, like adding a meeting to your calendar, call people, or set a timer or a reminder?
smallmancontrov 2y ago
Siri already has LLM integration... ...that will grind your request to set email Vacation Mode through the world's worst speech-to-text, jam the text into Chat GPT, and spend the next three minutes reading you an uninterruptible 3 minute essay about violence.
therealcamino 2y ago
Over on Android it's the opposite situation. The voice interface to Google Assistant was very reliable for simple things like reminders and appointments, and even for general knowledge questions. It was part of why I didn't switch to an iPhone. Then Gemini came along, and that core functionality got a lot worse.
madeofpalk 2y ago
> LLM Siri cannot come fast enough. I don't think it's been demonstrated that Apple could make Siri better with an LLM.
jonplackett 2y ago
I tried this with the new ‘apple intelligence’ that I thought could see my screen. I had a birthday invite with clear date and time: so I asked it to add to my calendar. It just said, “add what?” repeatedly until finally deciding it needed to send it to chatGPT to help. Which it did, then just returned the text in the image without taking any action. Then I say “can you add the event now?” “Add what?” So I try copying the text from the image in photos and giving that to Siri to add as an event. Surly this can work?!? Nope. It’s just utterly pitiful.
- wrboyce 2y ago
  I think that feature has been delayed.

Whoa, whoa, whoa, on occasion, if the planets align just right, I can also get Siri to set a reminder (and at least half the time Siri gets it 80% right). LLM Siri cannot come fast enough.

nottorp 2y ago

LLMs are there for generating/hallucinating text, not for understanding text. For natural language processing you need a different kind of neural network don't you?

drodgers 2y ago
What are you talking about? We've invented the Universal Translator already.
joshstrange 2y ago
I’m confident that LLM’s will not have hallucination problems in the type of requests that I send to Siri. I don’t ask Siri for facts (just like I don’t ask LLM’s for facts). As long as it can correctly, understand what and when I ask to be reminded about something, that would be a huge improvement for me. That and being able to map “Bedroom Fan”/“Bedroom Fan Light” to “Bedroom Fan Lights” without having to specify aliases (and even then it hearing me wrong). I’ve see Home Assistant working with LLMs and it can understand groupings that I never explicitly defined which is very nice. I can say “Turn off all overhead lights” and it will find all my overhead lights and turn them off. Siri/Alexa can’t handle those tasks currently.
zzbzq 2y ago
It's the other way around. The model is impeccable at "understanding text." It's a gigantic mathematical spreadsheet that quantifies meaning. The model probably "understands" better than any human ever could. Running that backwards into producing new text is where it gets hand-wavy & it becomes unclear if the generative algorithms are really progressing on the same track that humans are on, or just some parallel track that diverges or even terminates early.
- nottorp 2y ago
  I thought it quantifies the probability that a certain word (their output) follows a given word sequence (their training corpus and the prompt)?
  - ben_w 2y ago
    Only if you wildly oversimply to the level of being misleading. The precise mechanism LLMs use for reaching their probability distributions is why they are able to pass most undergraduate level exams, whereas the Markov chain projects I made 15-20 years ago were not. Even as an intermediary, word2vec had to build a space in which the concept of "gender" exists such that "man" -> "woman" ~= "king" -> "queen".
    - nottorp 2y ago
      > Only if you wildly oversimply to the level of being misleading. Maybe I'm asking for an explanation :) Since you seem to understand the mechanism, can you do a 3 line summary please?
      - ben_w 2y ago
        3 lines? That's still going to be oversimplifed to the point of being wrong, but OK. Make a bunch of neural nets to recognise every concept, the same way you would make them to recognise numbers or letters in handwiting recognition. Glue them together with more neural nets. Put another on the end to turn concepts back into words. For a less wrong but still introductory summary that still glosses over stuff, about 1.5 hours of 3blue1brown videos, #4-#8 in this playlist: https://youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&si=0EcPhIBdgIuAJQ-r https://youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_...
        nottorp 2y ago
        Thanks! ... Oh interesting. And those concepts are hand picked or generated automatically somehow? > For a less wrong but still introductory summary that still glosses over stuff, about 1.5 hours of 3blue1brown videos Sorry, my religion forbids me from watching talking heads. I'll have to live with your summary for now. Until I run into someone who condensed those 1.5 hours in text that takes at most 30 min to read...
jonplackett 2y ago
They are AMAZING at understanding. I think even more reliably than generating. And with context, and back and forth. Try asking ChatGPT to remember some obscure film you can only remember very hazy details about - really random stuff - I bet it will identify it for you I a few tries. You can also totally miss spell words, use messed up grammar and it has no problems at all

nessex 2y ago

If you haven't tried OpenAI's advanced voice mode, it's a mind blowing version of exactly what things like Siri really ought to become with a little more development. If that's what you mean by LLM Siri, I totally agree. Being able to chat casually with low latency, correct yourself, switch languages mid-sentence, incorporate context throughout a back-and-forth conversation etc. turns talking to these kinds of systems from a painful chore into something that can actually add value.

skydhash 2y ago
But does it do actual stuff, like adding a meeting to your calendar, call people, or set a timer or a reminder?

smallmancontrov 2y ago

Siri already has LLM integration... ...that will grind your request to set email Vacation Mode through the world's worst speech-to-text, jam the text into Chat GPT, and spend the next three minutes reading you an uninterruptible 3 minute essay about violence.

therealcamino 2y ago

Over on Android it's the opposite situation. The voice interface to Google Assistant was very reliable for simple things like reminders and appointments, and even for general knowledge questions. It was part of why I didn't switch to an iPhone. Then Gemini came along, and that core functionality got a lot worse.

madeofpalk 2y ago

> LLM Siri cannot come fast enough. I don't think it's been demonstrated that Apple could make Siri better with an LLM.

jonplackett 2y ago

I tried this with the new ‘apple intelligence’ that I thought could see my screen. I had a birthday invite with clear date and time: so I asked it to add to my calendar. It just said, “add what?” repeatedly until finally deciding it needed to send it to chatGPT to help. Which it did, then just returned the text in the image without taking any action. Then I say “can you add the event now?” “Add what?” So I try copying the text from the image in photos and giving that to Siri to add as an event. Surly this can work?!? Nope. It’s just utterly pitiful.

wrboyce 2y ago
I think that feature has been delayed.