4 ms·

In the not too distant future we may see integrations with always-on recording devices (yes, I know, shudder) transcribing our every conversation and interactio

In the not too distant future we may see integrations with always-on recording devices (yes, I know, shudder) transcribing our every conversation and interaction and incorporating the text in place of the current custom-corpus style addenda to LLMs to give a truly personal and social skew to the current capabilities in the form of automatically-compiled memories to draw on.

panosfilianos 4y ago
I'm not too sure Siri/ Google Assistant doesn't do this already, but to serve us ads.
- schrodinger 4y ago
  If Siri or Google were doing this, it would have been whistleblown by someone by now. As far I as understand, Siri works with a very simple "hey siri" detector that then fires up a more advanced system that verifies "is this the phone owner asking the question" before even trying to answer. I'm confident privacy-sensitive engineers would notice and flag any misuse;
- dbish 4y ago
  That would also be crazy expensive and hard to do well. They struggle with current speech reco that’s relatively simple, and can’t do this more complex always listening thing at high accuracy and identifying relevant topics worth serving an ad on even if they wanted to and it wasn’t illegal. This is always the thing people would say for Alexa and Facebook too. The reality is people see patterns where there aren’t any or forget they searched for something that they also talked about and that’s what actually drove the specific ad they saw.
  - jononor 4y ago
    A high-end phone is quite capable of doing automatic speech recognition continuously, as well as NLP topic analysis. The last years voice activity detection has moved down into the microphone itself, to enable ultra low power always-listening functionality. It then triggers further processing of the potentially-containing-speech audio. Modern SoC have dedicated microcontroller/microprocessor cores that can do further audio analysis, without involving the main cores or the OS. Typically deciding if something is speech or not. Today this is usually doing Keyword Spotting (hey Alexa etc). These are expected to get access to neural accelerators chips, which will further improve power efficiency and eventually having sufficient memory and computer to run speech recognition. So the technological barriers are falling one by one.
    - dbish 4y ago
      I worked on Alexa (and Cortana before that). I’m aware of the current tech. The tech barriers for doing this at high accuracy cheaply are still very much there.
      - jononor 4y ago
        Which barriers do you considered to be the most problematic (and why)? I think that using todays CPU/GPU on a modern phone, one would be able to run a model capable of a Word Error Rate of under 10%. I mean, it would take up maybe 1GB of RAM. And battery life would be impacted, but still a quite usable phone? And that seems like it would give workable quality data for some topic / user modelling? I am assuming one would limit triggers to cases where the person speaking is within say 2 meters of the phone, which seems ok limitation for a phone - ulike for a home device like Alexa).
- xputer 4y ago
  They're not. A breach of trust at that level would kill the product instantly.
  - LesZedCB 4y ago
    Call me jaded but I don't believe that anymore. They might lose 20%. Maybe that's enough to kill but I honestly believe people would just start rolling with it
- dragonwriter 4y ago
  > I’m not too sure Siri/ Google Assistant doesn’t do this already, but to serve us ads. If it did, traffic analysis would probably have revealed it.
- itake 4y ago
  I talked to an Amazon Echo engineer about how the sound recording works. They said there is just enough hardware on the device to understand "hello Alexa" and then everything else is piped to the cloud. Currently, ML models are too resource intensive ($$) for always on-recording.
seydor 4y ago
To me, the value of a local-LLM is that it can hold my life's notes and i d talk to it as if it was my alter ego until old age. One could say, it's the kind of "soul" that outlasts us
- LesZedCB 4y ago
  You know what's funny, that episode of black mirror about that I thought was so unbelievable when I saw it
  - mclightning 4y ago
    holy sh*t. that's so true! that could definitely be possible.
    - LesZedCB 4y ago
      besides the synthetic body, we have the text interaction, the text-to-speech in a persons voice, and avatar generation/deep fakes. almost the entirety of that episode is available today, which i didn't believe was even ten years away when i saw it. referring to s2e1: Be Right Back it really asks great questions about image/reality too
      - mclightning 4y ago
        Imagine training a GPT on your own whatsapp/fb/instagram/linked/emails conversations: all the conversations, posts. A huge part of our life is already happening online, and the conversations with it. It is not too much work to simply take that data and retrain GPT.
        LesZedCB 4y ago
        i initially tried to download a bunch of my reddit comments and try to get it to write "in my style" but i think i need to actually go through the fine tuning process to do that well.
  - 4y ago
ilaksh 4y ago
Look at David Shapiro's project on GitHub, not Raven but the other one that is more fleshed out. He already does the summarization of dialogue and retrieval of relevant info using the OpenAI APIs I believe. You could combine that with the Chrome web speech or speech-to-text API which can stay on continuously. You would need to modify it a bit to know about third party conversations and your phone would run out of battery. But you could technically make the code changes in a day or two I think.

panosfilianos 4y ago

I'm not too sure Siri/ Google Assistant doesn't do this already, but to serve us ads.

schrodinger 4y ago
If Siri or Google were doing this, it would have been whistleblown by someone by now. As far I as understand, Siri works with a very simple "hey siri" detector that then fires up a more advanced system that verifies "is this the phone owner asking the question" before even trying to answer. I'm confident privacy-sensitive engineers would notice and flag any misuse;
dbish 4y ago
That would also be crazy expensive and hard to do well. They struggle with current speech reco that’s relatively simple, and can’t do this more complex always listening thing at high accuracy and identifying relevant topics worth serving an ad on even if they wanted to and it wasn’t illegal. This is always the thing people would say for Alexa and Facebook too. The reality is people see patterns where there aren’t any or forget they searched for something that they also talked about and that’s what actually drove the specific ad they saw.
- jononor 4y ago
  A high-end phone is quite capable of doing automatic speech recognition continuously, as well as NLP topic analysis. The last years voice activity detection has moved down into the microphone itself, to enable ultra low power always-listening functionality. It then triggers further processing of the potentially-containing-speech audio. Modern SoC have dedicated microcontroller/microprocessor cores that can do further audio analysis, without involving the main cores or the OS. Typically deciding if something is speech or not. Today this is usually doing Keyword Spotting (hey Alexa etc). These are expected to get access to neural accelerators chips, which will further improve power efficiency and eventually having sufficient memory and computer to run speech recognition. So the technological barriers are falling one by one.
  - dbish 4y ago
    I worked on Alexa (and Cortana before that). I’m aware of the current tech. The tech barriers for doing this at high accuracy cheaply are still very much there.
    - jononor 4y ago
      Which barriers do you considered to be the most problematic (and why)? I think that using todays CPU/GPU on a modern phone, one would be able to run a model capable of a Word Error Rate of under 10%. I mean, it would take up maybe 1GB of RAM. And battery life would be impacted, but still a quite usable phone? And that seems like it would give workable quality data for some topic / user modelling? I am assuming one would limit triggers to cases where the person speaking is within say 2 meters of the phone, which seems ok limitation for a phone - ulike for a home device like Alexa).
xputer 4y ago
They're not. A breach of trust at that level would kill the product instantly.
- LesZedCB 4y ago
  Call me jaded but I don't believe that anymore. They might lose 20%. Maybe that's enough to kill but I honestly believe people would just start rolling with it
dragonwriter 4y ago
> I’m not too sure Siri/ Google Assistant doesn’t do this already, but to serve us ads. If it did, traffic analysis would probably have revealed it.
itake 4y ago
I talked to an Amazon Echo engineer about how the sound recording works. They said there is just enough hardware on the device to understand "hello Alexa" and then everything else is piped to the cloud. Currently, ML models are too resource intensive ($$) for always on-recording.

seydor 4y ago

To me, the value of a local-LLM is that it can hold my life's notes and i d talk to it as if it was my alter ego until old age. One could say, it's the kind of "soul" that outlasts us

LesZedCB 4y ago
You know what's funny, that episode of black mirror about that I thought was so unbelievable when I saw it
- mclightning 4y ago
  holy sh*t. that's so true! that could definitely be possible.
  - LesZedCB 4y ago
    besides the synthetic body, we have the text interaction, the text-to-speech in a persons voice, and avatar generation/deep fakes. almost the entirety of that episode is available today, which i didn't believe was even ten years away when i saw it. referring to s2e1: Be Right Back it really asks great questions about image/reality too
    - mclightning 4y ago
      Imagine training a GPT on your own whatsapp/fb/instagram/linked/emails conversations: all the conversations, posts. A huge part of our life is already happening online, and the conversations with it. It is not too much work to simply take that data and retrain GPT.
      - LesZedCB 4y ago
        i initially tried to download a bunch of my reddit comments and try to get it to write "in my style" but i think i need to actually go through the fine tuning process to do that well.
- 4y ago

ilaksh 4y ago

Look at David Shapiro's project on GitHub, not Raven but the other one that is more fleshed out. He already does the summarization of dialogue and retrieval of relevant info using the OpenAI APIs I believe. You could combine that with the Chrome web speech or speech-to-text API which can stay on continuously. You would need to modify it a bit to know about third party conversations and your phone would run out of battery. But you could technically make the code changes in a day or two I think.