11 ms·

Leon: An open-source personal assistant

332 points by yvonnick 8y ago

cstuder 8y ago
The webpage seems to be missing a list of capabilities. If you're looking for it you need to infer it from here: https://github.com/leon-ai/leon/tree/master/packages https://github.com/leon-ai/leon/tree/master/packages
- blowski 8y ago
  Yeah, the whole page focuses on how it's built, not so much on problems it solves. From the link you posted, it seems to: * say "hello" * tell jokes * generate random numbers * download videos from a given YouTube URL * check status of a domain I can't see why any of those need AI.
  - throwaway2016a 8y ago
    > I can't see why any of those need AI. The AI piece in virtual assistants usually doesn't refer to the skills themselves but the process of matching what is spoken (as audio) to a skill and passing that skill the appropriate context. Without having to have a developer program in all the hundreds of thousands of ways someone can ask for the same thing. With that said, I'm not sure how well this assistant does that. None of these skills seem to use context at all. The sentence structure to map to a skill seems relatively limited or at least not challenged by the initial skills.
  - ilaksh 8y ago
    Here is the demo https://www.youtube.com/watch?v=p7GRGiicO1c https://www.youtube.com/watch?v=p7GRGiicO1c Doing speech-to-text and text-to-speech in an accurate and convincing way is extremely difficult and its cutting edge AI. A big part of what the Leon dev has done is created a way to plug in to different providers for TTS and STT. In my opinion this is quite useful because it takes care of all of the hard parts. So its a good starting point for creating your own personal assistant with the features that you think are useful. The thing that is tough though is that the open source stuff out there that you can install offline for free i.e. DeepSpeech and Flite or similar is pretty inferior to the cloud-based or proprietary stuff. However, having the provider framework is a good starting point for comparing different systems that can plug-in to fill that role. And hopefully with this packaging as a personal assistant ready for offline mode, it will bring more attention to those open source TTS and STT engines/voices/datasets that are out there and people will improve them.
masha_sb 8y ago
1. How is it different from the existing open source naturagl language parsers? 2. what problem does it solve? Have you tried to create an NLU platform which can be extended? Have you tried to create a personal asst., if yes, then what specific use-cases does it address?
GNi33 8y ago
I've been planning to set up something like this for myself for quite some time now. Can anyone tell me what makes this different from something like Snips (https://snips.ai https://snips.ai)?
- ocdtrekkie 8y ago
  I have a longstanding view that the best personal assistant is one you write for yourself: It can cover your needs and respond to how you'd prefer to interact with it. A lot of sophistication isn't needed because it doesn't have to try to meet a lowest common denominator user.
  - sosodev 8y ago
    Absolutely. I think hackability is the best part of open source.
- j88439h84 8y ago
  You might like project naomi.
  - lukifer 8y ago
    For the lazy: https://github.com/NaomiProject/Naomi https://github.com/NaomiProject/Naomi
- StavrosK 8y ago
  I really hope SNIPS manages to launch some hardware like the Echo at an affordable price point. Their current kit is too expensive but the Echo sucks and I much prefer writing my own intents on an open source system that runs on-device.
- ilaksh 8y ago
  Snips is totally Cloud based and proprietary as far as I can tell. Leon has offline capabilities and also the ability to switch between different cloud providers.
machbio 8y ago
Video Demo - https://www.youtube.com/watch?v=p7GRGiicO1c https://www.youtube.com/watch?v=p7GRGiicO1c
LawnboyMax 8y ago
I highly recommend Rasa Core (https://github.com/RasaHQ/rasa_core https://github.com/RasaHQ/rasa_core) if you are looking for an open-source virtual assistant. Very active and helpful community, and lots of input channels for integrating your assistant with messaging platforms (https://rasa.com/docs/core/connectors/ https://rasa.com/docs/core/connectors/). I am not affiliated with Rasa, just had a really good experience developing a few projects with it.
- perturbation 8y ago
  I'll +1 SNIPs or Rasa, they're both really nice. It looks like the NLU part of Leon is a logistic regression classifier (https://github.com/leon-ai/leon/blob/360d1020c4bd8bf1df3764645c635c64e0364702/server/src/core/nlu.js https://github.com/leon-ai/leon/blob/360d1020c4bd8bf1df37646...) so it's just doing intent detection, not any slot filling. Maybe someone can add calls to Rasa's HTTP API (https://rasa.com/docs/core/server/# https://rasa.com/docs/core/server/#) to integrate with Leon?
  - tenkabuto 8y ago
    How well do these compare to Mycroft[1]? [1]: https://mycroft.ai/ https://mycroft.ai/
    - perturbation 8y ago
      Haven't looked at MyCroft before. It looks like MyCroft exposes less of the nuts-and-bolts of modeling? I'm not sure where I would plug in a custom entity extraction or intent detection model, but I do see that it lets you add custom 'skills'.
      - llamataboot 8y ago
        Yes, this can all be handled with skills and the dictionaries that go along with skills. I don't even program much in python, but I've found it pretty simple to make add-ons for mycroft that have multiple entities for different intents.
kijin 8y ago
> He does stuff when you ask for it. Is the name a homage to the Luc Besson movie starring Jean Reno? If so, he'd better be able to kill a process (if not a real person) when the user asks for it ;)
- chris_wot 8y ago
  The video seems to allude to it.
inthewings 8y ago
NIH. Another open source assistant... Not like there's already 3 or 4 around. You can even play with it online https://chat.susi.ai/overview https://chat.susi.ai/overview
TaylorAlexander 8y ago
So, I get that most folks aren’t tuned in to this and I will sound unreasonable, but I’m utterly tired of robot-like things being given gender. Every time I read “he”, “him” when learning about a piece of software I gag a little. I know someone will tell me I am wrong. But I’m not asking or telling anyone to change. I just want to share what unnecessary gendering looks like to me. And for the record, all my robots are non-gendered (“Scout, Skittles, Rover”).
- lucideer 8y ago
  > I get that most folks aren’t tuned in to this and I will sound unreasonable > someone will tell me I am wrong I know "+1" style comments are generally dissuaded on HN, but just chiming in to say that not everyone thinks this sounds unreasonable. It's a topic well worth opening discussions on. > And for the record, all my robots are non-gendered (“Scout, Skittles, Rover”). All of my robots are named "Larry"[0] [0] https://bugs.gentoo.org/27727 https://bugs.gentoo.org/27727
- TeMPOraL 8y ago
  You can choose between "he", "she" and "it", at least in regular English, sans the XXI-century additions. "It" usually refers to things, so if you want to humanize your bot/robot - as many people tend to - you're stuck with picking a gender. At least in English you don't have nouns automatically selecting gender for you, like in other languages.
  - nicoburns 8y ago
    > You can choose between "he", "she" and "it", at least in regular English, sans the XXI-century additions. Or "they". People seem to think that singluar they is some kind of modern invention, but most people also use it all the time without thinking when talking about people that they don't know the gender.
    - _emacsomancer_ 8y ago
      Not to mention it's used by Shakespeare and the King James Bible.
JohnFen 8y ago
Oooh, this looks promising indeed!
joshstrange 8y ago
I got excited when I saw the core was JS (as writing TypeScript/JS is something I enjoy) then I saw that all modules are written in python... That doesn't seem to make a lot of sense to me, allowing for any language or a subset of languages makes sense but writing the core and plugin system in different languages.
- JohnFen 8y ago
  I was disappointed that this was implemented in JS and Python. If I decide to experiment with this, the first thing I'll do is port it to a (compiled) language that I prefer.
- ilaksh 8y ago
  I did not see the Python code. I do see a bunch of JavaScript and a Node.js package.json. Where was the python code?
squarefoot 8y ago
Unless I misread the docs, offline use would be restricted to text-only. Is that limitation temporary? I would surely have some uses for one of these things, but no way I'm putting a closed source or cloud connected one in my home, not even an open source one if it uses any external services I can't trust.
- ilaksh 8y ago
  I looked at the source and there are scripts for offline that install Mozilla DeepSpeech for speech-to-text and CMU Flite for text-to-speech. I put links in my other comment in this thread.
ilaksh 8y ago
I think the most interesting part of this is the offline capability. From looking at the source code, it looks like the text to speech is provided by http://www.festvox.org/flite/ http://www.festvox.org/flite/ and the speech to text uses https://github.com/mozilla/DeepSpeech https://github.com/mozilla/DeepSpeech. How do those compare to the major online providers?
- JohnFen 8y ago
  > I think the most interesting part of this is the offline capability. I agree -- that's the aspect that makes this potentially appealing to me.
bmn__ 8y ago
Does any of you know of a speech-to-text software that runs continuously, works on Linux and accepts a Pulseaudio input (i.e. microphone) rather than files and outputs text as a stream?
- skoocda 8y ago
  Kaldi has a couple open-source online LVCSR models which can definitely do live decoding. Though I'm not 100% sure if there is support for PulseAudio, you may need an auxiliary service to pipe it in. Easiest interface for it is via gstreamer: github.com/alumae/kaldi-gstreamer-server

332 points by yvonnick 8y ago

The webpage seems to be missing a list of capabilities. If you're looking for it you need to infer it from here: https://github.com/leon-ai/leon/tree/master/packages https://github.com/leon-ai/leon/tree/master/packages

blowski 8y ago
Yeah, the whole page focuses on how it's built, not so much on problems it solves. From the link you posted, it seems to: * say "hello" * tell jokes * generate random numbers * download videos from a given YouTube URL * check status of a domain I can't see why any of those need AI.
- throwaway2016a 8y ago
  > I can't see why any of those need AI. The AI piece in virtual assistants usually doesn't refer to the skills themselves but the process of matching what is spoken (as audio) to a skill and passing that skill the appropriate context. Without having to have a developer program in all the hundreds of thousands of ways someone can ask for the same thing. With that said, I'm not sure how well this assistant does that. None of these skills seem to use context at all. The sentence structure to map to a skill seems relatively limited or at least not challenged by the initial skills.
- ilaksh 8y ago
  Here is the demo https://www.youtube.com/watch?v=p7GRGiicO1c https://www.youtube.com/watch?v=p7GRGiicO1c Doing speech-to-text and text-to-speech in an accurate and convincing way is extremely difficult and its cutting edge AI. A big part of what the Leon dev has done is created a way to plug in to different providers for TTS and STT. In my opinion this is quite useful because it takes care of all of the hard parts. So its a good starting point for creating your own personal assistant with the features that you think are useful. The thing that is tough though is that the open source stuff out there that you can install offline for free i.e. DeepSpeech and Flite or similar is pretty inferior to the cloud-based or proprietary stuff. However, having the provider framework is a good starting point for comparing different systems that can plug-in to fill that role. And hopefully with this packaging as a personal assistant ready for offline mode, it will bring more attention to those open source TTS and STT engines/voices/datasets that are out there and people will improve them.

masha_sb 8y ago

1. How is it different from the existing open source naturagl language parsers? 2. what problem does it solve? Have you tried to create an NLU platform which can be extended? Have you tried to create a personal asst., if yes, then what specific use-cases does it address?

GNi33 8y ago

I've been planning to set up something like this for myself for quite some time now. Can anyone tell me what makes this different from something like Snips (https://snips.ai https://snips.ai)?

ocdtrekkie 8y ago
I have a longstanding view that the best personal assistant is one you write for yourself: It can cover your needs and respond to how you'd prefer to interact with it. A lot of sophistication isn't needed because it doesn't have to try to meet a lowest common denominator user.
- sosodev 8y ago
  Absolutely. I think hackability is the best part of open source.
j88439h84 8y ago
You might like project naomi.
- lukifer 8y ago
  For the lazy: https://github.com/NaomiProject/Naomi https://github.com/NaomiProject/Naomi
StavrosK 8y ago
I really hope SNIPS manages to launch some hardware like the Echo at an affordable price point. Their current kit is too expensive but the Echo sucks and I much prefer writing my own intents on an open source system that runs on-device.
ilaksh 8y ago
Snips is totally Cloud based and proprietary as far as I can tell. Leon has offline capabilities and also the ability to switch between different cloud providers.

machbio 8y ago

Video Demo - https://www.youtube.com/watch?v=p7GRGiicO1c https://www.youtube.com/watch?v=p7GRGiicO1c

LawnboyMax 8y ago

I highly recommend Rasa Core (https://github.com/RasaHQ/rasa_core https://github.com/RasaHQ/rasa_core) if you are looking for an open-source virtual assistant. Very active and helpful community, and lots of input channels for integrating your assistant with messaging platforms (https://rasa.com/docs/core/connectors/ https://rasa.com/docs/core/connectors/). I am not affiliated with Rasa, just had a really good experience developing a few projects with it.

perturbation 8y ago
I'll +1 SNIPs or Rasa, they're both really nice. It looks like the NLU part of Leon is a logistic regression classifier (https://github.com/leon-ai/leon/blob/360d1020c4bd8bf1df3764645c635c64e0364702/server/src/core/nlu.js https://github.com/leon-ai/leon/blob/360d1020c4bd8bf1df37646...) so it's just doing intent detection, not any slot filling. Maybe someone can add calls to Rasa's HTTP API (https://rasa.com/docs/core/server/# https://rasa.com/docs/core/server/#) to integrate with Leon?
- tenkabuto 8y ago
  How well do these compare to Mycroft[1]? [1]: https://mycroft.ai/ https://mycroft.ai/
  - perturbation 8y ago
    Haven't looked at MyCroft before. It looks like MyCroft exposes less of the nuts-and-bolts of modeling? I'm not sure where I would plug in a custom entity extraction or intent detection model, but I do see that it lets you add custom 'skills'.
    - llamataboot 8y ago
      Yes, this can all be handled with skills and the dictionaries that go along with skills. I don't even program much in python, but I've found it pretty simple to make add-ons for mycroft that have multiple entities for different intents.

kijin 8y ago

> He does stuff when you ask for it. Is the name a homage to the Luc Besson movie starring Jean Reno? If so, he'd better be able to kill a process (if not a real person) when the user asks for it ;)

chris_wot 8y ago
The video seems to allude to it.

inthewings 8y ago

NIH. Another open source assistant... Not like there's already 3 or 4 around. You can even play with it online https://chat.susi.ai/overview https://chat.susi.ai/overview

TaylorAlexander 8y ago

So, I get that most folks aren’t tuned in to this and I will sound unreasonable, but I’m utterly tired of robot-like things being given gender. Every time I read “he”, “him” when learning about a piece of software I gag a little. I know someone will tell me I am wrong. But I’m not asking or telling anyone to change. I just want to share what unnecessary gendering looks like to me. And for the record, all my robots are non-gendered (“Scout, Skittles, Rover”).

lucideer 8y ago
> I get that most folks aren’t tuned in to this and I will sound unreasonable > someone will tell me I am wrong I know "+1" style comments are generally dissuaded on HN, but just chiming in to say that not everyone thinks this sounds unreasonable. It's a topic well worth opening discussions on. > And for the record, all my robots are non-gendered (“Scout, Skittles, Rover”). All of my robots are named "Larry"[0] [0] https://bugs.gentoo.org/27727 https://bugs.gentoo.org/27727
TeMPOraL 8y ago
You can choose between "he", "she" and "it", at least in regular English, sans the XXI-century additions. "It" usually refers to things, so if you want to humanize your bot/robot - as many people tend to - you're stuck with picking a gender. At least in English you don't have nouns automatically selecting gender for you, like in other languages.
- nicoburns 8y ago
  > You can choose between "he", "she" and "it", at least in regular English, sans the XXI-century additions. Or "they". People seem to think that singluar they is some kind of modern invention, but most people also use it all the time without thinking when talking about people that they don't know the gender.
  - _emacsomancer_ 8y ago
    Not to mention it's used by Shakespeare and the King James Bible.

JohnFen 8y ago

Oooh, this looks promising indeed!

joshstrange 8y ago

I got excited when I saw the core was JS (as writing TypeScript/JS is something I enjoy) then I saw that all modules are written in python... That doesn't seem to make a lot of sense to me, allowing for any language or a subset of languages makes sense but writing the core and plugin system in different languages.

JohnFen 8y ago
I was disappointed that this was implemented in JS and Python. If I decide to experiment with this, the first thing I'll do is port it to a (compiled) language that I prefer.
ilaksh 8y ago
I did not see the Python code. I do see a bunch of JavaScript and a Node.js package.json. Where was the python code?

squarefoot 8y ago

Unless I misread the docs, offline use would be restricted to text-only. Is that limitation temporary? I would surely have some uses for one of these things, but no way I'm putting a closed source or cloud connected one in my home, not even an open source one if it uses any external services I can't trust.

ilaksh 8y ago
I looked at the source and there are scripts for offline that install Mozilla DeepSpeech for speech-to-text and CMU Flite for text-to-speech. I put links in my other comment in this thread.

ilaksh 8y ago

I think the most interesting part of this is the offline capability. From looking at the source code, it looks like the text to speech is provided by http://www.festvox.org/flite/ http://www.festvox.org/flite/ and the speech to text uses https://github.com/mozilla/DeepSpeech https://github.com/mozilla/DeepSpeech. How do those compare to the major online providers?

JohnFen 8y ago
> I think the most interesting part of this is the offline capability. I agree -- that's the aspect that makes this potentially appealing to me.

bmn__ 8y ago

Does any of you know of a speech-to-text software that runs continuously, works on Linux and accepts a Pulseaudio input (i.e. microphone) rather than files and outputs text as a stream?

skoocda 8y ago
Kaldi has a couple open-source online LVCSR models which can definitely do live decoding. Though I'm not 100% sure if there is support for PulseAudio, you may need an auxiliary service to pipe it in. Easiest interface for it is via gstreamer: github.com/alumae/kaldi-gstreamer-server