7 ms·

Snips Uses Rust to Build an Embedded Voice Assistant

209 points by albi_lander 9y ago

oulipo 9y ago
Hi, I'm the co-founder of https://snips.ai https://snips.ai and we are building a 100% on-device Voice AI platform which runs on the Raspberry Pi 3 It is free to use for makers, and we will start open-sourcing the components a few weeks from now. We are using Rust because it is safe and very fast! We would love to tell more about what we are building if you have questions. The whole platform runs on-device which makes it ideal for privacy, cost, and to allow it to run when there is no network We are available in English, French, German, and soon Japanese and Korean and we are working on other European languages! We would love to see what you build with our platform to feature it on our website Take a look at what some people have built with it: https://github.com/snipsco/awesome-snips https://github.com/snipsco/awesome-snips and a few tutorials to get you started: https://medium.com/snips-ai/building-a-voice-controlled-home-sound-system-using-snips-and-sonos-2aaf16523ce9 https://medium.com/snips-ai/building-a-voice-controlled-home...
- t_fatus 9y ago
  nice to have projects linked, any live demo ?
  - mfester 9y ago
    Here are a few: - Music player with Spotify and Sonos: https://vimeo.com/237742054 https://vimeo.com/237742054 - News and weather forecasts: https://www.youtube.com/watch?v=IFQzDSWr2l0 https://www.youtube.com/watch?v=IFQzDSWr2l0 - Controlling blinds: https://www.youtube.com/watch?v=ukkOLqcm2CY&t=9s https://www.youtube.com/watch?v=ukkOLqcm2CY&t=9s - Zork game: https://www.youtube.com/watch?v=5RX4Dm9TmCY https://www.youtube.com/watch?v=5RX4Dm9TmCY - Prayer assistant: https://www.youtube.com/watch?v=x1AGaYBLzkA&feature=youtu.be https://www.youtube.com/watch?v=x1AGaYBLzkA&feature=youtu.be
- JustFinishedBSG 9y ago
  Hope you still exist in Jan 2021 when I'm getting (hopefully ah) my PhD ;) so I can apply
- bitL 9y ago
  Do you use Deep Learning, HMM or something else for your voice recognition? If so, how did you make it fast on RPi3? Can I use your tech on RPi2 as well (for controlling a humanoid robot)?
  - oulipo 9y ago
    Yes, this is a DL model! For the ASR and the wakeword recognition. We are doing some low-level optimizations For the Raspberry Pi 2 the model is a bit too large, but we will be able to handle them as hubs: your Raspberry Pi 2 in multiple rooms will detect the wakeword and stream the voice to a "hub" Raspberry Pi 3 which will do the speech recognition
    - riku_iki 9y ago
      what tools you use for building/training DL models? How do you connect them to Rust?
      - woodson 9y ago
        Judging by their employees' posts to the kaldi-help group, they most likely use the Kaldi toolkit.
- EugeneOZ 9y ago
  Rust's algebraic data types and patterns matching allows to write mathematically-complete decisions flows - do you use it?
- nerdponx 9y ago
  +1 for fully on-device. Where, generally, does a company like this get training data? Do you just hire a team to tag TV clips and YouTube videos?
  - oulipo 9y ago
    We do a mix of having people generate data, gathering data from public and commercial sources
    - nerdponx 9y ago
      And another +1 for transparency. This is a product I can get behind.
- jffry 9y ago
  I signed up for an account and plan on trying it out later, and I just want to say: thank you for explaining what your newsletter is, and for making it opt-in, instead of opt-out.
- bluGill 9y ago
  Are you aware of mycroft.ai? Your goals are similar, but the implementation is different. Any possibility of collaboration.
- squarefoot 9y ago
  This sounds very interesting. Being capable of working unconnected is the only way to get true privacy from these devices. Having them open sourced (hopefully 100% though I can understand the reasons for keeping some parts closed) would be even better. I have always been strongly opposed to networked voice assistants because the risk of them being used as spying tools by design is simply too high, And surely no Google, Amazon, Apple etc. assistant will never enter my home, but that one is a different beast. Keep up the good work! ps. Please add other European languages soon :)
  - oulipo 9y ago
    We are working on this! What language would you want?
    - squarefoot 9y ago
      I'm Italian. There are applications in the automotive field for a mostly disconnected device such as this one. If it can deal with a noisy environment like a car that would make it even more interesting.
      - oulipo 9y ago
        We are also working on car environment, and we expect to start italian this year :) Subscribe to our newsletter!
- just_testing 9y ago
  amazing! Do you have any timelines for portuguese? (And, quite specifically, brazilian portuguese?) Thanks!
  - jodureau 9y ago
    Hi! We have quite a few languages on our plate, but it's very likely we'll work on Brazilian Portuguese before the end of the year.
t_fatus 9y ago
I really love the idea of an off-line voice assistant, however most of my tech friends in France and I feel that Snips (which is very visible on the press here) still has to deliver something tangible (). This might be the first time I want to try one of their product though. they had announced a keyboard app for iOS and Android which did seemingly the same thing as the swipe keyboard.
- altharaz 9y ago
  I agree. Their IA assistant seems really interesting, and I would really love to give it a try, but I don't want to invest energy if there is a high risk that the service shuts down soon. I remember the Tranquilien app [0], which was also developed by Snips. There was a lot of media talking about it, but it has been quickly stopped :(. [0] https://www.digital.sncf.com/store/applications/tranquilien-0 https://www.digital.sncf.com/store/applications/tranquilien-...
  - __bee 9y ago
    I think that they shifted dramatically from experimenting around `smart cities` to be an AI Startup focusing on building voice assistants. On Techcrunch [0] > The team didn’t have a product or business model in mind. It experimented for a while, launching projects around smart cities, such as Tranquilien, a service that predicts how busy your train is going to be, or a smart empty parking spot prediction service.`` [0] https://techcrunch.com/2015/06/26/snips-grabs-6-3-million-to-add-an-artificial-intelligence-layer-to-your-phone/ https://techcrunch.com/2015/06/26/snips-grabs-6-3-million-to... [1] https://www.crunchbase.com/organization/snips-2 https://www.crunchbase.com/organization/snips-2
  - albi_lander 9y ago
    Actually the good thing is that if Snips disappears, you will still be able to run your snips assistant as everything runs locally.
    - StavrosK 9y ago
      Yeah, but that's until some library makes a backwards-incompatible change and the whole things stops building. Sure, it's better than a completely proprietary service, but support and active maintenance are very important too.
hardwaresofton 9y ago
Could you expand on why CMUSphinx[0]/Julius[1] not used? I'm all for using rust for the speed and safety it allows, and then FFI-ing to CMUSphinx (this is exactly what I want to do, given more free time in the future), could you explain why the available open source libraries (CMUsphinx is just one, there's also Julius) couldn't be used? [0] - http://cmusphinx.github.io/ http://cmusphinx.github.io/ [1] - https://github.com/julius-speech/julius https://github.com/julius-speech/julius
- IshKebab 9y ago
  Have you used Sphinx? Remember in the 90s and early 00's when speech recognition was laughable and barely worked, and it seemed like one of those thing that would never really work? Sphinx is like that.
  - hardwaresofton 9y ago
    Yes, I've used sphinx, not quite recently, and when I last used it (which was years ago), it was trivial to set up a working english install with the pre-provided models, as an amateur. From there it's only the problem of improving the model (which is obviously a pretty hard problem). Are you saying that the sphinx progress is at that same level currently as the state of the art in the 90s? Surely that's hyperbole. Also, using AI to generate even better models doesn't seem like a bad idea, if you're looking to improve it -- the fundamentals of speech recognition haven't changed, why build a completely separate open source product instead of contributing to something already well understood and accessible?
    - yorwba 9y ago
      As far as I can tell, CMU Sphinx is still based on HMMs, which were the previous state of the art before neural networks brought a breakthrough in model performance. So it is likely that CMU Sphinx is currently not much better than what was possible in the 90s. When I last looked into this, I found a mailing-list message by one of the maintainers, where he explicitly recommended to use Kaldi if you want better results. Kaldi does support neural network models in addition to good old HMMs, but it is very "researchy": everything is set up so that you can replace any step in the pipeline by your components (and then publish a paper about your results). But that also means that you pretty much have to be an expert to correctly assemble a working product from the available components, and it's pretty much assumed that you will be training your own models, which can be difficult when you don't have access to lots of data. So yes, the existing landscape for open-source speech recognition leaves something to be desired, and the focus of existing projects doesn't necessarily lend itself to turning them into what you want.
coldtea 9y ago
>How can Snips embed all the code for a voice assistant onto a single device? They wrote it using the Rust systems programming language. And that's what we call an "non sequitur".
- Jyaif 9y ago
  And it taints the whole piece. "Instead of having to write and then rewrite for each platform, he used Rust’s cross-compilation capability" also does not make a lot of sense.
  - Yoric 9y ago
    Well, to be fair, Rust's toolchain "speaks" cross-compilation natively. I haven't found any other toolchain with such a well-designed cross-compilation experience. The piece seems to have been a bit over-simplified for non-technical readers, though.
    - luciocorrea 9y ago
      Go has superb support for cross-compilation too.
quadrature 9y ago
Great project, definitely a worthy goal. I know several people who are skeptical of voice assistants because they aren't fully on device. The blog article was very lean on the technical details apart from some basics about rust's safety and portability features. could you elaborate on where the gains from rust come into play ?, is your model built on a DL framework that is written in rust ?.
- oulipo 9y ago
  The DL framework is classical, but the models can then be reimplemented in pure Rust for efficiency
  - quadrature 9y ago
    So you're taking the parameters from something like tensorflow and then exporting them to be executed in Rust ? and that is more efficient than the c++ backend of tensorflow ?.
    - throwawayendy 9y ago
      And also removing all of the overhead of the machine learning abstraction. Although I am skeptical that anyone would ever actually do that
      - quadrature 9y ago
        Not exactly sure what you mean. After the model is trained there isn't much "machine learning abstraction". you can serve the model using tensorflow serving using exported parameters. Its exactly what you want for serving the model.
      - danieldk 9y ago
        I used to train my parser (before switching to Tensorflow) using Caffe, dumped the parameters using a small program, and loaded them up in Go arrays slices and applied the network using simple C BLAS operations. This works fine, especially when you are using simpler networks. As a bonus, you don't have the overhead of Tensorflow session runs. It does become a bit of a drag when you are building more complex networks (e.g. with multiple RNN layers, batch normalization, etc.). In that case there are two straightforward options for Rust. There is a Tensorflow Rust binding against the tensorflow C API [1] with which you can just load and run a frozen Tensorflow graph. This is the approach that I am currently using, though I am running graphs on workstations/servers. Another option is compiling the graph with Tensorflow's XLA AOT compilation, which compiles the network to C++ classes (that you could bind from Rust). [1] https://github.com/tensorflow/rust https://github.com/tensorflow/rust
ziftface 9y ago
> It had the traits Snips needed Well done.
bfrog 9y ago
This sounds amazing. Looking forward to trying it out on my rpi
dustindriver 9y ago
Does the voice recognition AI use all of the Pi's processing power while running? I mean, can the Pi do anything else while the voice recog is running? Or does it context switch after a voice command? Sorry if this is a silly question...
- oulipo 9y ago
  No, the Pi can still do computations while the speech recognition runs

209 points by albi_lander 9y ago

Hi, I'm the co-founder of https://snips.ai https://snips.ai and we are building a 100% on-device Voice AI platform which runs on the Raspberry Pi 3 It is free to use for makers, and we will start open-sourcing the components a few weeks from now. We are using Rust because it is safe and very fast! We would love to tell more about what we are building if you have questions. The whole platform runs on-device which makes it ideal for privacy, cost, and to allow it to run when there is no network We are available in English, French, German, and soon Japanese and Korean and we are working on other European languages! We would love to see what you build with our platform to feature it on our website Take a look at what some people have built with it: https://github.com/snipsco/awesome-snips https://github.com/snipsco/awesome-snips and a few tutorials to get you started: https://medium.com/snips-ai/building-a-voice-controlled-home-sound-system-using-snips-and-sonos-2aaf16523ce9 https://medium.com/snips-ai/building-a-voice-controlled-home...

t_fatus 9y ago
nice to have projects linked, any live demo ?
- mfester 9y ago
  Here are a few: - Music player with Spotify and Sonos: https://vimeo.com/237742054 https://vimeo.com/237742054 - News and weather forecasts: https://www.youtube.com/watch?v=IFQzDSWr2l0 https://www.youtube.com/watch?v=IFQzDSWr2l0 - Controlling blinds: https://www.youtube.com/watch?v=ukkOLqcm2CY&t=9s https://www.youtube.com/watch?v=ukkOLqcm2CY&t=9s - Zork game: https://www.youtube.com/watch?v=5RX4Dm9TmCY https://www.youtube.com/watch?v=5RX4Dm9TmCY - Prayer assistant: https://www.youtube.com/watch?v=x1AGaYBLzkA&feature=youtu.be https://www.youtube.com/watch?v=x1AGaYBLzkA&feature=youtu.be
JustFinishedBSG 9y ago
Hope you still exist in Jan 2021 when I'm getting (hopefully ah) my PhD ;) so I can apply
bitL 9y ago
Do you use Deep Learning, HMM or something else for your voice recognition? If so, how did you make it fast on RPi3? Can I use your tech on RPi2 as well (for controlling a humanoid robot)?
- oulipo 9y ago
  Yes, this is a DL model! For the ASR and the wakeword recognition. We are doing some low-level optimizations For the Raspberry Pi 2 the model is a bit too large, but we will be able to handle them as hubs: your Raspberry Pi 2 in multiple rooms will detect the wakeword and stream the voice to a "hub" Raspberry Pi 3 which will do the speech recognition
  - riku_iki 9y ago
    what tools you use for building/training DL models? How do you connect them to Rust?
    - woodson 9y ago
      Judging by their employees' posts to the kaldi-help group, they most likely use the Kaldi toolkit.
EugeneOZ 9y ago
Rust's algebraic data types and patterns matching allows to write mathematically-complete decisions flows - do you use it?
nerdponx 9y ago
+1 for fully on-device. Where, generally, does a company like this get training data? Do you just hire a team to tag TV clips and YouTube videos?
- oulipo 9y ago
  We do a mix of having people generate data, gathering data from public and commercial sources
  - nerdponx 9y ago
    And another +1 for transparency. This is a product I can get behind.
jffry 9y ago
I signed up for an account and plan on trying it out later, and I just want to say: thank you for explaining what your newsletter is, and for making it opt-in, instead of opt-out.
bluGill 9y ago
Are you aware of mycroft.ai? Your goals are similar, but the implementation is different. Any possibility of collaboration.
squarefoot 9y ago
This sounds very interesting. Being capable of working unconnected is the only way to get true privacy from these devices. Having them open sourced (hopefully 100% though I can understand the reasons for keeping some parts closed) would be even better. I have always been strongly opposed to networked voice assistants because the risk of them being used as spying tools by design is simply too high, And surely no Google, Amazon, Apple etc. assistant will never enter my home, but that one is a different beast. Keep up the good work! ps. Please add other European languages soon :)
- oulipo 9y ago
  We are working on this! What language would you want?
  - squarefoot 9y ago
    I'm Italian. There are applications in the automotive field for a mostly disconnected device such as this one. If it can deal with a noisy environment like a car that would make it even more interesting.
    - oulipo 9y ago
      We are also working on car environment, and we expect to start italian this year :) Subscribe to our newsletter!
just_testing 9y ago
amazing! Do you have any timelines for portuguese? (And, quite specifically, brazilian portuguese?) Thanks!
- jodureau 9y ago
  Hi! We have quite a few languages on our plate, but it's very likely we'll work on Brazilian Portuguese before the end of the year.

t_fatus 9y ago

I really love the idea of an off-line voice assistant, however most of my tech friends in France and I feel that Snips (which is very visible on the press here) still has to deliver something tangible (). This might be the first time I want to try one of their product though. they had announced a keyboard app for iOS and Android which did seemingly the same thing as the swipe keyboard.

altharaz 9y ago
I agree. Their IA assistant seems really interesting, and I would really love to give it a try, but I don't want to invest energy if there is a high risk that the service shuts down soon. I remember the Tranquilien app [0], which was also developed by Snips. There was a lot of media talking about it, but it has been quickly stopped :(. [0] https://www.digital.sncf.com/store/applications/tranquilien-0 https://www.digital.sncf.com/store/applications/tranquilien-...
- __bee 9y ago
  I think that they shifted dramatically from experimenting around `smart cities` to be an AI Startup focusing on building voice assistants. On Techcrunch [0] > The team didn’t have a product or business model in mind. It experimented for a while, launching projects around smart cities, such as Tranquilien, a service that predicts how busy your train is going to be, or a smart empty parking spot prediction service.`` [0] https://techcrunch.com/2015/06/26/snips-grabs-6-3-million-to-add-an-artificial-intelligence-layer-to-your-phone/ https://techcrunch.com/2015/06/26/snips-grabs-6-3-million-to... [1] https://www.crunchbase.com/organization/snips-2 https://www.crunchbase.com/organization/snips-2
- albi_lander 9y ago
  Actually the good thing is that if Snips disappears, you will still be able to run your snips assistant as everything runs locally.
  - StavrosK 9y ago
    Yeah, but that's until some library makes a backwards-incompatible change and the whole things stops building. Sure, it's better than a completely proprietary service, but support and active maintenance are very important too.

hardwaresofton 9y ago

Could you expand on why CMUSphinx[0]/Julius[1] not used? I'm all for using rust for the speed and safety it allows, and then FFI-ing to CMUSphinx (this is exactly what I want to do, given more free time in the future), could you explain why the available open source libraries (CMUsphinx is just one, there's also Julius) couldn't be used? [0] - http://cmusphinx.github.io/ http://cmusphinx.github.io/ [1] - https://github.com/julius-speech/julius https://github.com/julius-speech/julius

IshKebab 9y ago
Have you used Sphinx? Remember in the 90s and early 00's when speech recognition was laughable and barely worked, and it seemed like one of those thing that would never really work? Sphinx is like that.
- hardwaresofton 9y ago
  Yes, I've used sphinx, not quite recently, and when I last used it (which was years ago), it was trivial to set up a working english install with the pre-provided models, as an amateur. From there it's only the problem of improving the model (which is obviously a pretty hard problem). Are you saying that the sphinx progress is at that same level currently as the state of the art in the 90s? Surely that's hyperbole. Also, using AI to generate even better models doesn't seem like a bad idea, if you're looking to improve it -- the fundamentals of speech recognition haven't changed, why build a completely separate open source product instead of contributing to something already well understood and accessible?
  - yorwba 9y ago
    As far as I can tell, CMU Sphinx is still based on HMMs, which were the previous state of the art before neural networks brought a breakthrough in model performance. So it is likely that CMU Sphinx is currently not much better than what was possible in the 90s. When I last looked into this, I found a mailing-list message by one of the maintainers, where he explicitly recommended to use Kaldi if you want better results. Kaldi does support neural network models in addition to good old HMMs, but it is very "researchy": everything is set up so that you can replace any step in the pipeline by your components (and then publish a paper about your results). But that also means that you pretty much have to be an expert to correctly assemble a working product from the available components, and it's pretty much assumed that you will be training your own models, which can be difficult when you don't have access to lots of data. So yes, the existing landscape for open-source speech recognition leaves something to be desired, and the focus of existing projects doesn't necessarily lend itself to turning them into what you want.

coldtea 9y ago

>How can Snips embed all the code for a voice assistant onto a single device? They wrote it using the Rust systems programming language. And that's what we call an "non sequitur".

Jyaif 9y ago
And it taints the whole piece. "Instead of having to write and then rewrite for each platform, he used Rust’s cross-compilation capability" also does not make a lot of sense.
- Yoric 9y ago
  Well, to be fair, Rust's toolchain "speaks" cross-compilation natively. I haven't found any other toolchain with such a well-designed cross-compilation experience. The piece seems to have been a bit over-simplified for non-technical readers, though.
  - luciocorrea 9y ago
    Go has superb support for cross-compilation too.

quadrature 9y ago

Great project, definitely a worthy goal. I know several people who are skeptical of voice assistants because they aren't fully on device. The blog article was very lean on the technical details apart from some basics about rust's safety and portability features. could you elaborate on where the gains from rust come into play ?, is your model built on a DL framework that is written in rust ?.

oulipo 9y ago
The DL framework is classical, but the models can then be reimplemented in pure Rust for efficiency
- quadrature 9y ago
  So you're taking the parameters from something like tensorflow and then exporting them to be executed in Rust ? and that is more efficient than the c++ backend of tensorflow ?.
  - throwawayendy 9y ago
    And also removing all of the overhead of the machine learning abstraction. Although I am skeptical that anyone would ever actually do that
    - quadrature 9y ago
      Not exactly sure what you mean. After the model is trained there isn't much "machine learning abstraction". you can serve the model using tensorflow serving using exported parameters. Its exactly what you want for serving the model.
    - danieldk 9y ago
      I used to train my parser (before switching to Tensorflow) using Caffe, dumped the parameters using a small program, and loaded them up in Go arrays slices and applied the network using simple C BLAS operations. This works fine, especially when you are using simpler networks. As a bonus, you don't have the overhead of Tensorflow session runs. It does become a bit of a drag when you are building more complex networks (e.g. with multiple RNN layers, batch normalization, etc.). In that case there are two straightforward options for Rust. There is a Tensorflow Rust binding against the tensorflow C API [1] with which you can just load and run a frozen Tensorflow graph. This is the approach that I am currently using, though I am running graphs on workstations/servers. Another option is compiling the graph with Tensorflow's XLA AOT compilation, which compiles the network to C++ classes (that you could bind from Rust). [1] https://github.com/tensorflow/rust https://github.com/tensorflow/rust

ziftface 9y ago

> It had the traits Snips needed Well done.

bfrog 9y ago

This sounds amazing. Looking forward to trying it out on my rpi

dustindriver 9y ago

Does the voice recognition AI use all of the Pi's processing power while running? I mean, can the Pi do anything else while the voice recog is running? Or does it context switch after a voice command? Sorry if this is a silly question...

oulipo 9y ago
No, the Pi can still do computations while the speech recognition runs