5 ms·

I see a lot of skeptical voices here, (somewhat warranted, given it's a voice assistant technology), but the fact remains that if we want open, on-device voice

I see a lot of skeptical voices here, (somewhat warranted, given it's a voice assistant technology), but the fact remains that if we want open, on-device voice recognition, we'll have to do the work and donate sample data. This extension is trying to provide some useful functionality in the hopes that Mozilla gets more data for https://commonvoice.mozilla.org https://commonvoice.mozilla.org I'd at least consider recording your voice, especially if you're a non-native English speaker, like myself, have an accent etc. It took many years for free software to start to take on the smartphone segment, with previous efforts, (including by Mozilla), failing and only now PinePhone & Librem 5 giving it another go, but unless you're a super hardcore enthusiast, you carry an iPhone/Android today. I see this as a way to push back on the likes of Amazon, Google and Apple with this. If regular Firefox users are able to use an on-device, privacy respecting voice assistance and other open-source projects can use Mozilla's tools and datasets to build compelling competitors to Alexa, I'd see that as proof that free software is able to address new, emerging markets too.

hnarn 6y ago
> This extension is trying to provide some useful functionality in the hopes that Mozilla gets more data for https://commonvoice.mozilla.org https://commonvoice.mozilla.org This is awesome! I love contributing to open source initiatives like this. I'm also a non-native speaker so hopefully I'll add some color to the voices recorded.
deleted 6y ago
[deleted]
panpanna 6y ago
> if we want open, on-device voice recognition, we'll have to do the work and donate sample data. Fair enough, but is any stopping Mozilla from _also_ selling your voice data to third parties including advertisers and commercial ML interests? (Asking this because Firefox has started sharing our data with leanplum)
- AsyncAwait 6y ago
  As long as they're upfront about what they collect and how to not send them data, am fine with it. Obviously opt-in is much proffered to opt-out, but from the Firefox Voice site, it clearly states sharing data is opt-in. My guess would be there isn't much of a point selling voice data from an open dataset. Also, since the code is in the open, it would be relatively easy to spot if they were sending data to somewhere they do not or record when they shouldn't.
- Cybiote 6y ago
  It still remains a risk well worth taking. With Mozilla, it is merely uncertain but for just about every other company, it is all but guaranteed. You can plainly see this philosophy in the industry's naming sense, where super-computers of yester-year are relegated to "edge" roles. As of today, the open source and free software equivalents to machine learning and AI products are sorely lacking when compared to commercial offerings. Whether it is open-ended speech to text with good ergonomics, text to speech, intent recognition, speaker recognition, OCR for text, OCR in the wild, translation, object recognition, image segmentation, image to text or natural language processing, commercial offerings are leagues ahead of what free software can do. If we look at one of the most impressive AI demonstrations in history, GPT-3, it is not apparent whether open source can even replicate it because with AI, unlike in the past, time and skill is no longer directly fungible with money. I would argue the concentration of such capabilities to Microsoft and Google servers is a threat to the ideals of free software as great as any it has seen before. Yet, relatively little attention is spent there because people are too focused on yesterday's problems. This concentration is difficult to avoid because current algorithms require large amounts of data and computing ability, which only large corporations can marshal. Mozilla is far from perfect but despite their many stumbles, they're the only large organization seriously attempting to address this imbalance. As much as these algorithms are marketed as AI to users, ML is better thought of as libraries, in the line of ffmpeg, to programmers. Mozilla still do seem to care about creating a local-first offering. If everyone stops using them then what is gained exactly?
  - est31 6y ago
    > it is not apparent whether open source can even replicate it because with AI, unlike in the past, time and skill is no longer directly fungible with money. Indeed. The creator of LuaJIT is only a single, very skilled, person with a desktop computer. People like Fabrice Bellard can produce gigantic amounts of FLOSS source code. Yes, those are only examples, but people with their skills and motivations to build FLOSS software will need access to lots of money in order to be able to build ML models.
dreamcompiler 6y ago
> if we want open, on-device voice recognition, we'll have to do the work and donate sample data. We absolutely will not. The only reason people believe this is that they've forgotten how to do speaker-dependent recognition (SDR), which is more accurate and more secure anyway. We were doing SDR in the 80s with 1/1000 the CPU power and 1/1000 the memory. SDR does require an initial training session, but once that's done any modern computer or smartphone should be able to handle it locally with no cloud server environment.
- eganist 6y ago
  Who's "we" in this context? Because just below you, HN has comments from willing donors. My point being that while there may still be a market for SDR, there's a broader market for speaker-independent recognition (SIR) simply because people want the tech to just work rather than feel like they messed up training the device when the device can't recognize them.
  - dividedbyzero 6y ago
    Using someone else's voice assistant is also a legitimate use case, especially if it's used to control music, lights, blinds, AC, car functionality ... that absolutely requires solid SIR.
  - codekansas 6y ago
    I think this can be viewed as a marketing and UX problem, sort of. It reminds me of the Wii Amiibo - people actually paid money to train their AI bots because of how Nintendo designed them. Not sure how many people, but a reasonable enough segment of the market that Nintendo thought it a worthwhile investment anyway
- oergiR 6y ago
  Training a speaker-specific recogniser that improves over a generic recogniser requires a lot more data nowadays. First, generic systems are a lot better and trained on a lot more data nowadays. Second, speaker adaptation worked better for the Gaussian mixture models from the late nineties (don’t know about the eighties) than for neural networks.
IgorPartola 6y ago
Very good point. Honestly I use my Echos for exactly two things: turning smart lights on and off and setting timers. I occasionally will ask it the weather or to play a song or a podcast. That’s about it. It seems like for my use cases it doesn’t need full on speech recognition and the million Alexa skills out there. Just a few simple phrases would suffice.
- StavrosK 6y ago
  I'd go farther and say that I specifically don't want the million Alexa skills out there. A system that let me write my own intents with access to the top-level namespace would be ideal, but it would absolutely need to come with good hardware. That's where Alexa/Google have the upper hand currently, I think.
anthony_barker 6y ago
Nokia n800 was crushed by internal fighting.
nshm 6y ago
You have wrong premise here. First of all, current Mozilla data is almost useless for training because it is carefully read speech. You do not need much of it, even accents. If you add 1000 hours of CV data to 1000 hours of random data the improvement in accuracy will be minimal. Same for the speech collected with Firefox Voice, it will be mostly a set of short commands, most likely it will not be very useful for generic transcription of random people speech. Second, you can build models much better than Mozilla ones simply from public data, there is no need to collect user voices. We at Vosk https://alphacephei.com/vosk/ https://alphacephei.com/vosk/ support 10+ languages for example without any user data. Everyone creates very good models from augmented text-to-speech data these days (Microsoft demonstrated in the last paper you can get almost as good as domain-specific data https://arxiv.org/abs/2007.15188 https://arxiv.org/abs/2007.15188). Given that it surprises me that Mozilla continues to insist they need the voices of their userbase.

hnarn 6y ago

> This extension is trying to provide some useful functionality in the hopes that Mozilla gets more data for https://commonvoice.mozilla.org https://commonvoice.mozilla.org This is awesome! I love contributing to open source initiatives like this. I'm also a non-native speaker so hopefully I'll add some color to the voices recorded.

deleted 6y ago

[deleted]

panpanna 6y ago

> if we want open, on-device voice recognition, we'll have to do the work and donate sample data. Fair enough, but is any stopping Mozilla from _also_ selling your voice data to third parties including advertisers and commercial ML interests? (Asking this because Firefox has started sharing our data with leanplum)

AsyncAwait 6y ago
As long as they're upfront about what they collect and how to not send them data, am fine with it. Obviously opt-in is much proffered to opt-out, but from the Firefox Voice site, it clearly states sharing data is opt-in. My guess would be there isn't much of a point selling voice data from an open dataset. Also, since the code is in the open, it would be relatively easy to spot if they were sending data to somewhere they do not or record when they shouldn't.
Cybiote 6y ago
It still remains a risk well worth taking. With Mozilla, it is merely uncertain but for just about every other company, it is all but guaranteed. You can plainly see this philosophy in the industry's naming sense, where super-computers of yester-year are relegated to "edge" roles. As of today, the open source and free software equivalents to machine learning and AI products are sorely lacking when compared to commercial offerings. Whether it is open-ended speech to text with good ergonomics, text to speech, intent recognition, speaker recognition, OCR for text, OCR in the wild, translation, object recognition, image segmentation, image to text or natural language processing, commercial offerings are leagues ahead of what free software can do. If we look at one of the most impressive AI demonstrations in history, GPT-3, it is not apparent whether open source can even replicate it because with AI, unlike in the past, time and skill is no longer directly fungible with money. I would argue the concentration of such capabilities to Microsoft and Google servers is a threat to the ideals of free software as great as any it has seen before. Yet, relatively little attention is spent there because people are too focused on yesterday's problems. This concentration is difficult to avoid because current algorithms require large amounts of data and computing ability, which only large corporations can marshal. Mozilla is far from perfect but despite their many stumbles, they're the only large organization seriously attempting to address this imbalance. As much as these algorithms are marketed as AI to users, ML is better thought of as libraries, in the line of ffmpeg, to programmers. Mozilla still do seem to care about creating a local-first offering. If everyone stops using them then what is gained exactly?
- est31 6y ago
  > it is not apparent whether open source can even replicate it because with AI, unlike in the past, time and skill is no longer directly fungible with money. Indeed. The creator of LuaJIT is only a single, very skilled, person with a desktop computer. People like Fabrice Bellard can produce gigantic amounts of FLOSS source code. Yes, those are only examples, but people with their skills and motivations to build FLOSS software will need access to lots of money in order to be able to build ML models.

dreamcompiler 6y ago

> if we want open, on-device voice recognition, we'll have to do the work and donate sample data. We absolutely will not. The only reason people believe this is that they've forgotten how to do speaker-dependent recognition (SDR), which is more accurate and more secure anyway. We were doing SDR in the 80s with 1/1000 the CPU power and 1/1000 the memory. SDR does require an initial training session, but once that's done any modern computer or smartphone should be able to handle it locally with no cloud server environment.

eganist 6y ago
Who's "we" in this context? Because just below you, HN has comments from willing donors. My point being that while there may still be a market for SDR, there's a broader market for speaker-independent recognition (SIR) simply because people want the tech to just work rather than feel like they messed up training the device when the device can't recognize them.
- dividedbyzero 6y ago
  Using someone else's voice assistant is also a legitimate use case, especially if it's used to control music, lights, blinds, AC, car functionality ... that absolutely requires solid SIR.
- codekansas 6y ago
  I think this can be viewed as a marketing and UX problem, sort of. It reminds me of the Wii Amiibo - people actually paid money to train their AI bots because of how Nintendo designed them. Not sure how many people, but a reasonable enough segment of the market that Nintendo thought it a worthwhile investment anyway
oergiR 6y ago
Training a speaker-specific recogniser that improves over a generic recogniser requires a lot more data nowadays. First, generic systems are a lot better and trained on a lot more data nowadays. Second, speaker adaptation worked better for the Gaussian mixture models from the late nineties (don’t know about the eighties) than for neural networks.

IgorPartola 6y ago

Very good point. Honestly I use my Echos for exactly two things: turning smart lights on and off and setting timers. I occasionally will ask it the weather or to play a song or a podcast. That’s about it. It seems like for my use cases it doesn’t need full on speech recognition and the million Alexa skills out there. Just a few simple phrases would suffice.

StavrosK 6y ago
I'd go farther and say that I specifically don't want the million Alexa skills out there. A system that let me write my own intents with access to the top-level namespace would be ideal, but it would absolutely need to come with good hardware. That's where Alexa/Google have the upper hand currently, I think.

anthony_barker 6y ago

Nokia n800 was crushed by internal fighting.

nshm 6y ago

You have wrong premise here. First of all, current Mozilla data is almost useless for training because it is carefully read speech. You do not need much of it, even accents. If you add 1000 hours of CV data to 1000 hours of random data the improvement in accuracy will be minimal. Same for the speech collected with Firefox Voice, it will be mostly a set of short commands, most likely it will not be very useful for generic transcription of random people speech. Second, you can build models much better than Mozilla ones simply from public data, there is no need to collect user voices. We at Vosk https://alphacephei.com/vosk/ https://alphacephei.com/vosk/ support 10+ languages for example without any user data. Everyone creates very good models from augmented text-to-speech data these days (Microsoft demonstrated in the last paper you can get almost as good as domain-specific data https://arxiv.org/abs/2007.15188 https://arxiv.org/abs/2007.15188). Given that it surprises me that Mozilla continues to insist they need the voices of their userbase.