6 ms·

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is th

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same phrase that worked a few hours before the thing first plays "dance charts" and when I repeat myself it plays "german rap" - if I repeat myself again I'm back to "dance charts". This happens only in the evening. Next day for daytime nap it all works as expected. Siri has the quirk that it won't turn off lights in the evening. During day it's all fine. In the evening "Siri turn off lights in the living room" ... nope. It fails. Every time. So I have to manually open the Home app and turn off the lights. (Turning on the lights via Siri works- turning off not). Have fun debugging these, Apple and Amazon.

alcover 4y ago
> "Siri turn off lights in the living room" No snark here : why don't you make a few steps and switch them off manually ? Saving these few steps doesn't make sense to me. You need to walk to be healthy.
- mynameisvlad 4y ago
  "Siri turn off all the lights" is a hell of a lot easier than going up and down the stairs to every room in my house to make sure I turned off all lights before going to bed.
- kybernetyk 4y ago
  Because the Hue lights aren't connected to a switch :) It's just accent lighting that's all over the room.
gcanyon 4y ago
Siri has gotten better in at least one measurable way: since it first came out, I've repeatedly cited this phrase as a problem: "Hey Siri, text this photo to <person's name>" For years Siri would respond with "I'm sorry, I don't have the ability to send photos." Specifically I would point out that Siri understood what I wanted, it just couldn't do it. Each time there is an OS update, I check that to see if it's been addressed, and I've been disappointed for years. Recently in a job interview in response to one of those "what's a product you like or don't like and why" questions, I said that I liked Siri, but here's my concrete example of how it needs to improve. I pulled out my phone, said the phrase... and Siri happily texted the photo. I didn't pass the interview :-)
- djmips 4y ago
  That's too bad about the interview. Funny story though. If that was the only reason you failed the interview then I'd say it was an unfair interview.
  - gcanyon 4y ago
    They gave me no feedback why they failed me. I did recover when Siri betrayed me by succeeding where I said it would fail, but poorly. It took time to say "Well, I guess at some point Apple released an update and I had given up checking at that point. That's one of the problems with voice: generally you only get one chance to demonstrate capabilities." It redirected the course of the interview.
worthless-trash 4y ago
At least google gets that right. I couldnt add items to my shopping list because I had too many deleted items in keep list.
bbstats 4y ago
Google homes works just fine! :)
visarga 4y ago
> you can't explain what's going on inside that black box Do you have a better solution for speech recognition? We all know how well speech recognition works in reality, and we know language models can accomplish more complex tasks than setting your music and lights. These models are not state of the art, they are cheap versions for scaling up to millions of users. It's sad but we rarely get to see SOTA in a product.
- mrguyorama 4y ago
  My mom's $50 LG flip phone from 2000 did voice control better than any of the "assistants" from nowadays. Classical, non-ai methods work really really well if you are allowed to constrain the expected vocab, and with how little these voice assistants are actually capable of doing, constraining the vocab and showing users the actual phrases expected will make the user experience much better. Stop pretending to be general purpose AI and making an utter fool of it, and produce a product and functionality that actually works for the users.
- BaseballPhysics 4y ago
  > Do you have a better solution for speech recognition? Must it be necessary for anyone critiquing a technology to also come armed with a solution?
- saiya-jin 4y ago
  OP just described ridiculously bad behavior that consistently changes over time, not randomly random results. That's more a case of bad engineering (ie light sensor interfering with microphone data feed) or some stupid management decision then 'AI is complicated but we don't have anything better'
  - visarga 4y ago
    I think it was management that chose to go with a cheaper model, not the engineers. They got to think about profits. You do realise that a neural net, given the same input, will always give the same output, if you set your random seeds of course. It doesn't depend on time of day unless you degrade the service when it is overloaded - say, you use a different model in the evening that is 10x faster but 2x worse.
hn_throwaway_99 4y ago
> My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I can't comment much on Siri or Alexa, but Google Assistant has gotten heaps better at understanding my voice and intent since it was first released about 6 years ago. It used to pretty reliably misunderstand at least 1 word per sentence (which made for an extremely frustrating experience originally); now, at least for me, it's extremely rare that it misunderstands me. This would point to the problem being company-specific, and not a fundamental issue with the "black box" aspect of the technology.
- o_____________o 4y ago
  Siri's total impotence was one of the most surprising issues when I switched from stock Android to iPhone a couple years ago. I'm curious what's going on inside Apple to produce something this poor in their core product.
- outworlder 4y ago
  Interesting. Alexa misunderstands me _a lot_. There are a few commands I use very often, but somehow they don't rank high enough. I have one Google Assistant I got for free. I'll try it. Most of what I want Alexa to do is interfacing with Home Assistant anyway.
- tuatoru 4y ago
  When asked, Google assistant claims it's sticking around for good. But we know better.
- drekipus 4y ago
  I have precisely the opposite with my Google home speaker. Also a reduction of functionality, as I used to be able to read some messages and set a reminder, but now it just gives me the time and plays Spotify music. I like voice commands, but I like them working reliably. I can't believe there was like an open-source zapier-like voice command API that could be used for everything and maintained properly.
P5fRxh5kUvp2th 4y ago
This is why I don't even like the "AI assisted autocomplete". I want consistency, not fanciness. consistency breeds productivity. autocompleting different things from 1 file to the next is not helpful.
- ohwellhere 4y ago
  Statistical inference also regresses to the mean. And the mean for spelling and grammar skills in society is regrettably low. In the last year or so I've noticed more and more that I have to fix misspellings or grammatical mistakes that were introduced by autocorrect. The most common culprits I notice are contractions (when I am sick I am not "I'll", "its" sometimes actually the contracted subject and verb, etc).
endtime 4y ago
I worked on Google Assistant for ~four years. The NLP is important, but it's still just a slice of making the thing work. And while there's some cutting edge stuff going on there, it's not so general or mysterious that it can't be understood or controlled, and I don't think it's making things worse. Educated guess: your music issues can be blamed on a ranking subroutine owned by a "media" or "music" team, not on "black box AI".
magic_hamster 4y ago
I must be in the minority but I really dislike voice commands. Every time I try it I feel stupid talking to appliances. But even if I didn't mind talking to lightbulbs and thermostats it would still bother me for several reasons: I don't like being listened to by these corporations, I don't like exposing my electronics to the internet, and I still don't see how any of this stuff is better than flicking a switch.
- outworlder 4y ago
  > I don't like exposing my electronics to the internet, and I still don't see how any of this stuff is better than flicking a switch. That assumes you are close to the switch, that you have switches for all the things you want to do and that you are even free to operate said switches. I do agree if all you are doing is turning a device on and off, it isn't that helpful(it can still be as it allows you to do something else in the mean time). However, you can operate multiple devices simultaneously by triggering a routine. A single command can arm your alarm system, turn off the lights across your entire house, close curtains, activate robot vacuums and whatever else you can think of. Could it be a button? Yeah, it could. Is it as convenient? No. And obviously messing with devices is not all they can do. You can ask questions. Also, to the security point, at least the Echo devices don't allow amazon to be "listening" to you all the time. They listen for the trigger word, and then they send your query. I know, because my devices are being monitored. If they start uploading data all the time I'll know about it. And I bet I'm not the only one doing that.
  - BuildTheRobots 4y ago
    > I know, because my devices are being monitored. If they start uploading data all the time I'll know about it. And I bet I'm not the only one doing that. I had previously been pretty happy with this explanation until I got my hands on the new Pixel 7 where the transcription/live subtitles/translation is scarily accurate and completely offline. I'll believe they're not streaming audio out of the house and "listening to me" that way, but it would be easy to exfiltrate a very accurate transcription.
  - namaria 4y ago
JohnJamesRambo 4y ago
Are you sure you are speaking as clearly in the evening? I don’t understand how voice command recognition could fluctuate based on AI and time of day.
- kybernetyk 4y ago
  Don't know ... I just have had a bottle of Scotch by that time ;) Nope, I speak clearly. I even tried _very_ clear pronunciation (like talking to a person with bad hearing, etc). My wife tried, too. It comes back to playing German Rap and Dance music. Maybe I have been classified by Amazon and people who are similar to me tend to listen to dance/rap in the evening. No idea. It's just a curious case of a bug. (I switched to piano music since - which works pretty well).
- BudaDude 4y ago
  I think its a Homekit to Siri issue. I have the opposite issue, Siri almost always works for me for turning off the lights but the Home app works only half of the time.
- counttheforks 4y ago
  The AI might be trying to be "smart" and factor in the time of day when interpreting commands.
Darkphibre 4y ago
My first thought is that there may be something environmental... interfering with your voice. Something like someone watching a show in the evening, or turning on a fan when it's time for bed.
lastofthemojito 4y ago
At one point I worked alongside some folks doing machine translation work and it was interesting seeing the older rules-based translation versus the new hotness neural translation. Often the neural models did better overall, but the inscrutability of those models meant sometimes people had no idea why weird output occurred and fixing such cases wasn't as simple as slapping on one more if-statement. It felt like a lower-stakes version of Tesla self-driving - they train it more and more, and some people report improvements, and other folks report regressions and it's not like you can go in and edit the if(carStoppedAhead) statement.

alcover 4y ago

> "Siri turn off lights in the living room" No snark here : why don't you make a few steps and switch them off manually ? Saving these few steps doesn't make sense to me. You need to walk to be healthy.

mynameisvlad 4y ago
"Siri turn off all the lights" is a hell of a lot easier than going up and down the stairs to every room in my house to make sure I turned off all lights before going to bed.
kybernetyk 4y ago
Because the Hue lights aren't connected to a switch :) It's just accent lighting that's all over the room.

gcanyon 4y ago

Siri has gotten better in at least one measurable way: since it first came out, I've repeatedly cited this phrase as a problem: "Hey Siri, text this photo to <person's name>" For years Siri would respond with "I'm sorry, I don't have the ability to send photos." Specifically I would point out that Siri understood what I wanted, it just couldn't do it. Each time there is an OS update, I check that to see if it's been addressed, and I've been disappointed for years. Recently in a job interview in response to one of those "what's a product you like or don't like and why" questions, I said that I liked Siri, but here's my concrete example of how it needs to improve. I pulled out my phone, said the phrase... and Siri happily texted the photo. I didn't pass the interview :-)

djmips 4y ago
That's too bad about the interview. Funny story though. If that was the only reason you failed the interview then I'd say it was an unfair interview.
- gcanyon 4y ago
  They gave me no feedback why they failed me. I did recover when Siri betrayed me by succeeding where I said it would fail, but poorly. It took time to say "Well, I guess at some point Apple released an update and I had given up checking at that point. That's one of the problems with voice: generally you only get one chance to demonstrate capabilities." It redirected the course of the interview.

worthless-trash 4y ago

At least google gets that right. I couldnt add items to my shopping list because I had too many deleted items in keep list.

bbstats 4y ago

Google homes works just fine! :)

visarga 4y ago

> you can't explain what's going on inside that black box Do you have a better solution for speech recognition? We all know how well speech recognition works in reality, and we know language models can accomplish more complex tasks than setting your music and lights. These models are not state of the art, they are cheap versions for scaling up to millions of users. It's sad but we rarely get to see SOTA in a product.

mrguyorama 4y ago
My mom's $50 LG flip phone from 2000 did voice control better than any of the "assistants" from nowadays. Classical, non-ai methods work really really well if you are allowed to constrain the expected vocab, and with how little these voice assistants are actually capable of doing, constraining the vocab and showing users the actual phrases expected will make the user experience much better. Stop pretending to be general purpose AI and making an utter fool of it, and produce a product and functionality that actually works for the users.
BaseballPhysics 4y ago
> Do you have a better solution for speech recognition? Must it be necessary for anyone critiquing a technology to also come armed with a solution?
saiya-jin 4y ago
OP just described ridiculously bad behavior that consistently changes over time, not randomly random results. That's more a case of bad engineering (ie light sensor interfering with microphone data feed) or some stupid management decision then 'AI is complicated but we don't have anything better'
- visarga 4y ago
  I think it was management that chose to go with a cheaper model, not the engineers. They got to think about profits. You do realise that a neural net, given the same input, will always give the same output, if you set your random seeds of course. It doesn't depend on time of day unless you degrade the service when it is overloaded - say, you use a different model in the evening that is 10x faster but 2x worse.

hn_throwaway_99 4y ago

> My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I can't comment much on Siri or Alexa, but Google Assistant has gotten heaps better at understanding my voice and intent since it was first released about 6 years ago. It used to pretty reliably misunderstand at least 1 word per sentence (which made for an extremely frustrating experience originally); now, at least for me, it's extremely rare that it misunderstands me. This would point to the problem being company-specific, and not a fundamental issue with the "black box" aspect of the technology.

o_____________o 4y ago
Siri's total impotence was one of the most surprising issues when I switched from stock Android to iPhone a couple years ago. I'm curious what's going on inside Apple to produce something this poor in their core product.
outworlder 4y ago
Interesting. Alexa misunderstands me _a lot_. There are a few commands I use very often, but somehow they don't rank high enough. I have one Google Assistant I got for free. I'll try it. Most of what I want Alexa to do is interfacing with Home Assistant anyway.
tuatoru 4y ago
When asked, Google assistant claims it's sticking around for good. But we know better.
drekipus 4y ago
I have precisely the opposite with my Google home speaker. Also a reduction of functionality, as I used to be able to read some messages and set a reminder, but now it just gives me the time and plays Spotify music. I like voice commands, but I like them working reliably. I can't believe there was like an open-source zapier-like voice command API that could be used for everything and maintained properly.

P5fRxh5kUvp2th 4y ago

This is why I don't even like the "AI assisted autocomplete". I want consistency, not fanciness. consistency breeds productivity. autocompleting different things from 1 file to the next is not helpful.

ohwellhere 4y ago
Statistical inference also regresses to the mean. And the mean for spelling and grammar skills in society is regrettably low. In the last year or so I've noticed more and more that I have to fix misspellings or grammatical mistakes that were introduced by autocorrect. The most common culprits I notice are contractions (when I am sick I am not "I'll", "its" sometimes actually the contracted subject and verb, etc).

endtime 4y ago

I worked on Google Assistant for ~four years. The NLP is important, but it's still just a slice of making the thing work. And while there's some cutting edge stuff going on there, it's not so general or mysterious that it can't be understood or controlled, and I don't think it's making things worse. Educated guess: your music issues can be blamed on a ranking subroutine owned by a "media" or "music" team, not on "black box AI".

magic_hamster 4y ago

I must be in the minority but I really dislike voice commands. Every time I try it I feel stupid talking to appliances. But even if I didn't mind talking to lightbulbs and thermostats it would still bother me for several reasons: I don't like being listened to by these corporations, I don't like exposing my electronics to the internet, and I still don't see how any of this stuff is better than flicking a switch.

outworlder 4y ago
> I don't like exposing my electronics to the internet, and I still don't see how any of this stuff is better than flicking a switch. That assumes you are close to the switch, that you have switches for all the things you want to do and that you are even free to operate said switches. I do agree if all you are doing is turning a device on and off, it isn't that helpful(it can still be as it allows you to do something else in the mean time). However, you can operate multiple devices simultaneously by triggering a routine. A single command can arm your alarm system, turn off the lights across your entire house, close curtains, activate robot vacuums and whatever else you can think of. Could it be a button? Yeah, it could. Is it as convenient? No. And obviously messing with devices is not all they can do. You can ask questions. Also, to the security point, at least the Echo devices don't allow amazon to be "listening" to you all the time. They listen for the trigger word, and then they send your query. I know, because my devices are being monitored. If they start uploading data all the time I'll know about it. And I bet I'm not the only one doing that.
- BuildTheRobots 4y ago
  > I know, because my devices are being monitored. If they start uploading data all the time I'll know about it. And I bet I'm not the only one doing that. I had previously been pretty happy with this explanation until I got my hands on the new Pixel 7 where the transcription/live subtitles/translation is scarily accurate and completely offline. I'll believe they're not streaming audio out of the house and "listening to me" that way, but it would be easy to exfiltrate a very accurate transcription.
- namaria 4y ago

JohnJamesRambo 4y ago

Are you sure you are speaking as clearly in the evening? I don’t understand how voice command recognition could fluctuate based on AI and time of day.

kybernetyk 4y ago
Don't know ... I just have had a bottle of Scotch by that time ;) Nope, I speak clearly. I even tried _very_ clear pronunciation (like talking to a person with bad hearing, etc). My wife tried, too. It comes back to playing German Rap and Dance music. Maybe I have been classified by Amazon and people who are similar to me tend to listen to dance/rap in the evening. No idea. It's just a curious case of a bug. (I switched to piano music since - which works pretty well).
BudaDude 4y ago
I think its a Homekit to Siri issue. I have the opposite issue, Siri almost always works for me for turning off the lights but the Home app works only half of the time.
counttheforks 4y ago
The AI might be trying to be "smart" and factor in the time of day when interpreting commands.

Darkphibre 4y ago

My first thought is that there may be something environmental... interfering with your voice. Something like someone watching a show in the evening, or turning on a fan when it's time for bed.

lastofthemojito 4y ago

At one point I worked alongside some folks doing machine translation work and it was interesting seeing the older rules-based translation versus the new hotness neural translation. Often the neural models did better overall, but the inscrutability of those models meant sometimes people had no idea why weird output occurred and fixing such cases wasn't as simple as slapping on one more if-statement. It felt like a lower-stakes version of Tesla self-driving - they train it more and more, and some people report improvements, and other folks report regressions and it's not like you can go in and edit the if(carStoppedAhead) statement.