11 ms·
A big problem with the assistants is that as soon as they fail at a query they seem stupid, I feel stupid, and I stop using them for a long time. Last weekend
by dheera 6y ago
A big problem with the assistants is that as soon as they fail at a query they seem stupid, I feel stupid, and I stop using them for a long time.
Last weekend I had the following failed queries:
"OK Google, what is the air quality like at Mt. Shasta today?"
"OK Google, add a waypoint for the last gas station before the mountain pass"
"OK Google, what percentage of people can you detect to be wearing masks on recent Instagram photos tagged at a location within a 10 mile radius of Mt. Shasta?"
These are all things I would expect a computer assistant to do really well. They have access to so much data, and so many APIs, that they should be able to break down these sentences into a SQL-like query and give me results. The third, for example:
"recent Instagram photos" -> Instagram has an API
"tagged within a 10 mile radius" -> parse the cities within a 10 mile radius and look for tags in all of them
"people" -> use your wonderful person detection networks your friends at Waymo developed
"wearing masks" -> I'm sure your internal datasets have this label, so run an object detector
Then compile and reduce the data to give me the number I want.
That's what I want an assistant to do. But it couldn't even do the first, which just involves a single API query to fetch air quality index information. Bleh. And as for the second, it has no idea what "last gas station" or "mountain pass" means; it's a query a human would know to be extremely commonplace.
It turns out that the current generation of "assistants" are mostly just template-matchers which really doesn't help me much at all. I can set my own alarms, thank you.
- jkaptur 6y ago> I can set my own alarms, thank you. And not only that, but a specialized user interface is often preferable to voice even when the assistant passes the Turing test. There's a reason that people use apps to get food delivery rather than calling.
- dreamer7 6y agoThat doesn't seem correct. People prefer apps because they are better at providing the information we need (menus, products, services etc) and the alternative for engaging through calls is either talking with overworked humans or a painful IVR service. For me, the ideal interface to engage (order something, create a note etc) when you know what you want is through speech. It just feels so effortless.
- dheera 6y agoI can definitely say that for food I MUCH prefer the web/app interface. I can swipe through a bunch of nice-looking food and just tap on something that looks appetizing. You can't get that with voice no matter how you do it. It's like if a restaurant had no menu and the wait staff just recited the menu in your face and asked you to pick. Even with real humans, voice isn't the best interface for presenting food choices.
- Godel_unicode 6y ago> if a restaurant had no menu and the wait staff just recited the menu That's exactly how food is presented to you at lots of high-end restaurants. Once you get above the fast-casual tier, menus never have pictures. If you know a lot about food and you trust the chef, voice is a perfect medium for explaining the choices. Pictures are just a discovery aid.
- deleted 6y ago[deleted]
- dreamer7 6y agoThat's exactly what I meant. The app interface is good for conveying information. But if you know what you'll order, eg - popcorn every weekend, just saying "Jarvis, order my usual popcorn combo" is so much more seamless.
- Wowfunhappy 6y ago> There's a reason that people use apps to get food delivery rather than calling. Well, if I had some type of personal assistant who worked for me (as in, a real human), I would just call out "Hey, Sam, please order pizza for me" and continue with whatever I was doing. The reason people don't make phone calls is they add a lot of additional friction. You have to dial the number, and wait for someone to pick up, and give them your address, and deal with frequently-questionable voice quality.
- dheera 6y agoI don't know, I like swiping through photos of food to make my decisions on what to eat, and then it only takes about 30 more seconds to complete the order at most.
- Godel_unicode 6y agoThis is just the optimal halting problem; sometimes you want to spend time finding the exact thing you want, and sometimes you just want something that's good enough. Both problems don't need the same solution.
- kibwen 6y agoPizza is a weird example because pizza joint menus are more-or-less standard and unsurprising. For any non-pizza restaurant I'll want to start by scanning a menu, which is way more efficient than listening to a menu be read to me.
- wtetzner 6y agoAnd if you're already scanning the menu, just clicking on the stuff you want is easier than explaining it to someone. However, if you frequently order the same thing from a given restaurant, having a way to order "the usual" from that place via voice might be convenient. But I'm not sure it's much more convenient than being able to do the same thing with a button click or something.
- ufmace 6y ago
- heleninboodler 6y agoIt's funny, but for years I've felt like "Hey Siri, set an alarm for 7am" was 1000x easier than using the clunky Clock UI and it's almost exclusively what I use Siri for. Tasks that are so perfectly well-defined are exactly what this "smart" tech is useful for. Except that recently, Siri screwed me. I said "hey Siri, set an alarm for 6:30am" and her response was "Ok, your 6:30am alarm is on," but what she actually meant was "Ok, I ensured that your standing alarm for every tuesday at 6:30am is on" which meant she wasn't enabling the alarm for 6:30am tomorrow, so I overslept. Very annoying and completely blew my trust in a very useful function.
- tsm 6y agoOn the google side, "wake me up at 7" results in a 7AM alarm 95% of the time and a 7 PM alarm the other 5%. Just frequent enough to screw you over…
- drunkpotato 6y agoAlarms and reminders are my most frequent uses of Siri (and to entertain/frustrate the kids...). It's also pretty good for hands-free quick texts while driving and hands-free calling, though I don't do those often. Your alarm example is frustrating. Of course, you could say "set alarm for tomorrow at 6:30 am" which would work, but then you're back in the realm not of natural language commands, but formal commands that exist in the uncanny valley, just similar enough to natural language to be irritating when they fail.
- heleninboodler 6y agoYeah, I would think it would be obvious to most humans that "set an alarm for 6:30am" never means "6:30am four days from now." It seems like it should just be interpreted as the next occurring 6:30am unless otherwise specified, but hey, I'm not a computer. Edit: also, your "tomorrow at 6:30am" example is also open to interpretation if you're saying it two minutes after midnight. I'd really like it to recognize these sort of ambiguities and prompt me for clarification. "It's after midnight. By 'tomorrow at 6:30am', do you mean you want your alarm to go off about 6 hours from now or the next day?"
- airstrike 6y agoThe only two questions I ever ask Siri are "how cold is it outside?" and "wake me up {at HHam, in X minutes}"
- pbhjpbhj 6y agoI'm curious that you want to know the temperature specifically, rather than the weather. I'm guessing that's for reasons other than clothing choice?
- airstrike 6y agoI just feel like I get a more actionable answer than the "It's a bit chilly and overcast" I get when I ask about the weather – how do I know if Siri and I agree on what's chilly or not?
- pbhjpbhj 6y agoCool, pretty sure for weather Alexa says something like "in $place today it's 10 degree with a light breeze and 40% chance of rain; with a high of 13 and a low of 8; have a good $day".
- oftenwrong 6y agoI still often use a phone call (or sms) for takeaway. It's faster than using an app or website if you already know what you want, and usually cheaper, since there is no middle-man. For delivery, I agree. It is easier and faster to use a service, since my address and payment information are already saved.
- Swizec 6y ago“Alexa, play punk rock” “Playing <super specific punk rock song>” “Alexa stop. Alexa play punk rock playlist” “Cannot find punk rock playlist” “Alexa play punk rock 00’s playlist” “Can’t find” “Alexa play early 2000’s punk rock” “Playing punk rock 00’s playlist” It’s like she’s trying to mock me.
- codeulike 6y agoBurglar: "Put your hands up and show me where the money is, I won't hurt you..." Me: "Alright man, I'll tell you where is money ... ALEXA CALL THE POLICE!" Alexa: "Shuffling songs by The Police" * EVERY BREATH YOU TAKE plays as I get punched 24 times * from https://twitter.com/ppathole/status/1092034892249079813 https://twitter.com/ppathole/status/1092034892249079813
- arkh 6y agoEven if it works: do you expect the police to stop the burglar before they shoot you?
- wongarsu 6y agoI don't expect a burglar to shoot me. Raising the charge from breaking and entering to armed robbery is dumb enough, raising it to murder is downright stupid. Drug addiction makes people do crazy things, but there's a reason why most burglars flee as soon as they are noticed
- ufmace 6y agoJokes aside, the police probably want to know whether your complaint is that there is a burglar holding you at gunpoint, or that you're trying to order a pizza. They probably get both types of calls at about the same frequency.
- joombaga 6y agoHow do you say "00's"? Does Alexa say it the same way?
- coldcode 6y agoI use my voice assistant on my phone to set timers. That's it.
- maneesh 6y agoWeather is also useful! Sometimes reminders, which are basically timers. But yea that's it.
- GloriousKoji 6y agoSiri doesn't get the weather right. "What's the weather in Yosemite" and "Yosemite weather" can get you two drastically different results. As of right now (10/26/2020 10:35 AM) the two results I get are 43F and 32F.
- maneesh 6y agoSiri is always mistaken. Alexa is pretty good.
- pbhjpbhj 6y agoAlexa is super easy to make a skill for (IIRC it's one XML/JSON file hosted somewhere and registered with Amazon). So you could probably make your own with ~IFTTT to scrape the data. Alexa has "daily briefing" which will then go through a list of skills which can include your own [public] skills. I made an "and finally ..." to add funny news to the end of the daily briefing and was super impressed how easy it was.
- bserge 6y agoThe other day, I wanted it to open an app and start recording when I say "Hey Google, take a voice note"... but no dice :/
- martyvis 6y agoJust say "Hey Google, open Live Transcribe" (assuming you had that app installed)
- lqet 6y agoWhich is exactly why I prefer formal query languages over NLP queries. In both cases (at least with most state of the art NLP techniques), you have to learn certain patterns and ways to phrase a query so that the system will reliably understand it. With formal query languages, these patterns are well-defined, can be looked up and will most likely not change significantly (so there is value in memorizing them). With NLP systems, the patterns are completely opaque, you have to learn them through trial and error, they may change anytime (e.g. because the model is retrained) and they are usually significantly less powerful. I sometimes feel that the trend to prefer NLP over formal query languages is comparable to the trend to prefer GUIs over consoles in the 80ies and 90ies.
- ketzo 6y agoWell, to extend the GUI/console metaphor, it means that at some point soon, we'll all be using NLP because it's dramatically more user-friendly for the vast majority of people.
- dec0dedab0de 6y agoI understand your point, but I'm not sure that GUIs won out because they were dramatically more user-friendly. It certainly helped, but I think they won because it made multitasking possible. Multitasking from the users perspective that is, the ability to interact with more than one application at the same time. That was just not possible on a console, so even people who didn't need user friendliness were able to do things they couldn't do before. I was young at the time, but that's how I remember it at least.
- Godel_unicode 6y agoThat's just not true, though. As with many console things, multi-tasking is totally possible but it's discoverability is terrible. Ctrl+z and `jobs` is the entry level, with tmux being the end-state reached via gnu screen. This lack of discoverability is the same problem voice assistants have only moreso; no `apropos` and no tab completion. GUIs are for discovery, CLIs are for power via composability, voice/NLP assistants are for convenience.
- xwdv 6y agoWhen they do work though you feel like you’re at the helm of the future. I’ve figured out what queries almost never seem to fail and use those almost exclusively. I don’t get creative.
- antisthenes 6y agoWell put. I don't know how much of FAANG's budget goes towards improving voice assistants, but considering how much cash these companies have on hand and their operating budgets (the size of some smaller European countries), the progress in that area is just super disappointing. The 3 most common use cases were refined a long time ago (Directions, Alarms and Play Music) and everything else ran into a hard wall.
- dheera 6y agoEven with directions, it fails miserably IMO. The most it seems to be able to do is "navigate to X". I want: - "Take me on the most scenic route to X." Can't you figure that out from social media tags? Simple first order solution: routes that have more photos with more likes = more scenic. Took me 1 minute to think of that. And 1000 engineers at Google couldn't implement that? These data crunching tasks are the kind of stuff computers are supposed to be good at. - "Navigate to X but make sure you stay on paved roads." An actual problem if you are trying to use Google Maps in the back roads of California and don't have a 4WD vehicle. Google Maps loves taking you on 4WD dirt road detours. Don't you have satellite maps and street view? Can't you differentiate paved roads from unpaved ones? What the hell does your machine learning department do? - "Stop at the last grocery store before the highway 120 junction." Yeah, it doesn't even begin to understand this type of query. - "OK Google zoom out the map slightly." Nope. Sorry.
- mikestew 6y agoScenic: Garmin's devices appear to do some calculation on number of times the road doesn't go straight over a given distance. Seems to work well enough. OTOH, someone at Google has to make this a feature. Paved/unpaved: that's one suck-ass assistant you've got there. I know the Garmin on the dash of my motorcycle will give that option. The Garmin RV-specific GPS has loads of other options, such as avoid any low overpasses that will rip the solar panels off my RV. Though it doesn't look like Apple is any better than Google, only because Apple Maps doesn't offer the option. Anyway, satellite and street view? Rand McNally had this information since dirt was first created, no need to get the satellites out.
- hirako2000 6y agoBut your examples are way harder than they sound. Speech or non speech analysers have a hard time with context. What do you mean by "recent" photos. And what percentage? Of people wearing mask in each photo, or of 1 or more people with a mask in the whole set of photos? Or the percentage of photo having all people wearing a mask. We humans make a lot of deduction from context. We haven't been able to teach computers this aspect for 25 years. It has started more recently, deep learning shows potential.
- traviscj 6y agoThen parameterize & return results for one specific duration, return that duration with results, and make it easy to adjust afterward. “Here are the results I found for the last week...” “Show the same for the last 10 days”
- bserge 6y agoRecognizing masks is the hard part, indeed. But it can be done semi-decently by just analyzing tags (#wearamask or whatever). The rest is easy, it could even be done completely on-device if you have a recent high end chipset. That's how powerful phones are nowadays. Tbf, it is easier to just run your own server, and pre-program everything for yourself, a truly personalized experience. Though Google could easily do it for the millions of people using Android. I really wish they allowed custom modules or something for their Assistant, the voice recognition is unmatched.
- dheera 6y agoAbsolutely, but you could start with reasonable assumptions and refine from there. Just the percentage of individual humans wearing a mask. Recent = past week. Start somewhere. Give me an answer first, or ASK for clarification instead of "Sorry, I couldn't understand your query." I understand it's a hard problem, but with current software capabilities and given Google's compute infrastructure I honestly think some of these things are well within the realm of what a team of several hundred Google engineers can do. I'm not asking it to write Shakespeare, I'm asking it to crunch data with a sentence that could reasonably easily be parsed into a graph and be turned into a MapReduce query. I thought they were good at that stuff. Context? I know it's hard, but I thought Google has been working on that. I'm very much adjusting my expectations to what I think Google should be able to accomplish in a decade. I expect some basic context capability now, at the very least these data-crunching type use cases.
- bserge 6y agoThat is all doable with Huggin/IFTTT/Automate/Tasker/Snowboy. It could even be done on the phone itself, although battery life and heat would likely be a problem. The fact that it doesn't work just shows that Google isn't that interested in their Assistant. Which is rather strange, but maybe they're looking for better ways to monetize it? Or maybe it's going the way of Reader once AR gains momentum or something... then again Google Lens is a really nice product and no one I know even heard of it. I'm amazed at how well it can recognize any kind of writing, even my chicken scratch, and how it can look up any products/labels with decent results.
- bbarnett 6y agoProbably, they get more ad revenue from: - you doing a manual search, and being tracked/targeted - them displaying ads as a result - them displaying contextual, higher paying ads I've noticed that some things over the years, many things in fact, have been removed from Google Maps. I hypothesize that the entire reason is, these things reduce profit. For example, you used to be able to pause Google Maps. You can't now, and therefore, if you have 'history' off, you have to stop, and manually re-start your destination by typing it in. Well, history profits them. And pausing is not the best, because then, you're not active 100% of the trip. Having you active lets them determine all sorts of things, like traffic patterns, where you go, who you're near, and so on. There are lots of little things like this, which seem to be gone from earlier versions of Maps. Again, I presume, all to make more $. Which is logical, and fine, but it gets a bit tiresome and sad at times.
- Spooky23 6y agoI think the issue is that the virtual assistant is there to assist, not replace an assistant. I have an admin. I can text, call, talk, email her and say “let’s get bob, mary and finance on the phone at 4” and she’ll move stuff around to make sure that happens. I can also delegate complex administrative tasks that need to be done in my name. When I’m at the dentist, I usually do a “hey Siri” to book it as it saves time. At home, “hey Siri turn on the lights” is helpful. It’s magic, but not the same. The tech companies try to frame this stuff “bigger”, but in doing so they create an unreasonable expectation. Google home, Siri, Alexa are amazing, but we bitch and moan about them as a result.
- dheera 6y agoMaybe it's the marketing. I don't have a human assistant, but if I did, the biggest things I would expect from them isn't about scheduling a meeting or scheduling a barber appointment. I can handle that stuff myself. What I would REALLY expect of a human assistant: "Can you call up my health insurance and fight this stupid bill of $400 for COVID testing that should have been $0. Please escalate if necessary. Thanks." "Can you figure out how to fight this red light ticket that I got due to a malfunctioning red light camera? Thanks." "Can you fight this parking ticket? I had a valid permit to park there, and here's documentation of that. Thanks." "Can you register me to vote? Here's my ID. Thanks. Make sure they don't sign me up for spam." "Can you call up Comcast and fight this bill increase? Threaten to switch to another provider if necessary, I heard that works." "Can you call up this company that posted my personal information and ask them to remove it? If they refuse, threaten legal action." "Can you dispute this electric bill for me? My heating shouldn't have been $400 a month. Something must be wrong with my meter or someone is leeching power from my line." "Can you call up the 10 different grocery stores in the area and figure out which one has X in stock?" In all honesty I do wish the Google automated assistant could do all of the above. Sorry Pichai, I don't care for scheduling haircuts automatically. I want your assistant to use its hundreds of thousands of hours of human conversations and use machine learning to craft and engineer responses to humans to know EXACTLY when to escalate, EXACTLY when to ask for a manager, HOW to threaten legal action, and basically fight tooth-and-nail with language to get me what I want against the companies and institutions I need to fight with on the phone. The job of an "assistant" should be to get sh*t done and get me what I want. Use machine learning and lots of data to master the art of negotiation with customer service reps.
- lm28469 6y agoAs soon as you see them for what they are it makes perfect sense. They're basically a proxy to google search with extra dumb features on top. Every single feature that isn't google search has been hard coded (map directions, setup alarms, &c.), as soon as you get out of the hard coded cases it's blatantly obvious these things are anything but "smart" imho they're just used to give google/amazon a few more data points for their graphs
- deleted 6y ago[deleted]
- jjoonathan 6y agoYeah, and that wouldn't be a problem if the discovery story wasn't dismal or the browsing story wasn't dismal, but it definitely is. On both counts.
- tschwimmer 6y agoDiscovery and browsing are not good on the assistant interface but I'd argue that's a constraint of voice vs visual. On a desktop/mobile, the screen holds the state so you can go back to previous entries, etc. Over voice, you mind holds the state which scales much much worse.
- krrrh 6y agoTo the credit of the Shortcuts team at Apple, being able to visually select and define certain phrases for commonly completed tasks is helpful in this regard, but I’d guess only a couple % of the user base is even aware that Siri Shortcuts are possible.
- part1of2 6y agoIs this just normal phone Siri or HomePod Siri?
- 6y ago
- duxup 6y agoAgreed, the error rate before I just give up on something is pretty low. The real frustration is when it fails on tasks I swear it has done before easily. Like 'ok google' ends up googling ... the ok google command word for word sometimes. 'ok google set timer 10 minutes' As soon as I have to babysit an assistant like that I'd rather just make sure the clock is on my home screen so I set a timer.
- jjk166 6y agoBut would you have gotten better results if you had typed those same queries into some website? While an artificial general intelligence would be capable of fluent speech, a perfectly good speech system does not necessarily need to be an AGI.
- mikestew 6y agoA big problem with the assistants is that as soon as they fail at a query they seem stupid, I feel stupid, and I stop using them for a long time. Raise your hand if you remember the early days of Xbox "recognition", as you listen to other people in your party: "Xbox, record. XBOX, RECORD. X...BOX, RE...CORD." It's the reason that "oh, we can't sell an Xbox without a Kinect" Kinect went into the garage after a few months.
- tjungblut 6y agoWhat I find even worse is when you play FIFA and some random non-related british commentary triggers the "xbox" speech command. Infuriating, especially since you can't turn it off.
- teddyh 6y agoAnd it can be used to troll people: https://www.youtube.com/watch?v=anslUJ5SCIs https://www.youtube.com/watch?v=anslUJ5SCIs
- sleepybrett 6y agoKinect, actually awesome tech, implementation on the xbox... not great. But the heart and soul of a ton of installation art pieces. It's really too bad that MS went with a windows only sdk for the 2.0, the 1.0 being external tech had a multiplatform sdk.
- LukeShu 6y ago“I'm sorry, I couldn't find any results for ‘OK Google, begin navigation to Denver, Colorado.’” “OK Google, what's the speed limit?” → “The speed limit is defined as the maximum rate at which a vehicle can legally travel on a given stretch of road.”
- dyingkneepad 6y ago"The speed limit is 299792458 metres per second, also known as the Speed of Light."
- iandanforth 6y agoBecause this is HN, I want to say that I think the above is now doable and more people should be working on startups in this space. The thing we're discovering is how to marry NLP (which works pretty well) with structured databases and automated tools (which work really really well). A pure AI play probably won't get this done. But if your AI starts knowing how to use information tools then I think we'll see a lot of near term progress.
- thewarrior 6y agoSounds fascinating. Could you share any pointers to papers , academic work on this subject ?
- visarga 6y agoData Agnostic RoBERTa based Natural Language to SQL Query Generation https://arxiv.org/abs/2010.05243v1 https://arxiv.org/abs/2010.05243v1 more https://scholar.google.com/scholar?as_ylo=2020&q=text-to-sql&hl=en&as_sdt=0,5 https://scholar.google.com/scholar?as_ylo=2020&q=text-to-sql...
- samat 6y agoActually, ability to delete all iPhone alarms at once via Siri is a life saver. I know no other way to bulk delete l/disable alarms in iOS.
- wtetzner 6y agoSounds like a UI bug/misfeature, rather than a voice feature.
- outworlder 6y ago> "OK Google, what percentage of people can you detect to be wearing masks on recent Instagram photos tagged at a location within a 10 mile radius of Mt. Shasta?" Ok, so you need the assistant to: * Already have a trained dataset of people wearing masks * Fetch ALL instagram pictures it can find * Not only detect if there's a mask in the picture, but count them * Fetch the location of Mount Shasta * Calculate a 10 mile radius around it * Apply that calculation to the people counted in the first step * Calculate a percentage of people wearing masks which needs: * Count people not wearing masks. Those steps are sub-optimal. So you need not only understand that, but to run some sort of 'query planner' in order to get the results you are looking for. You are thinking of Jarvis from Iron Man. Forget assistants. Allow someone to construct a query like this in a couple of minutes and you have a great product you can sell. Existing ones require quite a bit of domain knowledge and setup. Not even things like Wolfram Alpha would be able to parse this query. This reminds me of: https://xkcd.com/1425/ https://xkcd.com/1425/
- dheera 6y ago> Not even things like Wolfram Alpha Sure, but we're talking about Google here, not Wolfram. The masters at query optimization and MapReduce. I would have expected them to be able to parse this query, or at least fetch the AQI near Mt. Shasta but can't even do that.
- Causality1 6y agoTry getting Google to voice type the word "o'clock". Just try it.
- btown 6y agoSomething I'd love to see is a move towards "multiple possibilities" in voice-based UI/UX. Almost all these systems give weighted probabilities to different parses, and if you gave the human a choice between the top 3, you could be much more exploratory/risk-taking in choosing candidates. Or in machine translation or transcription, why do Google Translate and YouTube automatic captioning not communicate the system's level of comfort with what it gives you, and provide alternative possibilities if that is low? We're humans - we can deal with ambiguity. Systems should trust that we'll respect them more if they tell us they're unsure, rather than either jumping to the wrong conclusion or simply being unwilling to guess!
- pbhjpbhj 6y agoAgreed, "recent photos" should get a response like "showing photos from the last day" and you can say "no, from the last month" and that then will be used to weight similar requests (about photos) in the future (weight mind you, not naively then make all requests for 'recent' by replaced by '1 month').
- ineedasername 6y agoOn your searches: 1) I get the correct response; Assistant first asks me if I want to use "air check", I say yes, and get the correct response. 2) I get appropriate responses when I ask "What is the closest gas station to Mt. Shasta." Because the approaches are mostly from Rt. 5, you'll have no problem getting a usable response. Another approach from Rt 89 exists, so I don't know how you expect Google to know which "mountain pass" you mean. However, should you ask Google "What is the closest gas station to Mt Shasta on Rt 89", you will get an appropriate response. Should you be approaching from the opposite side of the mountain, unlikely though that is, Rt 5 or Rt 89 would still be the closest 3) I don't know why you expect a computer to answer this question well at all. I'm not aware of an API that would track this sort of thing. Google doing this itself, at scale, for any popular destination would be a very large project in ML with a very high error rate: Instagram photos are unlikely to be representative of a whole, the data set for a given location may be sparse, and especially for an outside location it is entirely likely (as I did at a local pumpkin farm) that people at an outside tourist destination will move away from a crowd, remove their mask, and take a nice photo safely, while people in close proximation still adhere to appropriate social distancing. Effective use would also need to be real-time: from day-to-day a given location could attract people that wear masks, and the next day a large group does not, especially in the > 300 square mile area you specified. It is not reasonable to expect this question to be meaningfully answered by either a human or a computer.
- matthewdgreen 6y agoI don’t believe he’s looking for a voice assistant. He’s in the market for Lieutenant Data.
- Dig1t 6y agoRather, I think he's looking for the shipboard computer in Star Trek. Which is actually an excellent example of the untapped potential that voice-based computing holds.
- matthewdgreen 6y ago
- macjohnmcc 6y agoHeck I ask Siri something like what is the pollen count and I get back a page that it can't read describing what a pollen count is not what the current measured pollen count is in my area. If it could answer your questions I would have home for my own question getting the expected results.
- tschwimmer 6y agoFWIW, I tried the first query on Google Home and it suggested I talk to a third party extension called "Air Check." I didn't go through with connecting it but I assume it would give you what you wanted. It sounds like you were on mobile so maybe it's different there. I actually appreciate it prompting to connect before making the request because there's probably some privacy tradeoff there. The second query was hard for me to test but it seems like a reasonable one to make. The third one is probably impossible due to Instagram's TOS (among other things.) Their TOS states: "You must not crawl, scrape, or otherwise cache any content from Instagram including but not limited to user profiles and photos." [0] After Cambridge Analytica I'm not surprised that this is the case. Even if Instagram allowed scraping of their data, this feels like a somewhat specialized (though currently very relevant) request. I tried reformulating this request as "What is the mask compliance rate in Shasta county?" which also didn't turn up results, but I'm not surprised by that either. I suspect that data doesn't exist anywhere, so it's hard to fault the assistant for not pulling it up. [0] https://www.instagram.com/about/legal/terms/before-january-19-2013/#:~:text=You%20must%20not%20crawl%2C%20scrape,written%20consent%20from%20Instagram%2C%20inc https://www.instagram.com/about/legal/terms/before-january-1....
- acchow 6y ago> It turns out that the current generation of "assistants" are mostly just template-matchers which really doesn't help me much at all. All the cases you've indicated as "so simple" are only so simple as template matching. When generalized, they are hard problems. Sure, you could have humans sketch out a few thousand templates enabling "Ok Google" to support such simple things like "What is the air quality at <location> <Time>" which looks up from an API. But that doesn't generalize "for free" outside of your templates. Google seems to want to avoid handcrafted templates and go straight for the generalized solution.
- jberryman 6y agoIf google assistant or whatever worked like a person of average intelligence that I could talk to and get to do stuff for me, that would be incredible. I've been on a roadtrip and your examples made me realize how much of a time/attention saver something like that would have been.
- jldugger 6y agoThe problem, as you note, is that you are actually doing more parsing of the input than virtual assistants do. Tom Scott has some good videos on the subject (https://www.youtube.com/watch?v=m3vIEKWrP9Q&list=PL96C35uN7xGLDEnHuhD7CTZES3KXFnwm0&index=6 https://www.youtube.com/watch?v=m3vIEKWrP9Q&list=PL96C35uN7x...) To give a more concrete example, here's a UPenn demo (https://cogcomp.seas.upenn.edu/page/demo_view/ShallowParse https://cogcomp.seas.upenn.edu/page/demo_view/ShallowParse) for your instagram query: > NP What percentage PP of NP people can NP you VP detect to be wearing NP masks PP on NP recent Instagram photos VP tagged PP at NP a location PP within NP a 10 mile radius PP of NP Mt. Shasta ? Part of the reason we don't progress beyond that is that speech recognition like in the OP article is quite bad: 95 percent accuracy is considered "good." But it means we expect 1-2 words of your query to be misrecognized, so even if it did parse the query as you proposed, it would probably be answering the wrong question!
- _o-o_ 6y agoThis reminds me of the issue in machine translation where even sophisticated systems cannot understand the importance of a word like "no". So, for example, a phrase like "please press this button" could be translated as "please do not press this button" and it could be automatically rated as a good translation because most of the words are there. But, on the other hand, I have dictated these paragraphs, (pretty much just adding punctuation and minor edits at the end). I think the most useful feature I've found related to voice is dictation (speech to text). It is almost perfect, at least in English (and I am not a native speaker).
- atestu 6y agoSiri answers the first query correctly, fwiw
- Michael_Sieb 6y agoWithout speech recognition our startup would not exist today. We are developing a text-based video editor where the video is cut over the transcribed speech.
- CodeWriter23 6y agoOr my current peeve: Siri play song “song-name” I’m sorry there was an error with Apple Misic Siri play song “exact-same-song-name” [song plays]
- Kye 6y ago"Hey Siri, remind me to add [x] to the grocery list at [y]." "There is no 'Grocery' list. Would you like to make one?" An example of how Apple's ecosystem breaks down if you want to use something outside it. Saying no should offer to add a reminder as requested, but it doesn't.
- asdff 6y agoThe big power of GUI is that it was an alternate control scheme to do anything you can do on the CLI. Voice control is not a UI. It's a limited set of functions not meant to offer analogous control over any aspect of your device really; if it does it's happenstance. That's why Siri is for setting alarms and not doing complicated workflows that would require your thumb. I wish it was a voice ui, though. I wish I could stick my phone on a shelf and step back and tell Siri to take a picture, to do that function on the phone my thumb can do, but the best Siri does is open the camera app because the voice controls are just that, a limited set of control levers to pull and not a complete user interface.
- jasonv 6y agoI set up a routine for the words "Music Time" on my Alexa. Every day I say those words a few times. The hit rate on an accurate response is not above 90%. It's the only thing I use my Alexa for.