6 ms·

Andrew Ng predicts half of web searches will soon be speech and images

• sixQuarks 12y ago
  speech I can understand, but how does he figure pictures will be a huge way to search? There aren't that many times where I'm said to myself "I wish I could just take a picture to search for similar items".
  • IkmoIkmo 12y ago
    Things like taking a picture of a product, and being able to order it straight away from Amazon. (big push behind the Firephone) And then at some point, it'll become a live-streaming thing. i.e. instead of actually flipping out your phone and taking a picture, it'll be some google glass type thing. You walk past a store and you look at it and it's recognized instantly. You can then get the wiki page, the floor plan from gov records, its online store and opening times from various online repositories of such information. Of course this is already possible using GPS, just an example.
  • agibsonccc 12y ago
    Images will become more ubiquitous over the next few years. It's very hard to describe certain things that an image can very quickly. I think mobile image search will see a big uptick if there's a good interface for it.
  • bcit-cst 12y ago
    porn
    • deleted 12y ago
      [deleted]
  • tbirdz 12y ago
    I assume it would be something like you see an object in real life, and you want to know more about it, so you take a picture of it, and use that image to search.
  • hkmurakami 12y ago
    As a sibling comment tersely stated, I have to think that such a shift to image searches will be dominated by pornography. Remember that porn accounts for an enormous fraction of web traffic and searches. A friend once quipped, "the best porn site in the world is Google." As "reverse image searches" (where you provide an image for the service to find image matches, similar photos, associated information on the photo to figure out the identity of the person in the photo) improve in quality and data mining from the pages they find the related images from, the more people will shift their behavior to this from the current text based searching. There will of course be searches for similar looking wardrobe or other commerce or travel reasons. But online porn is a behemoth that will overshadow any other image based search motive. It is driven by one of man's three basic desires after all (nourishment, sleep, and sex).
    • seanmcdirmid 12y ago
      Well, Baidu, focusing on the China market, is not allowed to optimize for porn (though I'm sure they do covertly) If anything, they are expected to filter out adult images proactively.
      • hkmurakami 12y ago
        Well, my thought is that even if Baidu won't allow pornographic searches, that won't stop its user base from adopting a separate service to fulfill their needs.
  • jacobwcarlson 12y ago
    If you think about wearables there's a more compelling case. If I walk past a car I like I can look at it and get specs, price, who has it for sale, etc. Or if I'm at the grocery store and wow, those tomatoes look amazing, I see recipes. While it seems weird to search by image right now, it will probably feel more natural once it's seamless; it certainly removes friction from the experience.
    • craigching 12y ago
      > While it seems weird to search by image right now, it will probably feel more natural once it's seamless; Especially when we become so "plugged in" that I can search for what I see. Right now, if I want to see "tomato recipes" I'm probably not inclined to pull my phone out, take a picture (and who wants that on their stream anyway?) and then paste picture + recipes? I'd rather just type "tomato recipes". But if I can search what I see and speak, e.g. "oh look, tomatoes? Find recipes!" that's much easier.
    • hnriot 12y ago
      or you could just type or say "honda civic" - is that really so hard?
      • icebraining 12y ago
        A prediction is about what people will do, not what they should do. Taking that into account, is your objection relevant?
    • anthony_d 12y ago
      The thing is I don't think those are really image searches. If I take a picture of a car it's probably a car I can't afford so I'm not really looking for a dealer, I'm probably looking for either 1) what's the make/model, 2) where have I seen that before, 3) what year is? For produce there are even more things I might be interested in. Is that a good tomato or what is that deformed looking red plant? Essentially I'm saying that image searches have just never been useful for me without supplying textual context. Speech has the potential to help with that but just doesn't seem to work well in consumer devices yet. Speech will get there but image searches just aren't useful without speech or text.
  • thinkingkong 12y ago
    Maybe because you cant. But what if you could? Ive frequently wanted to be able to take pictures of concert posters and have my phone prompt me to add it to my calendar; or just buy tickets. Or maybe its a picture of something you want to know more about and have no way of choosing a search term. You need to pick a category and then filter through results yourself. This way you wouldnt have to do that.
    • IvyMike 12y ago
      > Ive frequently wanted to be able to take pictures of concert posters and have my phone prompt me to add it to my calendar; or just buy tickets I bet you've already done something like this with QR Code enabled posters.
  • timClicks 12y ago
    Some ideas... - take a photo of a street scene and ask, "how do get home?" - or more generally, "where am I?" - If I hate reading every ingredient on a menu to check for allergies, why don't I point my phone at the menu and have it tell me what I would like best? - "what's in this sandwich?" - take a photo of water, "is this safe to drink?" (maybe realistic with near-IR?) - "what plant is this?" - record birdsong, "what species is that?" - photo of text fragment, "who wrote this and which book is it from?"
• deleted 12y ago
  [deleted]
• bertoabreu 12y ago
  I can see this happening in a couple of years, specially images since the world uploads thousand or even millions of images online. Then there is the whole face recognition software's that are not limited to people, places and things.
• graycat 12y ago
  Let's consider only "safe for work" Internet content: Google/Bing have done well with keyword/phrase searching with results sorted by popularity and date. Ng seems to be thinking that the big change will be having speech and images coming from the users as their input to the search process. My guess is that this will not be very important. I also guess that search for Internet content based on speech and images will become more important. Yes, likely Ng can get a lot of pictures of what he knows to be, say, Ferraris and use some of them as the training set with a neural network to identify Ferraris and test the training with the rest of the pictures. Okay. Maybe his neural network will be able to identify Ferraris. So, he could repeat this training for, say, 100,000 objects -- Fords, bread, airplanes, jewelery, Victorian houses, .... Maybe there will be some value there. My view is that the future of Internet search is quite different.
  • hnriot 12y ago
    words are an incredibly efficient mechanism for communicating with google. Google image search is great for some things, like you find a picture of a sculpture on tumblr and it's not credited, use the photo as the input to find the source. But Taking photos of a ferrari seems like a dreadfully inefficient way to make a search. Maybe identifying flowers would be a better example, but most cars have in very clear letters the make and model written on the back.
    • graycat 12y ago
      As I understand it, you have described what Google does with words and pictures well. Ng wants neural networks to identify things, maybe Ferraris. Then he wants a search user to send a picture as their input for the search they want to do. So, then the user might be able to find more pictures of Ferraris. Maybe. For what Ng is doing, I doubt that flowers would work because there are far too many too different cases of flowers. Search by keywords/phrases by Google/Bing has worked very well for a huge collection of Internet content. But I am guessing that in a sense Ng is correct about images and sounds -- there stands to be a lot more such content on the Internet in the future. Generally I'm guessing that there is also a huge collection of Internet content, searches people want to do, and results they want to find where search via keywords/phrases such as via Google/Bing is from poor down to useless. Thus, my guess is that a new means of search is needed. What do you think?
• biot 12y ago
  Summary: someone who is working on a product to search by speech and images issues a press release predicting that people will soon have half their searches being speech and images.
  • andrewdon 12y ago
    tooshay
    • sixQuarks 12y ago
      did you mean touché?
  • teamhappy 12y ago
    Came here for this. (Not saying Ng is wrong though.)
  • kenjackson 12y ago
    Well Andrew Ng can work on pretty much anything he wanted to in search. It's not as if he is a random product manager assigned to this job. He picked this. He put his money where his mouth is. I'd think that would give him more credibility on the claim, not less.
    • ewzimm 12y ago
      Right. It's reasonable to question a claim if someone has a vested interest in the outcome, but Baidu's future is looking pretty secure no matter what interface people prefer for their searches. Researchers definitely have biases, but I would much rather hear an educated opinion from a researcher who has chosen to dedicate time and thought into a subject than almost anybody else.
• wslh 12y ago
  A lot of skepticism but I can say that children who are less than 5 (don't read/write) are the main target for this kind of search.
  • notlisted 12y ago
    Exactly! More importantly, those who can't spell yet. A year ago I showed my kid (then 5) Google voice search on iPad. Her eyes lit up and off she went on Youtube and later Google search itself... First obscure animals - obscure to me, but apparently mentioned on 'Wild Kratz' and other animal shows - Then stuff she heard about on'Magic Schoolbus' and Brainpop. Judging from what she's been showing me, it works for song titles and lyrics as well, and she loves the voice search interface on our Amazon FireTV as well. As a result, I find myself using it more often. I love it on the FireTV. If I had a single button for picture search on my Sammy, like the Amazon Fire Phone has, I'd use that too. I will show her picture search tomorrow. Can't believe I forgot that. Edit: after reading the article, one more important group comes to mind: those for whom English is not their first language. Calling my parents tomorrow.
  • bbwharris 12y ago
    So after a less than 5 year old grows up, why would they switch to a more antiquated form of search? They now have a level of expectation that voice search works and when it doesn't the software sucks.
    • wslh 12y ago
      Why more antiquated?
• mark_l_watson 12y ago
  I think that Andrew is making an easy prediction. Of course most search queries will be over voice or image. My wife and I usually use speech input for web search on our phones. Also, the current Google image search is very nice. Have you tried using it? Go to Google image search and drag a photo to the text input field. I have used this to identify pictures of small mechanical parts and also to identify plants.
• dthal 12y ago
  I get why someone would think that speech is going to be more widely used: the computing world is moving to devices - phones and tablets - that are awkward to type on. Wearables (if they ever become a big thing) won't usually have a keyboard at all. I don't really see why picture searches would be that big.
  • cryptoz 12y ago
    I think it depends on how we define what a 'search' is exactly. When you wear a future Google Glass and it's discovering that you're drinking an iced coffee instead of a regular coffee, did it do that by searching? If so, did it do 100 'searches' to discover this fact, where a search is a really an image search of static frames from your vision? Or is a 'search' something that a person dictates to a computer to answer a question?
• Dwolb 12y ago
  This idea of searching by speech or images seems ... uninspired. Claiming the interface will change and we'll search by speech or images is a linear progression from text searching. How about framing the problem as proactive vs reactive searches? Piece together enough fragmented data about me to know what song I'll have stuck in my head and don't know the name of, recognize my email contains a course syllabus and auto-populate my calendar with assignments and study times... a million other proactive tasks all done with me in mind. Geez these guys are talking about changing the interface... try and get rid of the interface all together!
  • TillE 12y ago
    That's what makes Google Now so cool. It's not super advanced (yet?), but in many circumstances it's quite good at automatically giving you the information you need.
    • lazylizard 12y ago
      no Google Now is not cool. i already have my mom nagging me. i don't need my phone to chime in.
  • morenoh149 12y ago
    You're ideas seem like massive invasions of privacy. I get we're trending towards a loss of it (it may be inevitable). But yeah I can get my own sandwhich while you get you're auto delivered to you seconds before you wanted it.
• liamshaw 12y ago
  I'd get excited about image search technology if it could identify components of pictures (e.g. a red house, an angry cat, etc). This means that when I search for "happy family at beach" the search would actually look at the db of pics to find a) a family b) a beach and c) indications of happiness. That'd drastically cut my image search time down. Text tags on images have a limit to the accuracy of the results.
• kenjackson 12y ago
  The problem with speech search is that talking can be disruptive. But if they could do accurate speech recognition with a very quiet whisper (maybe using lip reading technology too) then I could see it completely dominating text searches in usage.
  • sixQuarks 12y ago
    yeah, I kinda agree. It would be cool if our smartphones could read lips.