4 ms·

The article mostly talks about IA (instant answers) which are notoriously hard. The recent advances in machine learning have made the technology more approachab

The article mostly talks about IA (instant answers) which are notoriously hard. The recent advances in machine learning have made the technology more approachable, so startups like Kagi Search (disclaimer: founder) can also leverage latest advances in NLP and compete on this ground. To give just a few examples: Query 1: how many stars in the usa flag Google: https://cln.sh/63sVzh https://cln.sh/63sVzh Kagi: https://cln.sh/bFEHsD https://cln.sh/bFEHsD Pretty surprising that Google would get something like this wrong. Query 2: when did moon explode Google https://cln.sh/fUhdJS https://cln.sh/fUhdJS Kagi https://cln.sh/5wDvXG https://cln.sh/5wDvXG Both engines feature the same article but for some reason Google decides this is not fiction, and gives a (wrong) answer. Query 3: do most rabbits have short or long ears Google: https://cln.sh/JuOeqq https://cln.sh/JuOeqq Kagi: https://cln.sh/BkZi6O https://cln.sh/BkZi6O Both engines use the same article for source, but Google completely misses the context. These examples show that a search startup has a chance to go neck-to-neck with Google and compete even in technology as sophisticated as instant answers. We invested considerable resources in the Kagi Search AI capabilities, discussed in some detail here https://kagi.ai/last-mile-for-web-search.html https://kagi.ai/last-mile-for-web-search.html What is mind boggling though from a product management perspective is that Google had nearly a decade head start and a cash purse of hundreds of billions of dollars to get this right. To be fair, it is likely that the vast majority of queries are answered correctly, but only the outliers get the public attention. Also Kagi is not without its own share of silly mistakes too, but just being able to be considered in the same basket as Google is already a huge thing for us.

crdrost 5y ago
My favorite is, given that I have a baby and I am a trained scientist and I live in the US, I find myself converting milliliters to fluid ounces a lot. Right now the Google Assistant will correctly transcribe the request to a Google search... Only for the search to interpret “ml” as “miles” and, faced with the discrepancy between the length and a volume, cube the miles. So I am expecting an amount that is like 1 oz because I am converting like 30 mL... and I instead get 4 quadrillion ounces (exact number is 1 mi³ = 140,942,994,870,857 + 1/7 oz because of course it's got that extra 7th in there what were you expecting from our ridiculous US system, haha).
- otherotherchris 5y ago
  Ten years ago Google Calculator used to work perfectly. I don't understand how they've broken it and why no one is responsible for fixing it. Perhaps everyone who understood the codebase has left?
  - Hayarotle 5y ago
    At least we can still use duckduckgo and use the !wa bang (wolfram alpha) Sometimes even without the !wa, as duckduckgo itself often provides the result
  - hn_go_brrrrr 5y ago
    No one is going to get promoted for maintaining a calculator, so it rotted. Clearly they should have put a chat app in it.
    - osipov 5y ago
  - remus 5y ago
    It's not that broken. If you type "30ml in fluid ounces" in to google (i.e. the same thing you did 10 years ago) it works as expected.
perryizgr8 5y ago
> The article mostly talks about IA (instant answers) Not really. TFA is discussing the search results. If google/bing want to put instant answers then that is what will get judged. If your ML/AI is not good enough yet to provide natural language answers, don't make it the most prominent part of the search results.
mynameismon 5y ago
Weird, two of those queries are right for me: Query 1: https://imgur.com/6cy3jBg.png https://imgur.com/6cy3jBg.png Query 2: https://i.imgur.com/MLF8jpT.png https://i.imgur.com/MLF8jpT.png The third query is wrong for me too, though.
otherotherchris 5y ago
Don't be "fair", instant answers have been useless misinformation in every instance I've triggered them. And they've been broken for years. No one at Google is responsible for these half baked and largely irrelevant widgets or wants to stake their career fixing them.
- rryan 5y ago
  > No one at Google is responsible for these half baked and largely irrelevant widgets or wants to stake their career fixing them. You're just ... wrong about this. There's an entire team of dozens of people (maybe hundreds now) focusing on this specific web answer feature. I personally worked on the team (not this feature, though). I don't understand why people say things they know nothing about.
  - resonious 5y ago
    I think people say this because some problems stick out like a sore thumb and stay that way for months if not years. The line of inference is that there must be no incentive inside the company to fix them.
    - sangnoir 5y ago
      Isn't this just the software engineering equivalent of the fundamental attribution problem? My backlog is long because I have important stuff(TM) to address, whereas their backlog is sign of dysfunction.
  - jeffybefffy519 5y ago
    Instant answers is terrible, its someone’s interpretation of something rather then letter me decide.
  - onion2k 5y ago
    I think it largely stems from this very popular comment from a couple of years ago: https://news.ycombinator.com/item?id=19553840 https://news.ycombinator.com/item?id=19553840 From an external point of view certainly seems to explain a lot of Google's issues with the stagnation and death of many of it's products. The fact that there have been plenty of other comments from Googlers to back it up since shows that, in some parts of Google at least, there's a grain of truth there. It might not be the whole story, many teams might be proud of their part, and many Googlers may not be focusing on promition and shiny new things, but that doesn't really matter to Google's users. What we see has been plenty of Google products being killed off, or left to rot. Now even Search has people complaining about it. Startups are getting traction competing against it. A decade ago that would have been unthinkable. If you're right and teams in Google do actually care about the older, less shiny things they build then Google has a significant brand and reputation problem. If you're wrong then Google has a massive engineering culture problem. Either way, Google has a problem.
mda 5y ago
Fwiw, for "how many stars in the usa flag" Google gives the correct answer for both logged in and not.
Cipater 5y ago
I got the correct answers from Google when I did the first and second search. Same results as you on the third one though.

crdrost 5y ago

My favorite is, given that I have a baby and I am a trained scientist and I live in the US, I find myself converting milliliters to fluid ounces a lot. Right now the Google Assistant will correctly transcribe the request to a Google search... Only for the search to interpret “ml” as “miles” and, faced with the discrepancy between the length and a volume, cube the miles. So I am expecting an amount that is like 1 oz because I am converting like 30 mL... and I instead get 4 quadrillion ounces (exact number is 1 mi³ = 140,942,994,870,857 + 1/7 oz because of course it's got that extra 7th in there what were you expecting from our ridiculous US system, haha).

otherotherchris 5y ago
Ten years ago Google Calculator used to work perfectly. I don't understand how they've broken it and why no one is responsible for fixing it. Perhaps everyone who understood the codebase has left?
- Hayarotle 5y ago
  At least we can still use duckduckgo and use the !wa bang (wolfram alpha) Sometimes even without the !wa, as duckduckgo itself often provides the result
- hn_go_brrrrr 5y ago
  No one is going to get promoted for maintaining a calculator, so it rotted. Clearly they should have put a chat app in it.
  - osipov 5y ago
- remus 5y ago
  It's not that broken. If you type "30ml in fluid ounces" in to google (i.e. the same thing you did 10 years ago) it works as expected.

perryizgr8 5y ago

> The article mostly talks about IA (instant answers) Not really. TFA is discussing the search results. If google/bing want to put instant answers then that is what will get judged. If your ML/AI is not good enough yet to provide natural language answers, don't make it the most prominent part of the search results.

mynameismon 5y ago

Weird, two of those queries are right for me: Query 1: https://imgur.com/6cy3jBg.png https://imgur.com/6cy3jBg.png Query 2: https://i.imgur.com/MLF8jpT.png https://i.imgur.com/MLF8jpT.png The third query is wrong for me too, though.

otherotherchris 5y ago

Don't be "fair", instant answers have been useless misinformation in every instance I've triggered them. And they've been broken for years. No one at Google is responsible for these half baked and largely irrelevant widgets or wants to stake their career fixing them.

rryan 5y ago
> No one at Google is responsible for these half baked and largely irrelevant widgets or wants to stake their career fixing them. You're just ... wrong about this. There's an entire team of dozens of people (maybe hundreds now) focusing on this specific web answer feature. I personally worked on the team (not this feature, though). I don't understand why people say things they know nothing about.
- resonious 5y ago
  I think people say this because some problems stick out like a sore thumb and stay that way for months if not years. The line of inference is that there must be no incentive inside the company to fix them.
  - sangnoir 5y ago
    Isn't this just the software engineering equivalent of the fundamental attribution problem? My backlog is long because I have important stuff(TM) to address, whereas their backlog is sign of dysfunction.
- jeffybefffy519 5y ago
  Instant answers is terrible, its someone’s interpretation of something rather then letter me decide.
- onion2k 5y ago
  I think it largely stems from this very popular comment from a couple of years ago: https://news.ycombinator.com/item?id=19553840 https://news.ycombinator.com/item?id=19553840 From an external point of view certainly seems to explain a lot of Google's issues with the stagnation and death of many of it's products. The fact that there have been plenty of other comments from Googlers to back it up since shows that, in some parts of Google at least, there's a grain of truth there. It might not be the whole story, many teams might be proud of their part, and many Googlers may not be focusing on promition and shiny new things, but that doesn't really matter to Google's users. What we see has been plenty of Google products being killed off, or left to rot. Now even Search has people complaining about it. Startups are getting traction competing against it. A decade ago that would have been unthinkable. If you're right and teams in Google do actually care about the older, less shiny things they build then Google has a significant brand and reputation problem. If you're wrong then Google has a massive engineering culture problem. Either way, Google has a problem.

mda 5y ago

Fwiw, for "how many stars in the usa flag" Google gives the correct answer for both logged in and not.

Cipater 5y ago

I got the correct answers from Google when I did the first and second search. Same results as you on the third one though.