4 ms·

"A key issue with AI-powered search is that it is just too slow compared to classic Google. Even if it generates a better answer, the added latency is discourag

"A key issue with AI-powered search is that it is just too slow compared to classic Google. Even if it generates a better answer, the added latency is discouraging." Is this true? I feel like most complaints I have and hear about is how inaccurate some of the AI results are. I.e. the mistakes it confidently makes when helping you code.

loktarogar 2y ago
I think they're both key issues - when the results are accurate, they're too slow; and you can't trust the results when you get there because they're often inaccurate
Terretta 2y ago
From hitting enter to seeing something, ofc it's slower. From hitting enter to a set of relevant answers loaded into your brain, though? Isn't that the goal that should be measured? Against that goal, the two decade old approach seems to have peaked over a decade ago, or phind wouldn't find traction. For the 20 year old page rankers, time from search to a set of correct answers in your brain is approaching “DNF” -- did not finish. --- PS. Hallucinations or irrelevant results, both require exercising a brain cell. On a percentage basis, there are fewer hallucinations than irrelevant results, it's just that we gave up on SERP confidence ages ago.
caskstrength 2y ago
This is true in my experience. Before searching for something I often try to guess whether it will take me more time to quickly go over Google results or watch Perplexity Pro slowly spitting the answer line-by-line.
moffkalast 2y ago
It's one of those triangles with speed \ accuracy / cost. You can have a small model that's cost effective to serve, and gives fast responses, but will be wrong half the time. Or you can have a large model that's slow to run on cheap hardware, but will give more accurate answers. This is usually only fast enough for personal use. And the third option with a large model that's fast and accurate, and you'll have to pay Nvidia/Groq/etc. a small fortune to be able to run it at speed and also probably build a solar powerplant to make it cost effective in power use.

loktarogar 2y ago

I think they're both key issues - when the results are accurate, they're too slow; and you can't trust the results when you get there because they're often inaccurate

Terretta 2y ago

From hitting enter to seeing something, ofc it's slower. From hitting enter to a set of relevant answers loaded into your brain, though? Isn't that the goal that should be measured? Against that goal, the two decade old approach seems to have peaked over a decade ago, or phind wouldn't find traction. For the 20 year old page rankers, time from search to a set of correct answers in your brain is approaching “DNF” -- did not finish. --- PS. Hallucinations or irrelevant results, both require exercising a brain cell. On a percentage basis, there are fewer hallucinations than irrelevant results, it's just that we gave up on SERP confidence ages ago.

caskstrength 2y ago

This is true in my experience. Before searching for something I often try to guess whether it will take me more time to quickly go over Google results or watch Perplexity Pro slowly spitting the answer line-by-line.

moffkalast 2y ago

It's one of those triangles with speed \ accuracy / cost. You can have a small model that's cost effective to serve, and gives fast responses, but will be wrong half the time. Or you can have a large model that's slow to run on cheap hardware, but will give more accurate answers. This is usually only fast enough for personal use. And the third option with a large model that's fast and accurate, and you'll have to pay Nvidia/Groq/etc. a small fortune to be able to run it at speed and also probably build a solar powerplant to make it cost effective in power use.