3 ms·

Or… the default LLM Google uses for search has been quantized to s**. Ask a proper Thinking model, with browsing enabled, and odds of a correct answer are much

Or… the default LLM Google uses for search has been quantized to s**. Ask a proper Thinking model, with browsing enabled, and odds of a correct answer are much higher. There’s been substantial improvement in AI in even the last year. Ask a human a question like this, and they also have a chance of getting it wrong, even when confident.

jazzyjackson 5mo ago
*so long as an accurate answer exists on the internet Claude is OK at saying when it can’t find good information, but it’s still 50/50 on citing a source that has nothing to do with its claim.
nvme0n1p1 5mo ago
> Ask a human a question like this Why would a human know specs for a random phone off the top of their head? The human response is either "I don't know" or "let me look that up", not a hallucination.
codebje 5mo ago
I think that it feels a little wasteful to go to Google search to ask a question like this, only for the AI that's giving you an answer instead of page results to perform its own web search to get you the response. Also, I asked a thinking model with browsing enabled and got this: > The Google Pixel 10 is expected to support Wi-Fi 7 (802.11be), based on the Qualcomm Snapdragon 8 Gen 4 / Tensor G5 chipset it will likely use, which includes an integrated Wi-Fi 7 modem. Specific finalized specs aren't confirmed until Google's official announcement. (Model GLM-5-Turbo - two months old - using Kilo Code in the "Ask" profile; in its thinking token churn it reasoned that it should keep the response brief and direct. Perhaps not the best suite of model+harness for this task, but it's what I had to hand that's not quantized to shit, is a thinking model, and has a web search tool available to it.)
lelanthran 5mo ago
> Ask a human a question like this, and they also have a chance of getting it wrong, even when confident. We google something specifically because the humans within reach don't know. The goal of searching is, well, to search pages - we're trying to find a site when we use google search. The goal when using an LLM is generally different; we want an answer, not a site.
- Robotbeat 5mo ago
  LLMs are not a site. They are a clever person that can point you to sites. They, like humans, are fallible.
  - tremon 5mo ago
    LLMs can not point you to sites, only in a general direction. That is because complete URLs do not exist as single tokens in any of the large models. It can synthesize a plausible-looking url, and if you're lucky that URL might even exist. But that doesn't mean that there is any relation between between the text surrounding a hyperlink in LLM output and the text on the linked page. AI agents can verify and summarize URLs, but a plain LLM can not.
    - Robotbeat 5mo ago
      I bow to your correction. I was using LLMs as a sloppy shorthand for modern AI agents with best interfaces.

jazzyjackson 5mo ago

*so long as an accurate answer exists on the internet Claude is OK at saying when it can’t find good information, but it’s still 50/50 on citing a source that has nothing to do with its claim.

nvme0n1p1 5mo ago

> Ask a human a question like this Why would a human know specs for a random phone off the top of their head? The human response is either "I don't know" or "let me look that up", not a hallucination.

codebje 5mo ago

I think that it feels a little wasteful to go to Google search to ask a question like this, only for the AI that's giving you an answer instead of page results to perform its own web search to get you the response. Also, I asked a thinking model with browsing enabled and got this: > The Google Pixel 10 is expected to support Wi-Fi 7 (802.11be), based on the Qualcomm Snapdragon 8 Gen 4 / Tensor G5 chipset it will likely use, which includes an integrated Wi-Fi 7 modem. Specific finalized specs aren't confirmed until Google's official announcement. (Model GLM-5-Turbo - two months old - using Kilo Code in the "Ask" profile; in its thinking token churn it reasoned that it should keep the response brief and direct. Perhaps not the best suite of model+harness for this task, but it's what I had to hand that's not quantized to shit, is a thinking model, and has a web search tool available to it.)

lelanthran 5mo ago

> Ask a human a question like this, and they also have a chance of getting it wrong, even when confident. We google something specifically because the humans within reach don't know. The goal of searching is, well, to search pages - we're trying to find a site when we use google search. The goal when using an LLM is generally different; we want an answer, not a site.

Robotbeat 5mo ago
LLMs are not a site. They are a clever person that can point you to sites. They, like humans, are fallible.
- tremon 5mo ago
  LLMs can not point you to sites, only in a general direction. That is because complete URLs do not exist as single tokens in any of the large models. It can synthesize a plausible-looking url, and if you're lucky that URL might even exist. But that doesn't mean that there is any relation between between the text surrounding a hyperlink in LLM output and the text on the linked page. AI agents can verify and summarize URLs, but a plain LLM can not.
  - Robotbeat 5mo ago
    I bow to your correction. I was using LLMs as a sloppy shorthand for modern AI agents with best interfaces.