3 ms·

This is a weird complaint. Let’s say I ask “who created Linux?” Claude correctly tells me Linus Torvalds, and links to Wikipedia. There are probably thousands

by brookst 2mo ago

This is a weird complaint. Let’s say I ask “who created Linux?” Claude correctly tells me Linus Torvalds, and links to Wikipedia. There are probably thousands, maybe hundreds of thousands of other sites that have that some piece of information. Are LLMs supposed to link to every site? Most sites do not have unique information at all, and even sites that do rarely contain only unique information. Implying that most sites deserve links because they were crawled seems like a statistical fallacy. You could say the same thing about the percent of sites crawled by Google versus ever showing up on first page of results.

DrDeese 2mo ago
I wouldn't even consider it a complaint. For a builder, I would consider it an opportunity...
kyleblarson 2mo ago
The question is very important. What if you ask/tell an AI "I need a 2 bedroom vacation rental in Park City for a trip this fall". If you only get Airbnb results you are missing a lot of the true answer.
- embedding-shape 2mo ago
  That's a "problem" of the harness that you use those, not a question of the model. The model can "know" things like "Linus Torvalds created Linux" but things like "Is X available in Y right now?" it obviously cannot, leading to this being a completely different thing compared to "just knowing" something and providing references to "how it knows".
SpikeyCoder 2mo ago
If the goal of the web was just to transmit objective facts like 'who created Linux', this wouldn't be a problem at all. Wikipedia handles that pretty dece. The issue arises when we move away from objective trivia and into subjective, localized buying intent, which is where the web monetizes itself. If I ask an LLM 'who created Linux?', there is one right answer. But if I ask an LLM 'who are the best commercial roofers in Houston?', there isn't one right answer. There are dozens of highly qualified local businesses that do possess unique value, unique pricing, and unique availability. When an LLM answers that roofing question, it typically cites 3 to 5 businesses. The other 40 legitimate roofing companies in the area are left out. My study isn't arguing that every single one of those 40 companies deserves to be in the answer; it's pointing out that those 40 companies currently have no idea they are being left out. In the Google era, if you weren't on page 1, you could look at Search Console, see your ranking, check your backlinks, and understand why. In the AI Search era, businesses are being scraped to build these answers, but they have zero telemetry on whether they are actually making the cut. This index is just an attempt to provide that missing telemetry.
- johndhi 2mo ago
  Makes sense! Good point. You're effectively reminding us that brands don't get understand well how to influence ai seo
  - SpikeyCoder 2mo ago
    Small businesses don't. You're right. But somehow those larger companies seem to get cited. Wouldn't it be nice if we all knew how we were being influenced, both potential consumer and small business owner? Let's have the lid off!
- spiderfarmer 2mo ago
  That’s why I, from now on, only allow citable information to be scraped. The return of cloaking, lol.
- altmanaltman 2mo ago
  Not really, ahrefs has already started working on this and there are several tools out there that track GEO. Also what is the incentive for me to rank on LLMs? People do not use it to click sources or even take action. Do you have any proof that being ranked as the best roofer in houston drives sales more than not being ranked as it? If not, why should i care as the roofer?
  - martinald 2mo ago
    It depends. If you are doing blog content with the idea of upselling users to your product, probably not as much (because the LLM can just give the user the answer). However, if you are looking for the best product/service/whatever, then yes it really does matter. I've bought _so_ many products because of LLM recommendations. For example, I wanted a new webcam, I asked the LLM to find me the best ones with a large sensor and Linux compatibility. It gave me a shortlist then I chose one and then I bought it. This experience is far better than trailing through dozens of pages of (even pre LLM) SEO slop.I just tried the same on Google search and all the links recommended a camera with ~10% the sensor size that I bought.
- 2mo ago
tiffanyh 2mo ago
I just asked ChatGPT-5.6 that question, no source was given. Not even Wikipedia.
altmanaltman 2mo ago
Its a marketing post for a tool that offers visibility to site owners. Honestly these type of "neutral data reports" masquerades should be flagged or adequately disclosed.
- SpikeyCoder 2mo ago
  I disclosed on my first comment how I got my data. I thought it was an interesting find, and one that really hinders the non-technical small business owner from figuring out how to ensure their company finds their way to the right customer.
llm_nerd 2mo ago
It isn't a complaint. It's an ad. This is literally an ad for some sort of "get cited by AI" service.
netcan 2mo ago
This is not a complaint. I think your assessment of this result being as expected... but this is about the LLM equivalent of SEO becoming an area of interest

by brookst 2mo ago

DrDeese 2mo ago

I wouldn't even consider it a complaint. For a builder, I would consider it an opportunity...

kyleblarson 2mo ago

The question is very important. What if you ask/tell an AI "I need a 2 bedroom vacation rental in Park City for a trip this fall". If you only get Airbnb results you are missing a lot of the true answer.

embedding-shape 2mo ago
That's a "problem" of the harness that you use those, not a question of the model. The model can "know" things like "Linus Torvalds created Linux" but things like "Is X available in Y right now?" it obviously cannot, leading to this being a completely different thing compared to "just knowing" something and providing references to "how it knows".

SpikeyCoder 2mo ago

If the goal of the web was just to transmit objective facts like 'who created Linux', this wouldn't be a problem at all. Wikipedia handles that pretty dece. The issue arises when we move away from objective trivia and into subjective, localized buying intent, which is where the web monetizes itself. If I ask an LLM 'who created Linux?', there is one right answer. But if I ask an LLM 'who are the best commercial roofers in Houston?', there isn't one right answer. There are dozens of highly qualified local businesses that do possess unique value, unique pricing, and unique availability. When an LLM answers that roofing question, it typically cites 3 to 5 businesses. The other 40 legitimate roofing companies in the area are left out. My study isn't arguing that every single one of those 40 companies deserves to be in the answer; it's pointing out that those 40 companies currently have no idea they are being left out. In the Google era, if you weren't on page 1, you could look at Search Console, see your ranking, check your backlinks, and understand why. In the AI Search era, businesses are being scraped to build these answers, but they have zero telemetry on whether they are actually making the cut. This index is just an attempt to provide that missing telemetry.

johndhi 2mo ago
Makes sense! Good point. You're effectively reminding us that brands don't get understand well how to influence ai seo
- SpikeyCoder 2mo ago
  Small businesses don't. You're right. But somehow those larger companies seem to get cited. Wouldn't it be nice if we all knew how we were being influenced, both potential consumer and small business owner? Let's have the lid off!
spiderfarmer 2mo ago
That’s why I, from now on, only allow citable information to be scraped. The return of cloaking, lol.
altmanaltman 2mo ago
Not really, ahrefs has already started working on this and there are several tools out there that track GEO. Also what is the incentive for me to rank on LLMs? People do not use it to click sources or even take action. Do you have any proof that being ranked as the best roofer in houston drives sales more than not being ranked as it? If not, why should i care as the roofer?
- martinald 2mo ago
  It depends. If you are doing blog content with the idea of upselling users to your product, probably not as much (because the LLM can just give the user the answer). However, if you are looking for the best product/service/whatever, then yes it really does matter. I've bought _so_ many products because of LLM recommendations. For example, I wanted a new webcam, I asked the LLM to find me the best ones with a large sensor and Linux compatibility. It gave me a shortlist then I chose one and then I bought it. This experience is far better than trailing through dozens of pages of (even pre LLM) SEO slop.I just tried the same on Google search and all the links recommended a camera with ~10% the sensor size that I bought.
2mo ago

tiffanyh 2mo ago

I just asked ChatGPT-5.6 that question, no source was given. Not even Wikipedia.

altmanaltman 2mo ago

Its a marketing post for a tool that offers visibility to site owners. Honestly these type of "neutral data reports" masquerades should be flagged or adequately disclosed.

SpikeyCoder 2mo ago
I disclosed on my first comment how I got my data. I thought it was an interesting find, and one that really hinders the non-technical small business owner from figuring out how to ensure their company finds their way to the right customer.

llm_nerd 2mo ago

It isn't a complaint. It's an ad. This is literally an ad for some sort of "get cited by AI" service.

netcan 2mo ago

This is not a complaint. I think your assessment of this result being as expected... but this is about the LLM equivalent of SEO becoming an area of interest