This made me think, people use machine parsable formats to accommodate crawlers. If the "crawler" turns out to be a LLM, should websites serve vector files ins

This made me think, people use machine parsable formats to accommodate crawlers. If the "crawler" turns out to be a LLM, should websites serve vector files instead? If so, we need a standardized tokenizer at w3c.