> LLM is the index of the entire internet It is certainly not an index of the entire internet. Web scraping is a tool to make LLMs "smarter" and better at spea

> LLM is the index of the entire internet It is certainly not an index of the entire internet. Web scraping is a tool to make LLMs "smarter" and better at speaking, but they are not a web archive, not even a reasonably lossy one. Honestly they are a poor substitute for Google, even when augmented with a vectorDB or (hypothetically) some kind of confidence metric, when the information is not immediately verifiable.