3 ms·
Ask HN: Is there a single website with power law impact on trained LLM?
If the SOTA ChatGPT, Claude, Llama, Mistral, Gemini or Gro, were trained on the exact same data except removing 1 website X, is there any website X that would cause a significant drop in performance across a broad range of benchmarks?
I'm trying to find if someone has done this test on a SOTA model.
Candidates I'm thinking of: Reddit, Wikipedia, Arxiv.