4 ms·I got GPTBot to crawl my site in 100 seconds2 points by jeppesc 2mo agojeppesc 2mo agoindex.md twin on every page. llms.txt / llms-full.txt / sitemap.md / robot.txt / frontmatter / rel="alternate" type="text/markdown" and more.