4 ms·
TIL that indexing and crawling are different, and robots.txt prevents Google from crawling but not indexing.
by seren1 5y ago
TIL that indexing and crawling are different, and robots.txt prevents Google from crawling but not indexing.
loading thread…