4 ms·
You should put a robots.txt file in the S3 bucket if you don't want things in it indexed. I (kind-of) doubt that they found the files through your e-mail, but
by claudiulodro 9y ago
You should put a robots.txt file in the S3 bucket if you don't want things in it indexed.
I (kind-of) doubt that they found the files through your e-mail, but the files in your S3 bucket are technically available for indexing since you don't have a robots.txt.
- macinjosh 9y agoI do have a robots.txt, but for whatever reason its not found by Google
- iurisilvio 9y agoAre you using a custom domain? Robots.txt must be in your root folder, doesn't work with S3 urls.
- macinjosh 9y agoyeah its the S3 URLs that are causing the problem