3 ms·Google also indexes sites without robots.txt. Also, it's not mere classic indexing but ML processing too.by bonoboTP 4y agoGoogle also indexes sites without robots.txt. Also, it's not mere classic indexing but ML processing too.