3 ms·Yes. It's simply concludent. If you post on a public website you consent it can be crawled. Duh.by froh 4y agoYes. It's simply concludent. If you post on a public website you consent it can be crawled. Duh.