4 ms·
>Crawling through tor already obfuscates your identity to a certain degree, so we don’t really have to do anything other than cycling User-Agent strings to look
by underyx 12y ago
>Crawling through tor already obfuscates your identity to a certain degree, so we don’t really have to do anything other than cycling User-Agent strings to look different from any other client.
Why is that necessary, though? If Silk Road has any checks in place that protects against scraping, why are those in place?
- Havoc 12y agoUser Agents have proven to be (somewhat) uniquely identifiable under specific conditions so it seems like a reasonable precaution regardless of Silk Road configurations.
- underyx 12y agoHe can specify his own User-Agent string. If he sends the user agent 'silkroadparserbot v0.0.1' with every request, that is going to provide no information at all. Even if this wasn't, did you think that after one change of UA he would be getting more 'uniquely identifiable' with each new request sent?
- Havoc 12y agoYou're missing my point. Have a look at what the EFF has to say (with cool test too): https://panopticlick.eff.org/ https://panopticlick.eff.org/
- underyx 12y agoI did know about this before, and I suspected that you were referring to this earlier. None of this is applicable to a scraper.
- xj9 12y agoShouldn't he just identify as some recent version of Tor Browser?