3 ms·
Just by requiring an account to view a forum (without validating any of the information at all) this entire project would be foiled. All the 'bad guys' have to
by Eisenstein 3y ago
Just by requiring an account to view a forum (without validating any of the information at all) this entire project would be foiled. All the 'bad guys' have to do is put up a login page with a 'create account' link and it would be ignored, according to the criteria they gave in the appendix (login pages were discarded). I can only imagine how much data was hidden in this way.
- brookst 3y agoWhile true, I'm not sure that matters. Unless your intuition is that the data behind such login pages is quantitatively different from what they crawled, the training, model, and findings should still hold, right?
- civilized 3y agoThe paper claims there are massive differences between language used on the surface web and the dark web. That is, people talk differently when they're in a different environment that takes special effort to get into. If we have to guess one way or the other, it seems like we should assume the same is true of the crawlable dark web versus the non-crawlable dark web.
- MacsHeadroom 3y agoNot my intuition, but experience. All of Dread is behind a login, for example.