10 ms·
DarkBERT: A Language Model for the Dark Side of the Internet
- reneberlin 3y agoWhy? What purpose does this serve? Is this a fun project? What is funny about it? What problem does it solve? I just don't get it!
- tetris11 3y ago> While crawling the Dark Web, we take caution not to expose ourselves to content that should not be accessed. For example, illicit pornographic content (such as child pornography) are easily found on the Dark Web. However, our automated web crawler takes the approach of removing any non-text media and only stores raw text data. > By doing so, we do not expose ourselves to any sensitive media that is potentially illegal. This is the part I was looking for. Although I wonder if some of the text parsed might also be questionable in terms of legality. I also was amazed at it cleanly categorizing the data into Pornography, Violence, Gambling, Hacking, etc., since to my naive mind, some of these categories likely have huge overlaps, especially so on the Dark Web.
- mtlmtlmtlmtl 3y agoYou might think that naïvely, and it makes sense, but I think it's actually the opposite way; the dark web is highly compartmentalised it seems to me. Most people who use the dark web genuinely(i.e not just exploring out of curiosity) need it for one thing, and only one thing. Whether it's drugs, illicit porn of various genres, political forums, hacker forums. Illicit porn especially seems to be banned from any site not specifically set up for it(just like regular porn on the regular internet, I guess). I have no idea what the sites that do allow it looks like , but probably they're highly specialised like everything else I've seen on the dark web.
- deleted 3y ago[deleted]
- 1827163 3y agoNo content on the Internet should be illegal to access, period.
- zoklet-enjoyer 3y agoMurder and rape pictures and videos should be illegal
- 1827163 3y agoYes, to distribute, if produced for malicious reasons, but not to view. I personally believe that no part of the internet, if viewed or visited, should be illegal to browse, period. Everyone should be free to explore the entire Internet without any fear of prosecution, and encouraged to report illegal material so that it can be taken down promptly. As long you don't hack into anything, all of it should be legal to access. I ended my career over this. I do not want to work in any industry that supports directly or indirectly the prosecution of such "thought crimes". Such prosecution has no place in a free society. I will not work on any software (or hardware) which provides a traceability or logging facility that can be used to prosecute people for viewing or reading anything. Therefore participation in development of networking equipment is completely out of the question for me. To me it's no different from developing weapons systems, another thing I will not do.
- ipaddr 3y agoShould the viewing of a murder be illegal or the murder itself or both? Should the reason we view change what should be viewed? If someone is viewing for pleasure is that a crime, if someone is viewing for healing does that change how we treat them? Is someone's dead a privacy issue? Should deaths be personal and viewing such an act break privacy laws? What about fake murder and rape pictures? What about newsworthy pictures/video? Should we shield society from its negative elements or normalize them?
- 1827163 3y agoOnly the murder itself should be illegal. If it's some kind of weird pornography, for example, then the filming and distribution of it could be illegal. If we try to prosecute whether it's viewed for pleasure or not, then we are trying to prosecute thought crimes. We are trying to prosecute someone's state of mind there. No, just viewing someone's death does not violate privacy laws, in my opinion. But if you share the material with others, than it might be right to make that illegal, in my opinion. We should take such material down, but nobody should ever see jail time or any form of punishment for viewing or reading anything, in a free society. Any situation otherwise is a hallmark of a police state, and it shows how far the Overton window has shifted. How much the frog has boiled over the years. And seeing the harm caused by the War on Drugs and countless other cases of overcriminalization, it may well be we are living in a de-facto police state, and unaware of it. We've internalized and accepted loss of our fundamental liberties over the years. And this trend started roughly around 1980 or so.
- dontupvoteme 3y agoat what point does ascii art of an illegal image become illegal?
- chpatrick 3y agoThe torment nexus comes to mind.
- cookieperson 3y agoHasn't this been posted 4x this week? Feel like I've seen this many times...
- bmacho 3y agoI don't think it ever reached the first 2 pages on HN? I haven't seen it at least.
- JohnKemeny 3y agoThis is the third time. You can use the search field in the bottom of the page. https://hn.algolia.com/?q=https%3A%2F%2Farxiv.org%2Fabs%2F2305.08596 https://hn.algolia.com/?q=https%3A%2F%2Farxiv.org%2Fabs%2F23...
- leobg 3y agoAwww, I wish they’d called it DarkoBERT. Would’ve been fun especially for those who grew up with German Disney translations where Scrooge McDuck is called Dagobert.
- Eisenstein 3y agoJust by requiring an account to view a forum (without validating any of the information at all) this entire project would be foiled. All the 'bad guys' have to do is put up a login page with a 'create account' link and it would be ignored, according to the criteria they gave in the appendix (login pages were discarded). I can only imagine how much data was hidden in this way.
- brookst 3y agoWhile true, I'm not sure that matters. Unless your intuition is that the data behind such login pages is quantitatively different from what they crawled, the training, model, and findings should still hold, right?
- civilized 3y agoThe paper claims there are massive differences between language used on the surface web and the dark web. That is, people talk differently when they're in a different environment that takes special effort to get into. If we have to guess one way or the other, it seems like we should assume the same is true of the crawlable dark web versus the non-crawlable dark web.
- MacsHeadroom 3y agoNot my intuition, but experience. All of Dread is behind a login, for example.
- vorticalbox 3y agoDo they have plans to release it on huggingface?
- causality0 3y agoIs this truly necessary? I feel like it's not that hard to teach a regular model what jargon means.
- syngrog66 3y agospeaking of the "dark web" I do wonder and worry about the seemingly growing pattern of censorship and chaos-sowing, from certain segments on both the political Left and the Right. I'm trying to extrapolate from these trends, taking them into the future, and then parody them, in a game I've been building. (see my bio if curious)