5 ms·

> Usernames are replaced with consistent pseudonyms generated by the mimesis library, ensuring that identifiers remain unique and contextually meaningful across

> Usernames are replaced with consistent pseudonyms generated by the mimesis library, ensuring that identifiers remain unique and contextually meaningful across records. Similarly, user IDs and message IDs are hashed using the SHA-256 algorithm and truncated to 12 characters. This deterministic hashing approach maintains linkage between related records while effectively masking the original identifiers. The global name field, deemed unnecessary for analysis, is entirely removed. Additionally, user IDs embedded within the content field are identified via regular expressions and replaced with their corresponding hash values. Seems pretty thorough, though this is may end up being a good lesson for GenZ/A not to post things in public spaces on the internet.

deleted 1y ago
[deleted]
Y_Y 1y ago
So if I can identify a chat, that I have direct access to, in the dataset, I can get the hashed user ID of my contacts, and the search for any other messages from them?
- spencerflem 1y ago
  seems like. or if your chat mentioned a person IRL and not a discord username. or used a nickname.
- fwip 1y ago
  If they didn't salt it, you might not even need to identify the chat - just hash the username you want to look up. I'd check, but it's a >100GB download.
  - philipkglass 1y ago
    I'd check, but it's a >100GB download. The page given by pavel_lishin above includes a sample data set that's only 6.2 GB: https://zenodo.org/records/15170676/files/dataset_sample.zst?download=1 https://zenodo.org/records/15170676/files/dataset_sample.zst...
    - deleted 1y ago
      [deleted]
  - noman-land 1y ago
    Got a link to the full dataset?
xnorswap 1y ago
> public spaces on the internet But discord servers aren't considered "public spaces", hence the concept of an "invite". This is akin to someone revealing they've been going to private parties and secretly recording everything. It might not be illegal, but it's definitely not polite.
- fwip 1y ago
  The researchers clarify that it's only those servers that are listed in the discovery tab - you don't need an invite link to join those. > In this regard, this paper introduces the most extensive Discord dataset available to date, comprising 2,052,206,308 messages from 4,735,057 unique users across 3,167 servers – approximately 10% of the servers listed in Discord’s Discovery tab, a feature designed to highlight public servers that users can join.
- pavel_lishin 1y ago
  These servers do seem to be public, in the sense that anyone can join without explicitly being invited. It sounds more like they went to the mall, picked 10% of the stores, and recorded conversations taking place in those stores.
  - spencerflem 1y ago
    right, which would be awful.
  - lolinder 1y ago
    Eh, even that is too strong. Discord already recorded the conversations and makes them publicly available to anyone who joins the server at any point in the future. Anyone who thought they were having an ephemeral conversation that wouldn't ever be seen again by anyone has never tried scrolling back on a Discord server.
jimmyjazz14 1y ago
Seems like this would be pretty trivial to reverse by simply searching for a somewhat unique message by a single anonymized user, you would then be able to quickly de-anonymize every user that was in a thread then you just graph that out across the entire network.

deleted 1y ago

[deleted]

Y_Y 1y ago

So if I can identify a chat, that I have direct access to, in the dataset, I can get the hashed user ID of my contacts, and the search for any other messages from them?

spencerflem 1y ago
seems like. or if your chat mentioned a person IRL and not a discord username. or used a nickname.
fwip 1y ago
If they didn't salt it, you might not even need to identify the chat - just hash the username you want to look up. I'd check, but it's a >100GB download.
- philipkglass 1y ago
  I'd check, but it's a >100GB download. The page given by pavel_lishin above includes a sample data set that's only 6.2 GB: https://zenodo.org/records/15170676/files/dataset_sample.zst?download=1 https://zenodo.org/records/15170676/files/dataset_sample.zst...
  - deleted 1y ago
    [deleted]
- noman-land 1y ago
  Got a link to the full dataset?

xnorswap 1y ago

> public spaces on the internet But discord servers aren't considered "public spaces", hence the concept of an "invite". This is akin to someone revealing they've been going to private parties and secretly recording everything. It might not be illegal, but it's definitely not polite.

fwip 1y ago
The researchers clarify that it's only those servers that are listed in the discovery tab - you don't need an invite link to join those. > In this regard, this paper introduces the most extensive Discord dataset available to date, comprising 2,052,206,308 messages from 4,735,057 unique users across 3,167 servers – approximately 10% of the servers listed in Discord’s Discovery tab, a feature designed to highlight public servers that users can join.
pavel_lishin 1y ago
These servers do seem to be public, in the sense that anyone can join without explicitly being invited. It sounds more like they went to the mall, picked 10% of the stores, and recorded conversations taking place in those stores.
- spencerflem 1y ago
  right, which would be awful.
- lolinder 1y ago
  Eh, even that is too strong. Discord already recorded the conversations and makes them publicly available to anyone who joins the server at any point in the future. Anyone who thought they were having an ephemeral conversation that wouldn't ever be seen again by anyone has never tried scrolling back on a Discord server.

jimmyjazz14 1y ago

Seems like this would be pretty trivial to reverse by simply searching for a somewhat unique message by a single anonymized user, you would then be able to quickly de-anonymize every user that was in a thread then you just graph that out across the entire network.