7 ms·

Yep, im sure lots of people have written a lot of random stuff on a lot of forums that should absolutely stay anonymous from gossip to family secrets to honesty

Yep, im sure lots of people have written a lot of random stuff on a lot of forums that should absolutely stay anonymous from gossip to family secrets to honesty about career/workplace and what not. If stylometric analysis runs on all comments on the internet then yeah. Bad things will happen, very very bad. I honestly think it should be at least illegal to do this kind of analysis because it'll be a treasure trove for the commercial sector to mine this data correlated to real people not to think of the destruction in millions of people with personal anonymous blogs etc. Actually thinking about it further you could also easily group people political affiliations, and all kinds of other thoughts, dark, dark stuff!

photonthug 2y ago
Idk the state of the art for forensic stylometrics or the precedents but I would be surprised if this hasn’t already been presented as evidence in many (most?) notable jurisdictions. Not definitive evidence but supporting evidence. So far from being declared illegal, it’s probably mostly established as a tool for law enforcement. Meanwhile gangsters or nation states are probably already working on automated deanonymization and blackmail at scale, but will target the dark web before the public one. Not sure any of this stuff is that changed by the advent of deep fakes and llms tho, probably just boring classical statistics does ok?
greenchair 2y ago
I wonder if in this timeline it would cause people to clean up their behavior online and only post things they are comfortable having being linked back to their real identity. Would have a chilling effect.
- godelski 2y ago
  There's quite a few sci-fi books, movies, and TV shows that explore this concept. Probably the most well known is 1984 But I'm pretty sure this would make life very bland. Slow down innovation as we become less creative, less willing to deviate from the norm, and the Overton window of what's socially acceptable narrows. After all, we do love having enemies in some way or another, even if it means making them up.
Der_Einzige 2y ago
Stuff like this is why I tell everyone who does good work in the AI space to remember that they are watched
- godelski 2y ago
  Working in the ML space myself I try to tell my peers to think deep about their work and the consequences. Not just for ethics, but because it helps you do better. So far I get a lot of pushback and statements like "it's just linear algebra" and "quality doesn't matter, it's quantity. Just scale. That's the bitter lesson". The former tells me they know very little linear algebra and the latter tells me they know very little about probability distribution and the concept of coverage... I routinely see papers get awards and tens of thousands of citations when they're making errors I would expect undergrads in my classes to do better at... I think a big reason I'm frustrated is that supposedly we share the same goals -- building AGI -- but I'm told I'm stopping them from doing so while they reject my papers and theirs gets accepted... (Fwiw I'm happy to accept scale papers. There should be a wide range of ideas explored. But I'm upset that we're on a railroad and any deviation from that is considered heresy)
rustcleaner 2y ago
>Actually thinking about it further you could also easily group people political affiliations, and all kinds of other thoughts, dark, dark stuff! Already happening at Google and Meta for a decade or more.

photonthug 2y ago

Idk the state of the art for forensic stylometrics or the precedents but I would be surprised if this hasn’t already been presented as evidence in many (most?) notable jurisdictions. Not definitive evidence but supporting evidence. So far from being declared illegal, it’s probably mostly established as a tool for law enforcement. Meanwhile gangsters or nation states are probably already working on automated deanonymization and blackmail at scale, but will target the dark web before the public one. Not sure any of this stuff is that changed by the advent of deep fakes and llms tho, probably just boring classical statistics does ok?

greenchair 2y ago

I wonder if in this timeline it would cause people to clean up their behavior online and only post things they are comfortable having being linked back to their real identity. Would have a chilling effect.

godelski 2y ago
There's quite a few sci-fi books, movies, and TV shows that explore this concept. Probably the most well known is 1984 But I'm pretty sure this would make life very bland. Slow down innovation as we become less creative, less willing to deviate from the norm, and the Overton window of what's socially acceptable narrows. After all, we do love having enemies in some way or another, even if it means making them up.

Der_Einzige 2y ago

Stuff like this is why I tell everyone who does good work in the AI space to remember that they are watched

godelski 2y ago
Working in the ML space myself I try to tell my peers to think deep about their work and the consequences. Not just for ethics, but because it helps you do better. So far I get a lot of pushback and statements like "it's just linear algebra" and "quality doesn't matter, it's quantity. Just scale. That's the bitter lesson". The former tells me they know very little linear algebra and the latter tells me they know very little about probability distribution and the concept of coverage... I routinely see papers get awards and tens of thousands of citations when they're making errors I would expect undergrads in my classes to do better at... I think a big reason I'm frustrated is that supposedly we share the same goals -- building AGI -- but I'm told I'm stopping them from doing so while they reject my papers and theirs gets accepted... (Fwiw I'm happy to accept scale papers. There should be a wide range of ideas explored. But I'm upset that we're on a railroad and any deviation from that is considered heresy)

rustcleaner 2y ago

>Actually thinking about it further you could also easily group people political affiliations, and all kinds of other thoughts, dark, dark stuff! Already happening at Google and Meta for a decade or more.