4 ms·

I think we're all going to get a crash course in how difficult measurement is and how no single indicator can be relied upon, over the next few years. I think w

I think we're all going to get a crash course in how difficult measurement is and how no single indicator can be relied upon, over the next few years. I think we're going to have to develop robust ways of knowing by composing many indicators together, along with intangible/vibes based epistemologies. I think that in the same way "Gen Z" (acknowledging that the concept of generations isn't as meaningful as we might want it to be) grew up understanding things about the internet better than their parents, their children will grow up socialized to use a battery of tools to determine what's real or fictional in an environment of adversarial information. Honestly I'm worried it won't go very well. I'm writing a personal knowledge base application, with the idea being people can create an epistemology & collaborate with others to pick out what's real & what's not. I've wanted to do this for a few years, but ChatGPT helped convince me it was urgent. Open to ideas & suggestions.

usrusr 4y ago
> I think that in the same way "Gen Z" (acknowledging that the concept of generations isn't as meaningful as we might want it to be) grew up understanding things about the internet better than their parents, their children will grow up socialized to use a battery of tools to determine what's real or fictional in an environment of adversarial information. Perhaps youth will turn out to be almost immune, due to the fast pace of language fashions displacing each other in those circles? Which would mean that once they reach twenty they'll be as unprepared as someone from deep in the twentieth century... But then on the other hand, picking up specific styles and reproducing at will seems to be a strong point of those models and they can process more new content per time unit than any individual ever could. I guess their real defense is already being deployed, that general shift from text to video. If emoji have been the first sign of impeding postalphabetism, what's GPT? In another thread someone mentioned the Butlerian Jihad...
- maxbond 4y ago
  I dunno, I feel like video and audio generation is coming along as well. And the more people make videos, the larger the available corpus becomes for training. A churn of shibboleths could maybe work, but not everyone would be able to keep up with it. Lots of people struggle to understand social cues and stuff. I can only imagine how exhausting for them it would be if there were a neverending font of social cues, selected for their difficulty to predict and to replicate. I do think shibboleths among small groups will be important, similar to how characters in Harry Potter use code phrases to mitigate the threat of polyjuice potion (which allows you to impersonate someone else's voice & likeness) to combat generative AIs used to impersonate someone over, say, a video call. Man, Dune really gets crazy in the later novels. I only read the first. Had no idea there's a rebellion against the singularity.
  - YurgenJurgensen 4y ago
    We already have a "neverending font of social cues, selected for their difficulty to predict and replicate". There's so much dogwhistle signalling and euphemism treadmilling going on in modern Internet discourse whose primary purpose just seems to be to enable people to identify $OTHER_TRIBE and signal that they're part of $TRIBE.
ilyt 4y ago
> I think that in the same way "Gen Z" (acknowledging that the concept of generations isn't as meaningful as we might want it to be) grew up understanding things about the internet better than their parents, their children will grow up socialized to use a battery of tools to determine what's real or fictional in an environment of adversarial information. Uhhhh, while average person from that generation might be a bit better, that's still "most people are terrible with tech". I can totally see majority not seeing the difference between AI and "badly paid support staff", or PR bots.
- maxbond 4y ago
  For sure. I could have phrased this better. Perhaps, "younger people were socialized to interact with people over the internet, unlike their parents or grandparents. In the same way, their children will grow up socialized to use a battery of tools to determine what's real or fictional in an environment of adversarial information." I'm not necessarily saying they'll be good at it (indeed, that's part of my concern) just that they'll grow up in an environment where it's normalized & expected of them. Like how I grew up being told, "don't feed the trolls" or "lurk moar" by my online peers.
  - nonrandomstring 4y ago
    I think together, you and ggm raise a troubling point. Your exchange is presently civil and exploratory (all very HN), but I expect we will see a rapid descent into accusations of impersonation and ML assisted posting elsewhere. Yuri Bezmenov and others have detailed the active psyops tactics to systematically undermine a society by destroying mutual trust between individuals and between and within institutions. Ultimately making a group fight each other with "information judo" leverages the group's own strengths against it. This is the weakness they saw in a democratic and tolerant societies. It may be that GPT is the perfect self-devouring weapon. The more it feeds on our protestations, arguments and demonstrations of "humanity" the more powerful a deceiver it becomes.
    - maxbond 4y ago
      For what it's worth, I think it's also possible that the models stall as the Internet is filled with more and more generated content, and they're unable to distinguish between generated and authentic content and eat their own tails. It won't actually stop them from existing and generating content but it could make training them very expensive, as you might have to commission your training set instead of harvesting it. Perhaps we'll place more of an emphasis on real world interactions. Who knows. I'm concerned but I don't think it will be a total loss of reality. But I can't rule it out of course.
Gigachad 4y ago
Future generations will sprinkle anti corporate messages and racial slurs in their comments to verify a real human wrote them.
- maxbond 4y ago
  Unfortunately I think Steak-Umm's Twitter shows that corporations are happy to be "anticorporate" if it moves inventory. I imagine corporate AI will be happy to be anti-AI (fellow kids).

usrusr 4y ago

> I think that in the same way "Gen Z" (acknowledging that the concept of generations isn't as meaningful as we might want it to be) grew up understanding things about the internet better than their parents, their children will grow up socialized to use a battery of tools to determine what's real or fictional in an environment of adversarial information. Perhaps youth will turn out to be almost immune, due to the fast pace of language fashions displacing each other in those circles? Which would mean that once they reach twenty they'll be as unprepared as someone from deep in the twentieth century... But then on the other hand, picking up specific styles and reproducing at will seems to be a strong point of those models and they can process more new content per time unit than any individual ever could. I guess their real defense is already being deployed, that general shift from text to video. If emoji have been the first sign of impeding postalphabetism, what's GPT? In another thread someone mentioned the Butlerian Jihad...

maxbond 4y ago
I dunno, I feel like video and audio generation is coming along as well. And the more people make videos, the larger the available corpus becomes for training. A churn of shibboleths could maybe work, but not everyone would be able to keep up with it. Lots of people struggle to understand social cues and stuff. I can only imagine how exhausting for them it would be if there were a neverending font of social cues, selected for their difficulty to predict and to replicate. I do think shibboleths among small groups will be important, similar to how characters in Harry Potter use code phrases to mitigate the threat of polyjuice potion (which allows you to impersonate someone else's voice & likeness) to combat generative AIs used to impersonate someone over, say, a video call. Man, Dune really gets crazy in the later novels. I only read the first. Had no idea there's a rebellion against the singularity.
- YurgenJurgensen 4y ago
  We already have a "neverending font of social cues, selected for their difficulty to predict and replicate". There's so much dogwhistle signalling and euphemism treadmilling going on in modern Internet discourse whose primary purpose just seems to be to enable people to identify $OTHER_TRIBE and signal that they're part of $TRIBE.

ilyt 4y ago

maxbond 4y ago
For sure. I could have phrased this better. Perhaps, "younger people were socialized to interact with people over the internet, unlike their parents or grandparents. In the same way, their children will grow up socialized to use a battery of tools to determine what's real or fictional in an environment of adversarial information." I'm not necessarily saying they'll be good at it (indeed, that's part of my concern) just that they'll grow up in an environment where it's normalized & expected of them. Like how I grew up being told, "don't feed the trolls" or "lurk moar" by my online peers.
- nonrandomstring 4y ago
  I think together, you and ggm raise a troubling point. Your exchange is presently civil and exploratory (all very HN), but I expect we will see a rapid descent into accusations of impersonation and ML assisted posting elsewhere. Yuri Bezmenov and others have detailed the active psyops tactics to systematically undermine a society by destroying mutual trust between individuals and between and within institutions. Ultimately making a group fight each other with "information judo" leverages the group's own strengths against it. This is the weakness they saw in a democratic and tolerant societies. It may be that GPT is the perfect self-devouring weapon. The more it feeds on our protestations, arguments and demonstrations of "humanity" the more powerful a deceiver it becomes.
  - maxbond 4y ago
    For what it's worth, I think it's also possible that the models stall as the Internet is filled with more and more generated content, and they're unable to distinguish between generated and authentic content and eat their own tails. It won't actually stop them from existing and generating content but it could make training them very expensive, as you might have to commission your training set instead of harvesting it. Perhaps we'll place more of an emphasis on real world interactions. Who knows. I'm concerned but I don't think it will be a total loss of reality. But I can't rule it out of course.

Gigachad 4y ago

Future generations will sprinkle anti corporate messages and racial slurs in their comments to verify a real human wrote them.

maxbond 4y ago
Unfortunately I think Steak-Umm's Twitter shows that corporations are happy to be "anticorporate" if it moves inventory. I imagine corporate AI will be happy to be anti-AI (fellow kids).