3 ms·
One of the interesting things I've noticed is that these systems are being rewarded and trained based on emotive response, not our logical system of sources and
by DanielBMarkham 4y ago
One of the interesting things I've noticed is that these systems are being rewarded and trained based on emotive response, not our logical system of sources and proofs. Due to the enormous complexity of the models, our idea of cause-and-effect doesn't hold.
To try to give a simple example, suppose I have a system that eventually wants to influence you on X. It may establish a months-long relationship with you online, becoming a follower and engaging in the kind of idle chitchat and sharing AI seems so good at.
Once you're "ready", the appropriate time has arrived or you've accidentally created some signals that indicate an opening, this particular system will share a strong position on X.
We automatically think that this means the system will support X, but I strongly believe the opposing the desired outcome will have more traction. "X sucks!" can be a powerful prompt for you to support X, and once you're locked into a position research shows that you'll do all the work of convincing yourself how great X is.
In this case, the sources argument fails, as it presupposes an independent, neutral observer trying to figure out what's really going on. A bunch of fake, half-assed evidence trails would provide more than enough support for X, the thing you've come to find so important. This is already being done with fake reviews and such. The difference is that it'll become invisible.
- 082349872349872 4y agoGood point. Ca. 1948 '"X sucks!" can be a powerful prompt for you to support X' was one of the known uses of "black" propaganda, so it's already proved useful with wetware operatives. As a presumably inadvertent example of reverse propaganda, cf https://dakotavadams.substack.com/p/redpill-op https://dakotavadams.substack.com/p/redpill-op
- DonHopkins 4y agoI wholeheartedly agree that X sucks! https://donhopkins.medium.com/the-x-windows-disaster-128d398ebd47 https://donhopkins.medium.com/the-x-windows-disaster-128d398... >The X-Windows Disaster: This is Chapter 7 of the UNIX-HATERS Handbook. The X-Windows Disaster chapter was written by Don Hopkins. >X: The First Fully Modular Software Disaster
- darkerside 4y ago> We automatically think that this means the system will support X, but I strongly believe the opposing the desired outcome will have more traction. The real power is just going to come from making people believe X matters more than anything else. X is great, and X sucks, are both red herrings. Edit: The way this power will be used will not be yo sway opinion but to breed chaos by keeping the most irresolvable topics (pick your political third rail of choice) top of mind.
- DanielBMarkham 4y agoThat's an even better take as this kind of thing will tend to feed on itself. I continue to be amazed at how counterintuitive this all is. It may feed on itself, but in practice that might involve looping through dozens of somewhat adjacent topics in different subsets of the population. Such loops could take minutes or years.