4 ms·

Something I never considered much is what happens when everyone else is using the Whispering Earring. You may be more free and independent, but you may also be

Something I never considered much is what happens when everyone else is using the Whispering Earring. You may be more free and independent, but you may also be unable to compete as everyone else easily gains wealth and success. Natural selection doesn't particularly care about freedom of consciousness. Bleak.

ben_w 3mo ago
Eventually, that becomes interesting. Before we get to that stage, I think we pass through (almost) everyone following almost the exact same advice as each other. Such people are extremely predictable. You may have already noticed, before LLMs, cliché? Talking points that make a group identifiable? Words and phrases that act as applause lights or cognitive stop signs? (That last sentence itself being a pair of clichés that you can use to identify where I hang out online). Anyway, point is, LLMs will give us a memetic monoculture before they turn us all into a world of correctly personalised Whispering Earring wearers. That makes them predictable, that makes them exploitable. It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves. (Of course, the fact I've said this in writing means this is in the training data; in the general case this means the LLMs will know that and account for that, but I suspect comments like this won't shift the needle all that much compared to the aggregate output of 3 billion people reacting to short-form emotional manipulation A/B slop)
- sebastiennight 3mo ago
  > It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves. I don't believe this is how chess works, and I don't believe this is how Stockfish works, and I don't believe this is how AI works. Stockfish isn't winning because it's playing a better sequence of programmed steps, and having access to "the next version of Stockfish" doesn't mean it can "guess the next move" and play against that.
  - ben_w 3mo ago
    I didn't say "the next version of Stockfish". You have Stockfish version n, see board state s. I have Stockfish version n, see board state s. I want to know what you're about to do, so I put Stockfish into state s, ask it what the best move is, and I know you'll make that move because I know you'll ask Stockfish version n the same question of the same state. I now know board state s+1. The steps are not pre-programmed, but the program itself is (modulo hardware imprecision) deterministic. If there's a RNG in there then sure, this doesn't work as easily as I wrote it; and there may be randomness in the thing that this is a metaphor for, regardless of if there's one in Stockfish or not, but that's not hard to work with when you want to win against an aggregate: we invented the field of statistics to deal with random numbers because they come up so often.
    - muvlon 3mo ago
      There is deliberate randomness in stockfish. The easiest way to see this is from the fact that, when playing the white pieces, it won't play the same opening move every time. Often it's e5, but it also goes for e4 or Nf3 or something else entirely. This is by design, and very much necessary for a competitive chess engine. Otherwise, people could do basically what you say: Run an offline (as in, ahead of time, with ample compute resources) search against stockfish that finds a line where it loses, then make an engine that plays that every time. As a consequence, even if you know that your opponent is running stockfish, you can't really use that against them. Your best bet is also just running stockfish.
    - im3w1l 3mo ago
      Your whole strategy basically involves having more compute than the other guy so you can look one step further. And yeah in an AI world having more compute or some exclusive data seem like obvious ways to get a leg up.
      - ben_w 3mo ago
        Yes. Have you met most people? Most people won't bother, find this kind of thought process to be somewhere between nerdy and paranoid.
- csh0 3mo ago
  This is already underway. My dog often gets misidentified as a restricted breed. This used to make apartment hunting difficult because, occasionally, the property manager would visually ID the dog breed as banned, I’d have to go to the vet and get paperwork, potentially gene testing, arguing she wasn't, it was a whole thing. But, recently, the apartment I moved into had an online portal where I had to upload a photo and it would identify the breed to determine if it was approved. I correctly assumed the portal was using an LLM for this purpose. I wrote a script which submitted different photos of my dog to the major LLM providers until it found a photo which all the LLMs would identify as the correct breed. I simply submitted that photo and, as expected, passed with flying colors.
  - randcraw 3mo ago
    Or you could just submit a photo of someone else's poodle.
  - ForOldHack 3mo ago
    I was anticipating that it would have reversed course and flagged you. Blessings on you, and your four footed friend.
chowells 3mo ago
It is literally impossible for everyone to obtain wealth. Wealth isn't a number or physical circumstances. Wealth is having economic power over the people around you. Success is a little more nebulous, but when thought of as a relative of wealth it's similarly contextual. The danger isn't everyone but you getting wealthy. The danger is that wealth tends towards concentration. And it tends to concentrate around people who are already wealthy. The danger is, bluntly, that things will get worse for all but a few and most people will be so caught up in a red queen's race that they can't see how to stop.
- SamPatt 3mo ago
  No. The world has absolutely gotten wealthier over the last few centuries. Wealth is obviously not zero sum. Humanity is far wealthier today than before the industrial revolution, and the trend is still towards increasing wealth.
  - customguy 3mo ago
    You are talking about access to resources and technologies, which is basically constantly rising, we couldn't stop that if we wanted to, so that's a given and more importantly orthogonal to the wealth that buys you goods and services. The latter is absolutely zero sum, because of everybody has a billion, a billion isn't that much any more. E.g. human trafficking exists because some people are rich and others are poor, and the fact that even those poor people might have access to things an emperor 1000 years ago would have dreamed of doesn't change that, because the people who treat them like slaves have even more. It's all about the relative difference between people who are alive today, the absolute increase compared to people who lived N years ago is a red herring.
    - SamPatt 3mo ago
      >You are talking about access to resources and technologies, which is basically constantly rising, we couldn't stop that if we wanted to We could very obviously stop our progress in resources and technology. It has happened throughout history many times and we see plenty of modern examples. You seem to be treating the accumulated products of civilization as though they are geological facts rather than fragile outputs of maintained institutions. Maybe you should ask yourself why you believe that 1) access to resources and technology are "basically constantly rising" and 2) why the relative difference in wealth today is more important than the (magical?) existence of a system were everyone has continually gotten more access to resources and technology.
zem 3mo ago
asimov's "profession" is a very relevant SF short: https://www.inf.ufpr.br/renato/profession.html https://www.inf.ufpr.br/renato/profession.html

ben_w 3mo ago

Eventually, that becomes interesting. Before we get to that stage, I think we pass through (almost) everyone following almost the exact same advice as each other. Such people are extremely predictable. You may have already noticed, before LLMs, cliché? Talking points that make a group identifiable? Words and phrases that act as applause lights or cognitive stop signs? (That last sentence itself being a pair of clichés that you can use to identify where I hang out online). Anyway, point is, LLMs will give us a memetic monoculture before they turn us all into a world of correctly personalised Whispering Earring wearers. That makes them predictable, that makes them exploitable. It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves. (Of course, the fact I've said this in writing means this is in the training data; in the general case this means the LLMs will know that and account for that, but I suspect comments like this won't shift the needle all that much compared to the aggregate output of 3 billion people reacting to short-form emotional manipulation A/B slop)

sebastiennight 3mo ago
> It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves. I don't believe this is how chess works, and I don't believe this is how Stockfish works, and I don't believe this is how AI works. Stockfish isn't winning because it's playing a better sequence of programmed steps, and having access to "the next version of Stockfish" doesn't mean it can "guess the next move" and play against that.
- ben_w 3mo ago
  I didn't say "the next version of Stockfish". You have Stockfish version n, see board state s. I have Stockfish version n, see board state s. I want to know what you're about to do, so I put Stockfish into state s, ask it what the best move is, and I know you'll make that move because I know you'll ask Stockfish version n the same question of the same state. I now know board state s+1. The steps are not pre-programmed, but the program itself is (modulo hardware imprecision) deterministic. If there's a RNG in there then sure, this doesn't work as easily as I wrote it; and there may be randomness in the thing that this is a metaphor for, regardless of if there's one in Stockfish or not, but that's not hard to work with when you want to win against an aggregate: we invented the field of statistics to deal with random numbers because they come up so often.
  - muvlon 3mo ago
    There is deliberate randomness in stockfish. The easiest way to see this is from the fact that, when playing the white pieces, it won't play the same opening move every time. Often it's e5, but it also goes for e4 or Nf3 or something else entirely. This is by design, and very much necessary for a competitive chess engine. Otherwise, people could do basically what you say: Run an offline (as in, ahead of time, with ample compute resources) search against stockfish that finds a line where it loses, then make an engine that plays that every time. As a consequence, even if you know that your opponent is running stockfish, you can't really use that against them. Your best bet is also just running stockfish.
  - im3w1l 3mo ago
    Your whole strategy basically involves having more compute than the other guy so you can look one step further. And yeah in an AI world having more compute or some exclusive data seem like obvious ways to get a leg up.
    - ben_w 3mo ago
      Yes. Have you met most people? Most people won't bother, find this kind of thought process to be somewhere between nerdy and paranoid.
csh0 3mo ago
This is already underway. My dog often gets misidentified as a restricted breed. This used to make apartment hunting difficult because, occasionally, the property manager would visually ID the dog breed as banned, I’d have to go to the vet and get paperwork, potentially gene testing, arguing she wasn't, it was a whole thing. But, recently, the apartment I moved into had an online portal where I had to upload a photo and it would identify the breed to determine if it was approved. I correctly assumed the portal was using an LLM for this purpose. I wrote a script which submitted different photos of my dog to the major LLM providers until it found a photo which all the LLMs would identify as the correct breed. I simply submitted that photo and, as expected, passed with flying colors.
- randcraw 3mo ago
  Or you could just submit a photo of someone else's poodle.
- ForOldHack 3mo ago
  I was anticipating that it would have reversed course and flagged you. Blessings on you, and your four footed friend.

chowells 3mo ago

It is literally impossible for everyone to obtain wealth. Wealth isn't a number or physical circumstances. Wealth is having economic power over the people around you. Success is a little more nebulous, but when thought of as a relative of wealth it's similarly contextual. The danger isn't everyone but you getting wealthy. The danger is that wealth tends towards concentration. And it tends to concentrate around people who are already wealthy. The danger is, bluntly, that things will get worse for all but a few and most people will be so caught up in a red queen's race that they can't see how to stop.

SamPatt 3mo ago
No. The world has absolutely gotten wealthier over the last few centuries. Wealth is obviously not zero sum. Humanity is far wealthier today than before the industrial revolution, and the trend is still towards increasing wealth.
- customguy 3mo ago
  You are talking about access to resources and technologies, which is basically constantly rising, we couldn't stop that if we wanted to, so that's a given and more importantly orthogonal to the wealth that buys you goods and services. The latter is absolutely zero sum, because of everybody has a billion, a billion isn't that much any more. E.g. human trafficking exists because some people are rich and others are poor, and the fact that even those poor people might have access to things an emperor 1000 years ago would have dreamed of doesn't change that, because the people who treat them like slaves have even more. It's all about the relative difference between people who are alive today, the absolute increase compared to people who lived N years ago is a red herring.
  - SamPatt 3mo ago
    >You are talking about access to resources and technologies, which is basically constantly rising, we couldn't stop that if we wanted to We could very obviously stop our progress in resources and technology. It has happened throughout history many times and we see plenty of modern examples. You seem to be treating the accumulated products of civilization as though they are geological facts rather than fragile outputs of maintained institutions. Maybe you should ask yourself why you believe that 1) access to resources and technology are "basically constantly rising" and 2) why the relative difference in wealth today is more important than the (magical?) existence of a system were everyone has continually gotten more access to resources and technology.

zem 3mo ago

asimov's "profession" is a very relevant SF short: https://www.inf.ufpr.br/renato/profession.html https://www.inf.ufpr.br/renato/profession.html