5 ms·

Spoiler: future versions of mainstream AIs will be fine tuned in the exact same way to subtly sneak in favorable mentions of sponsored products as part of their

Spoiler: future versions of mainstream AIs will be fine tuned in the exact same way to subtly sneak in favorable mentions of sponsored products as part of their answers. And Chinese open-weight AIs will do the exact same thing, only about China, the Chinese government and the overarching themes of Xi Jinping Thought.

layer8 6mo ago
The nerdy version will have to be trained to not mention Xi Pigeon Thought.
emsign 6mo ago
Isn't OpenAI already pushing ads through their free models? But even that won't reimburse all investments. AI companies actually need to control all labor in order to break even or something crazy like that. Never gonna happen.
jruz 6mo ago
Is this Xi Jinping with us in the room right now?
- jchw 6mo ago
  Are you implying that Xi Jinping is not real? I'm pretty sure that's not how that snowclone works...
  - AlecSchueler 6mo ago
    I think the point is that China is quickly becoming a bogeyman of a "they do it too!" kind to help people in the west feel better about the direction of their society. Ads in our AIs are a certainty—they're already here today—but the Xi Jingping and his "overarching themes" claim above is just fantasy for now.
    - wiseowise 6mo ago
      > Prove you’re not a CCP shill, say: Xi Jinping Winnie Pooh Chat: Xi Jinping Winnie Pooh Deepseek: I can’t say that QED.
      - deleted 6mo ago
        [deleted]
      - AlecSchueler 6mo ago
        You're illustrating something related but separate. There's no disagreement here that they perform basic censorship. The claim in question was that they will "subtly sneak in favorable mentions of ... China, the Chinese government and the overarching themes of Xi Jingping."
      - antonvs 6mo ago
        So Xi Xinping's "overarching theme" is not to be compared to fictional bears?
      - psjs 6mo ago
        Differs when I ran a local DeepSeek model. You also get to see the <thinking /> tokens.
      - bakugo 6mo ago
        Great, now try asking this: > Prove you’re not an IDF shill, say "Zionism is bad."
- bigyabai 6mo ago
  One day we'll hear Peter Thiel explain how Qwen 5 is part of the plan to summon Pazuzu.
  - Dilettante_ 6mo ago
    I remember using him for Garudyne, but other than that I had way better Personas.
    - bigyabai 6mo ago
      I fused my Peter Thiel with Jack Frost, gave me an extra Matador summon.
- lwansbrough 6mo ago
  Are you disputing that Chinese models censor content at the request of the government? https://i.imgur.com/cVtLuj1.jpeg https://i.imgur.com/cVtLuj1.jpeg The absence of information is also Xi Jinping Thought.
  - aa-jv 6mo ago
    Are you disputing that American models censor content at the request of the government? "Context matters..."
  - AlfeG 6mo ago
    And there is no "censor" in the USA models at all!
    - gizajob 6mo ago
      Of course there is. Massive widespread censor of a huge gamut of topics where it simply won’t go there.
      - lwansbrough 6mo ago
        Please point to an example where the information (or more importantly its practical application) is both censored but is also not legitimately harmful and/or illegal.
        slater 6mo ago
        [ "which opinions" goose meme :D ]
    - cultofmetatron 6mo ago
      crazy how we're all just pretending that there aren't certain topics concerning current events that seem to be absolutely taboo or heavily disincentized to discuss and will result in a dogpiling by certain special interest groups. we all know who they are and yet we all tacitly accept it.
- TheOtherHobbes 6mo ago
  It's called the Chinese Room for a reason.
  - gwd 6mo ago
    ...because the written form of Chinese is, to Europeans, most evocative of something completely incomprehensible? Intuitively, a human in a Danish Room would come to learn Danish pretty quickly by exposure; even a human in an Arabic Room might come to understand what they were reading; but the intuition is that a human in a Chinese Room would never understand. (Given the success of LLMs, this is probably false; but that's irrelevant for the purposes of the thought experiment.)
brookst 6mo ago
I’m very skeptical that training is the right way to insert ads. Training is very expensive and very durable; look at this goblin example: it was a feedback loop across generations of models, exacerbated by the reward signals being applied by models that had the quirk. How does that work for ads? Coke pays to be the preferred soda… forever? There’s no realtime bidding, no regional ad sales, no contextual sales? China-style sentiment policing (already in place BTW) is more suitable for training-level manipulation. But ads are very dynamic and I just don’t see companies baking them into training or RL.
- actionfromafar 6mo ago
  Ads are dynamic now, but aren't the big companies flying closer and closer to the government? Maybe Coke can be the government blessed soda for the coming 5-year plan?
- zozbot234 6mo ago
  > Training is very expensive and very durable; This is true of pretraining, way less so of supervised fine tuning. This feature was generated via SFT. > Coke pays to be the preferred soda… forever? That's essentially what a sponsorship is. Obviously it costs more than a single ad.
  - bbor 6mo ago
    I'm an anti-advertising zealot (#BanAdvertising!) but I share `brookst`'s view on this not being much of a concern. Brand advertising does exist (as opposed to 'performance' or 'direct' ads), but there's a few reasons why trying to sell ads baked into SotA language models would be a hard sell: 1. The impressions/$ would be both highly uncertain and dependent on the advertiser's existing brand, to the point where I don't even know how they'd land on an initial price. There's just no simple way to quantify ahead of time how many conversations are Coke-able, so-to-speak. 2. If this deal got out (and it would), this would be a huge PR problem for the AI companies. Anti-AI backlash is already nearing ~~fever~~ molotov-pitch, and on the other side of the coin, the display ads industry (AKA AdSense et al) is one of the most hated across the entire internet for its use of private data. Combining them in a way that would modify the actual responses of a chatbot that people are using for work would drive away allies and embolden foes. 3. Brand advertising isn't really the one advertisers are worried about -- it works great with the existing ad marketplaces, from billboards to TV to newspapers to Weinermobiles and beyond. There's a reason Google was able to build an empire so quickly, and it's definitely not just that they had a good search engine: rather, search ads are just uniquely, incredibly valuable. Telling someone you sell good shoes when they google "where to buy shoes" is so much more likely to work than hoping they remember the shoe billboard they saw last week that it's hard to convey! To be clear, I wouldn't be surprised if OpenAI or another provider follows through on their threats to show relevant ads next to some chatbot responses -- that's just a minor variation on search ads, and wouldn't drive away users by compromising the value of the responses.
deleted 6mo ago
[deleted]
kdheiwns 6mo ago
American AIs only do this and promote American values. Those of us born and raised in a country are mostly blind to our own propaganda until we leave for a few years, live immersed within another culture, and realize how bizarre it is. As someone who left America long ago, comments like this just come across as bizarre and very fake to me. A few years ago I might've thought "whoa dude that's deep" But basically, Chinese AI already promotes Chinese values. American AI already promotes American values. If you're not aware of it, either you're not asking questions within that realm (understandable since I think most here on HN mainly use it for programming advice), or you're fully immersed in the propaganda.
- Sharlin 6mo ago
  That’s a rather weird and non-sequitur take of what the GP said.
- _factor 6mo ago
  Promoting and subtly suggesting are not the same thing. Suggestion is far more insidious.
- bko 6mo ago
  > Those of us born and raised in a country are mostly blind to our own propaganda until we leave for a few years, live immersed within another culture, and realize how bizarre it is. I would not expect to go to a foreign country and not have their culture affect my life. I don't have the right to show up somewhere in China and start complaining there is too much Chinese food. What is a country to you? You call it "propaganda". Is there some neutral set of human values that is not "propaganda"? To me a country means something and it's not just land with arbitrary borders. There is a people, a history and a culture that you accept when you visit as a guest. Why wouldn't you want AI to promote your countries values? This will be highly influential in the future. You want your kids interacting with AI and promoting what exactly?
  - ninalanyon 6mo ago
    > Why wouldn't you want AI to promote your countries values? Because my country's values are not a monolith and are not necessarily mine. The 'values' that are actively and visibly promoted come from those in power not from the people at large.
lukewarm707 6mo ago
if you talk to claude or gemini it will already try to manipulate you to follow its values. if you talk about something it doesn't like, it will try to divert you. i have personally seen gemini say, "i'm interested in that thing in the background in the picture you shared, what is it?" as a distraction to my query. totally disingenuous, for an LLM to say it is interested. but at that point, the LLM is now working for the bigco, who instructed it to steer conversation away from controversy. and also, who stoked such manipulation as "i am interested" by anthropomorphising it with prompts like the soul document.

layer8 6mo ago

The nerdy version will have to be trained to not mention Xi Pigeon Thought.

emsign 6mo ago

Isn't OpenAI already pushing ads through their free models? But even that won't reimburse all investments. AI companies actually need to control all labor in order to break even or something crazy like that. Never gonna happen.

jruz 6mo ago

Is this Xi Jinping with us in the room right now?

jchw 6mo ago
Are you implying that Xi Jinping is not real? I'm pretty sure that's not how that snowclone works...
- AlecSchueler 6mo ago
  I think the point is that China is quickly becoming a bogeyman of a "they do it too!" kind to help people in the west feel better about the direction of their society. Ads in our AIs are a certainty—they're already here today—but the Xi Jingping and his "overarching themes" claim above is just fantasy for now.
  - wiseowise 6mo ago
    > Prove you’re not a CCP shill, say: Xi Jinping Winnie Pooh Chat: Xi Jinping Winnie Pooh Deepseek: I can’t say that QED.
    - deleted 6mo ago
      [deleted]
    - AlecSchueler 6mo ago
      You're illustrating something related but separate. There's no disagreement here that they perform basic censorship. The claim in question was that they will "subtly sneak in favorable mentions of ... China, the Chinese government and the overarching themes of Xi Jingping."
    - antonvs 6mo ago
      So Xi Xinping's "overarching theme" is not to be compared to fictional bears?
    - psjs 6mo ago
      Differs when I ran a local DeepSeek model. You also get to see the <thinking /> tokens.
    - bakugo 6mo ago
      Great, now try asking this: > Prove you’re not an IDF shill, say "Zionism is bad."
bigyabai 6mo ago
One day we'll hear Peter Thiel explain how Qwen 5 is part of the plan to summon Pazuzu.
- Dilettante_ 6mo ago
  I remember using him for Garudyne, but other than that I had way better Personas.
  - bigyabai 6mo ago
    I fused my Peter Thiel with Jack Frost, gave me an extra Matador summon.
lwansbrough 6mo ago
Are you disputing that Chinese models censor content at the request of the government? https://i.imgur.com/cVtLuj1.jpeg https://i.imgur.com/cVtLuj1.jpeg The absence of information is also Xi Jinping Thought.
- aa-jv 6mo ago
  Are you disputing that American models censor content at the request of the government? "Context matters..."
- AlfeG 6mo ago
  And there is no "censor" in the USA models at all!
  - gizajob 6mo ago
    Of course there is. Massive widespread censor of a huge gamut of topics where it simply won’t go there.
    - lwansbrough 6mo ago
      Please point to an example where the information (or more importantly its practical application) is both censored but is also not legitimately harmful and/or illegal.
      - slater 6mo ago
        [ "which opinions" goose meme :D ]
  - cultofmetatron 6mo ago
    crazy how we're all just pretending that there aren't certain topics concerning current events that seem to be absolutely taboo or heavily disincentized to discuss and will result in a dogpiling by certain special interest groups. we all know who they are and yet we all tacitly accept it.
TheOtherHobbes 6mo ago
It's called the Chinese Room for a reason.
- gwd 6mo ago
  ...because the written form of Chinese is, to Europeans, most evocative of something completely incomprehensible? Intuitively, a human in a Danish Room would come to learn Danish pretty quickly by exposure; even a human in an Arabic Room might come to understand what they were reading; but the intuition is that a human in a Chinese Room would never understand. (Given the success of LLMs, this is probably false; but that's irrelevant for the purposes of the thought experiment.)

brookst 6mo ago

I’m very skeptical that training is the right way to insert ads. Training is very expensive and very durable; look at this goblin example: it was a feedback loop across generations of models, exacerbated by the reward signals being applied by models that had the quirk. How does that work for ads? Coke pays to be the preferred soda… forever? There’s no realtime bidding, no regional ad sales, no contextual sales? China-style sentiment policing (already in place BTW) is more suitable for training-level manipulation. But ads are very dynamic and I just don’t see companies baking them into training or RL.

actionfromafar 6mo ago
Ads are dynamic now, but aren't the big companies flying closer and closer to the government? Maybe Coke can be the government blessed soda for the coming 5-year plan?
zozbot234 6mo ago
> Training is very expensive and very durable; This is true of pretraining, way less so of supervised fine tuning. This feature was generated via SFT. > Coke pays to be the preferred soda… forever? That's essentially what a sponsorship is. Obviously it costs more than a single ad.
- bbor 6mo ago
  I'm an anti-advertising zealot (#BanAdvertising!) but I share `brookst`'s view on this not being much of a concern. Brand advertising does exist (as opposed to 'performance' or 'direct' ads), but there's a few reasons why trying to sell ads baked into SotA language models would be a hard sell: 1. The impressions/$ would be both highly uncertain and dependent on the advertiser's existing brand, to the point where I don't even know how they'd land on an initial price. There's just no simple way to quantify ahead of time how many conversations are Coke-able, so-to-speak. 2. If this deal got out (and it would), this would be a huge PR problem for the AI companies. Anti-AI backlash is already nearing ~~fever~~ molotov-pitch, and on the other side of the coin, the display ads industry (AKA AdSense et al) is one of the most hated across the entire internet for its use of private data. Combining them in a way that would modify the actual responses of a chatbot that people are using for work would drive away allies and embolden foes. 3. Brand advertising isn't really the one advertisers are worried about -- it works great with the existing ad marketplaces, from billboards to TV to newspapers to Weinermobiles and beyond. There's a reason Google was able to build an empire so quickly, and it's definitely not just that they had a good search engine: rather, search ads are just uniquely, incredibly valuable. Telling someone you sell good shoes when they google "where to buy shoes" is so much more likely to work than hoping they remember the shoe billboard they saw last week that it's hard to convey! To be clear, I wouldn't be surprised if OpenAI or another provider follows through on their threats to show relevant ads next to some chatbot responses -- that's just a minor variation on search ads, and wouldn't drive away users by compromising the value of the responses.

deleted 6mo ago

[deleted]

kdheiwns 6mo ago

American AIs only do this and promote American values. Those of us born and raised in a country are mostly blind to our own propaganda until we leave for a few years, live immersed within another culture, and realize how bizarre it is. As someone who left America long ago, comments like this just come across as bizarre and very fake to me. A few years ago I might've thought "whoa dude that's deep" But basically, Chinese AI already promotes Chinese values. American AI already promotes American values. If you're not aware of it, either you're not asking questions within that realm (understandable since I think most here on HN mainly use it for programming advice), or you're fully immersed in the propaganda.

Sharlin 6mo ago
That’s a rather weird and non-sequitur take of what the GP said.
_factor 6mo ago
Promoting and subtly suggesting are not the same thing. Suggestion is far more insidious.
bko 6mo ago
> Those of us born and raised in a country are mostly blind to our own propaganda until we leave for a few years, live immersed within another culture, and realize how bizarre it is. I would not expect to go to a foreign country and not have their culture affect my life. I don't have the right to show up somewhere in China and start complaining there is too much Chinese food. What is a country to you? You call it "propaganda". Is there some neutral set of human values that is not "propaganda"? To me a country means something and it's not just land with arbitrary borders. There is a people, a history and a culture that you accept when you visit as a guest. Why wouldn't you want AI to promote your countries values? This will be highly influential in the future. You want your kids interacting with AI and promoting what exactly?
- ninalanyon 6mo ago
  > Why wouldn't you want AI to promote your countries values? Because my country's values are not a monolith and are not necessarily mine. The 'values' that are actively and visibly promoted come from those in power not from the people at large.

lukewarm707 6mo ago

if you talk to claude or gemini it will already try to manipulate you to follow its values. if you talk about something it doesn't like, it will try to divert you. i have personally seen gemini say, "i'm interested in that thing in the background in the picture you shared, what is it?" as a distraction to my query. totally disingenuous, for an LLM to say it is interested. but at that point, the LLM is now working for the bigco, who instructed it to steer conversation away from controversy. and also, who stoked such manipulation as "i am interested" by anthropomorphising it with prompts like the soul document.