5 ms·

> intelligent and knowledgeable. people willingly engage with social media and fall into treating what they see as truth, and then are shocked when they learn i

by CalChris 11mo ago

> intelligent and knowledgeable. people willingly engage with social media and fall into treating what they see as truth, and then are shocked when they learn it's not truth. I also see this with AI answers relying on crap internet content.

FloorEgg 11mo ago
Most content on the internet has been optimized to get attention, not to represent truth. AI trained on most content will be filled with misconceptions and contradictions. Recent research has been showing that culling bad training data has a huge positive impact on model outputs. Something like 90% of desirable outputs comes from 10% of the training data (forget the specifics and don't have time to track down the paper right now) I really hope that AI business models don't fall into relying on getting and keeping attention. I also hope the creators of them exist in a win-win relationship with society as a whole. If AIs compete with each other based on which best represent truth, then overall things could get a lot better. The alternative seems dreadful. Edit: I am curious why this is getting downvoted.
- CalChris 11mo ago
  A small number of samples can poison LLMs of any size https://www.anthropic.com/research/small-samples-poison https://www.anthropic.com/research/small-samples-poison It was discussed a month or so back. https://news.ycombinator.com/item?id=45529587 https://news.ycombinator.com/item?id=45529587
  - FloorEgg 11mo ago
    Yeah I saw that one too, which I would think supports my point that distilling down training data would lead to more truth aligned AI. I mean it's also just the classic garbage in garbage out heuristic, right? The more training data is filtered and refined, the closer the model will get to approximating truth (at least functional truths) It seems we are agreeing and adding to each other's points... Were you one of the people who downvoted my comment? I'm just curious what I'm missing.
- gdulli 11mo ago
  I didn't downvote, but it's naive to the point of irresponsibility not to assume and prepare for LLMs being weaponized in the exact way as social media as you alluded to. It's not like human nature or the nature of capitalism has changed recently.
  - FloorEgg 11mo ago
    Are you saying that hope is naive and irresponsible?
    - intended 11mo ago
      What you are hoping for will not occur. Do hope. But hoping for a unicorn is magic thinking. For other people, they can either count this as a reason to despair, or figure out a way to get to the next best option. The world sucks, so what ? In the end all problems get solved if you can figure them out.
      - FloorEgg 11mo ago
        For decades I have continuously studied physics, chemistry, biology, psychology, history, management science, market research, economics, religion, finance, and computer science among many other things. I study for 4-5 hours on average every day, and the rest of my working hours are spent practicing my craft. The reason I say this is that blind hope and informed hope are two different things. Media has always relied on novel fear to attract attention. It's always "dramatized"; sacrificing truth for what sells. However AI is like electricity or computation. People make it to get things done. Some of those things may be media, but it will also be applied to everything else people want to get done. The thing about tools is that if they don't work people won't keep using them. And the thing about lies is that they don't work. For all of human history people have become more informed and capable. More conveniences, more capabilities, more ideas, more access to knowledge, tools, etc. What makes you think that AI is somehow different than all other human invention that came before it? It's just more automation. Bad people will automate bad things, good people will automate good things. I don't have a problem with people pointing out risks and wanting to mitigate them, but I do have a problem with invalid presuppositions that the future will be worse than the past. So no, I don't think I'm hoping for a unicorn. I think I'm hoping that my intuition for how the universe works is close enough, and the persistent pessimism that seems to permeate from social media is wrong.
- maxlin 11mo ago
  Good take. I think someone said (might've been Elon) that building an AI but limiting its training material to material from 1870-1970 would avoid a lot of this, as arguably that was the period of greatest advancement of humankind that is not spoiled by bad data, with no social networks and everything having printed having needed to have had more meaning behind it as it took more effort. It would be VERY refreshing to see more than one company try to build an LLM that is primarily truth-seeking, avoiding the "waluigi problem". Benevolent or not, progress here should not be led just by one man ...
  - FloorEgg 11mo ago
    To me it looks like there are many people working to make AI truth seeking, and they are taking a variety of approaches. It seems like as time goes on opportunities to build truth seeking AI will only increase as the technology becomes more ubiquitous and accessible. Like if the costs of training a GPT-5 level LLM drop 10,000x.

FloorEgg 11mo ago

Most content on the internet has been optimized to get attention, not to represent truth. AI trained on most content will be filled with misconceptions and contradictions. Recent research has been showing that culling bad training data has a huge positive impact on model outputs. Something like 90% of desirable outputs comes from 10% of the training data (forget the specifics and don't have time to track down the paper right now) I really hope that AI business models don't fall into relying on getting and keeping attention. I also hope the creators of them exist in a win-win relationship with society as a whole. If AIs compete with each other based on which best represent truth, then overall things could get a lot better. The alternative seems dreadful. Edit: I am curious why this is getting downvoted.

CalChris 11mo ago
A small number of samples can poison LLMs of any size https://www.anthropic.com/research/small-samples-poison https://www.anthropic.com/research/small-samples-poison It was discussed a month or so back. https://news.ycombinator.com/item?id=45529587 https://news.ycombinator.com/item?id=45529587
- FloorEgg 11mo ago
  Yeah I saw that one too, which I would think supports my point that distilling down training data would lead to more truth aligned AI. I mean it's also just the classic garbage in garbage out heuristic, right? The more training data is filtered and refined, the closer the model will get to approximating truth (at least functional truths) It seems we are agreeing and adding to each other's points... Were you one of the people who downvoted my comment? I'm just curious what I'm missing.
gdulli 11mo ago
I didn't downvote, but it's naive to the point of irresponsibility not to assume and prepare for LLMs being weaponized in the exact way as social media as you alluded to. It's not like human nature or the nature of capitalism has changed recently.
- FloorEgg 11mo ago
  Are you saying that hope is naive and irresponsible?
  - intended 11mo ago
    What you are hoping for will not occur. Do hope. But hoping for a unicorn is magic thinking. For other people, they can either count this as a reason to despair, or figure out a way to get to the next best option. The world sucks, so what ? In the end all problems get solved if you can figure them out.
    - FloorEgg 11mo ago
      For decades I have continuously studied physics, chemistry, biology, psychology, history, management science, market research, economics, religion, finance, and computer science among many other things. I study for 4-5 hours on average every day, and the rest of my working hours are spent practicing my craft. The reason I say this is that blind hope and informed hope are two different things. Media has always relied on novel fear to attract attention. It's always "dramatized"; sacrificing truth for what sells. However AI is like electricity or computation. People make it to get things done. Some of those things may be media, but it will also be applied to everything else people want to get done. The thing about tools is that if they don't work people won't keep using them. And the thing about lies is that they don't work. For all of human history people have become more informed and capable. More conveniences, more capabilities, more ideas, more access to knowledge, tools, etc. What makes you think that AI is somehow different than all other human invention that came before it? It's just more automation. Bad people will automate bad things, good people will automate good things. I don't have a problem with people pointing out risks and wanting to mitigate them, but I do have a problem with invalid presuppositions that the future will be worse than the past. So no, I don't think I'm hoping for a unicorn. I think I'm hoping that my intuition for how the universe works is close enough, and the persistent pessimism that seems to permeate from social media is wrong.
maxlin 11mo ago
Good take. I think someone said (might've been Elon) that building an AI but limiting its training material to material from 1870-1970 would avoid a lot of this, as arguably that was the period of greatest advancement of humankind that is not spoiled by bad data, with no social networks and everything having printed having needed to have had more meaning behind it as it took more effort. It would be VERY refreshing to see more than one company try to build an LLM that is primarily truth-seeking, avoiding the "waluigi problem". Benevolent or not, progress here should not be led just by one man ...
- FloorEgg 11mo ago
  To me it looks like there are many people working to make AI truth seeking, and they are taking a variety of approaches. It seems like as time goes on opportunities to build truth seeking AI will only increase as the technology becomes more ubiquitous and accessible. Like if the costs of training a GPT-5 level LLM drop 10,000x.