3 ms·

AI Is Breaking This Thing We Call Trust

128 points by matheusml 23d ago

sigalor 23d ago
I wonder when the people will get that intelligence is not only directed at the external, but only really starts when you look at the internal (joy, pleasure, traumas, taboos, awkwardness, abuse, trust etc.). Look up the word "interoception".
georgemcbay 23d ago
> AI Is Breaking This Thing We Call Trust An increasingly large percentage of people being lying POS grifters is breaking trust. AI just happens to be a tool that the laziest of that group of people leans on a lot to supercharge their grifts. But our massive and growing trust problem predates and is way deeper than AI.
- imawakegnxoxo 23d ago
  Yeah, I think what they really mean is people, especially those with power, broke this thing called trust and now they're trying to use """ai""" as a scapegoat. It'd be along the same lines as saying "I was drunk driving 75km/hr over the limit down a busy suburban road during the middle of the day and 5 people died because of it so we CAN NOT TRUST CARS" And everyone stands up and claps
  - duskdozer 22d ago
    Both, in a way. The car that can drive 75km/hr existing enabled the person to choose to use it while drunk. As in, there was no trust beforehand, but the means used to "break" it were not as accessible.
plaidfuji 23d ago
> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it. This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some point in the near future, you will just trust that the software code works. Instead you’ll just debate the spec (the prompt). Your thinking can move one level up the system hierarchy. Everyone’s can. We’ve done it before, we’ll do it again, and this won’t even be the last time it happens. Let go of the anxiety and let it happen.
- coldtea 23d ago
  >This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. Just because something wasn't required at all levels or 100% it doesn't mean it wasn't required or it wasn't important though. I trusted that the machine code worked because it was a mostly mechanical transformation, with little to zero non-determinism involved, and the design decisions for the high level code were still mine and understood. Not the same with an LLM, even when see just as an added level of abstraction. Given some LLM produced code, I also trust the compiler will produce working machine code matching it. Doesn't mean I trust the LLM code in the same way.
- meander_water 23d ago
  You trust machine code because it was produced by a deterministic system that was written and tested thoroughly by other engineers. AI systems are not the same. You can't guarantee deterministic output. Also, code is still the best language we have to convey our intentions as engineers into function. Natural language is too imprecise and ambiguous. Understanding the code allows us to understand the intention behind the code and identify bugs, plan and architect better. Inspecting prompts gives you a blinkered view of the system meaning you're more likely to make incorrect assumptions leading to serious bugs.
  - Terr_ 23d ago
    Right, it's a chain of trust between the layers. The link I choose next to me helps determine the quality of successive links, whether I inspect them or not. When I buy a wooden shack from a carpenter, I'd never expect them to core-sample every plank to ensure it's good wood. That's because a good carpenter habitually buys wood from a good supplier, and will have the experience to detect other suspicious indicators. Similarly, my coworker might not have verified the machine code, but I can be pretty confident they didn't vibe-install a compiler by Ken Thompson that puts backdoors into the binary.
  - faangguyindia 23d ago
    Are your teammates deterministic output machines?
deleted 23d ago
[deleted]
lovich 23d ago
The elite broke trust. No one made people like Sam Altman into a compulsive liar. No one made Musk call a service full self driving, and then make it return liability to the user the second it decides an accident is about to happen. Stop removing agency from the people making the top level decisions and reaping all the rewards. Would you claim that someone who took 5 shots in a row and drove home wasn’t responsible for an accident because the car was what physically did the damage?
- mitxela 23d ago
  There were always a spectrum of people to choose from to become elites. We chose a system that chose the ones who compulsively lied.
  - asdff 23d ago
    To be fair, it has probably only been that way forever. At least we live in a time where some people might realize it.
  - rcxdude 23d ago
    That choice was also made within a framework of another system. It's systems all the way down.
- leobg 21d ago
  You are removing agency from everyone who is not visible, even though they might buy trash and make it profitable for thousands of sellers on Amazon to lie, and to fraudsters to lie, etc.. Pointing fingers at those who are the most visible and happened to be richer than you is something that people have always done as it is so convenient. It becomes even more convenient if those people have money, so you can turn the punishment into robbery. Doing good and distributing wealth at the same time. How convenient! Greetings from Germany.
- leobg 21d ago
  And by the way, I would really love to somewhere find a log file of your personal life and see how often you have promised something, and it didn’t turn out quite to be as perfect as you thought, or took longer than you thought. Or were you perhaps thought that you had achieved a goal when somebody from the outside might disagree. Difference between you and someone in the spotlight is that for them such a log exists; for you it doesn’t.
Havoc 23d ago
I worry for the next generation. You and I have a baseline. A brain that did most of it’s developmental phase while there was still some unified accepted version of truth (even if dubious) Future generations will have to develop in a more splintered reality. A world where there is nothing to hold on to. Not even a dubious commonly accepted narrative
- mitxela 23d ago
  note this started years before generative AI, arguably around 2014 with the gamergate era
  - nonsensical_ 23d ago
    You’ll have to enlighten how GG plays into all this. Sounds like a curious tie-in.
    - mitxela 23d ago
      It was the spark that started the online culture war that metastasized into the real world and brought about the post-truth era we are now living in. Who knows whether it was a symptom or a cause. Either way it marks the beginning.
      - anon1097 22d ago
        I will admit that GG may have been a symptom, but I don't think it was the cause. The cause, as such, was the interblag and incorporation of advertising corrupting touch into everything.
      - scrollaway 22d ago
        It may have been when you noticed it but it wasn’t the spark. If there was an actual spark, it was the 2016 election as this was the first time when all the disinformation, professional troll farms and Russian influence truly came into play on a world changing scale. But there was splintering of online communities and the death of nuance for years prior as well. Twitter contributed a lot by giving voices to everyone, and promoting those who’d shout the loudest. Facebook and its nasty algorithm also greatly contributed to all this (thanks, everyone who worked there, for the world we have now… hope you’re proud and putting all that ad-sponsored salary to good use). Culture war was brewing long before, it just got amplified more and more.
Art9681 23d ago
Every generation has had tech doomers. Every single one. History repeats. Yet we collectively move forward. This time it will be no different. These folks are wasting time when they could be doing something productive.
- Grimblewald 23d ago
  plenty of ex-civs had techdoomers who were right. When tevh doomers have no salient points to make, its fair to not listen, but this time around i think something is different to historic doomerism. Trust is a fundamental thing, and i cannot fathom how a post-trust society could possibly host life as we know it. Life ad we know it comes from systemic efficiencies that are rooted in trust, requiring it deeply. no trust means extreme loss in system efficiency, in a system already creaking under the straign of supporting current populations. post trust is in my reakoning a serious issue to engage with.
  - mitxela 23d ago
    There are plenty of low-trust societies. They have things like armed security guards and barbed wire fences outside every apartment complex. Coughsouthafrica
    - Grimblewald 22d ago
      right, but that surive in isolation, heard immunity to collapse so to speak, but globe wide? pandemonium. we're back to fiefdoms and warring city-states in a generation.
    - bluefirebrand 22d ago
      > There are plenty of low-trust societies And high trust societies have spent the past two decades importing tons of H1B style workers from many of the lower trust countries
      - mitxela 22d ago
        Which ones are those? Sweden?
philbo 23d ago
I think one way to approach this is by deliberately choosing less capable tools (but still AI-assisted). You don't have to delegate huge tasks to agents wholesale, but it requires fighting against mainstream harnesses to do so because they exist to sell you more tokens. Model providers are not incentivised to offer less autonomous flows. I wrote about this frustration a few months ago [1], then realised that if the model providers won't do it perhaps I should try fixing it myself. So, shameless plug klaxon, I wrote an open-source harness that tries to work more like pair programming and less like code review [2]. As usual, my "Show HN" post [3] sank like a stone, so I think perhaps most engineers aren't really interested in this direction. It is possible though, we don't have to just accept whatever tooling the labs throw at us. [1]: https://philbooth.me/blog/agentic-coding-and-mental-models https://philbooth.me/blog/agentic-coding-and-mental-models [2]: https://www.opairdev.org/ https://www.opairdev.org/ [3]: https://news.ycombinator.com/item?id=49625009 https://news.ycombinator.com/item?id=49625009
- eigencoder 22d ago
  Have you tried `pi`? What do you think of it?
  - philbo 21d ago
    I have. I thought it was okay, but it wasn't really the radical departure from other coding harnesses that I was looking for. Perhaps I could have made it that with more customization, I'm not sure. My own harness doesn't provide the agent with shell access, so everything has to go through focused tools with limited functionality. There's also a navigator mode where the agent has no writable tools at all, instead it monitors the project for changes you make in your regular editor and comments on them in real-time. That stuff seemed easier to implement from the ground up.
danjl 22d ago
The assumptions we made about code always led to the biggest issues. If you assumed a junior engineer, or even a senior engineer not familiar with the local practices, would not make certain mistakes, then those mistakes always slipped through. If anything, the AI makes us be _better_ reviewers, since we need to check more of our assumptions at the door. I used to check the compiler assembly output, in case I could optimize it either by hand, or by changing my code to use a do-while instead of a for loop. I don't do that anymore. The same thing is happening with AI-generated code. I used to have to check it all very carefully, and regenerate code when the AI chose to add a whole new mechanism to the code base rather than re-using an existing mechanism, to keep the code solid. That happens less and less, and soon, it won't make sense to read over the code at all. Maybe in some specific circumstances that will be necessary, but less and less overall. And that's ok. We have moved to higher and higher levels of abstraction over the decades that I've been coding, and each time we can take on bigger problems and invent new things.
xenospn 21d ago
AI is not breaking trust. People using AI are breaking our trust.
hulitu 16d ago
No not AI. People who believe in it. But the main purpose of AI is to give answers. Not good/right answers, but just answers. That's why it is used to manufacture consent.

128 points by matheusml 23d ago

I wonder when the people will get that intelligence is not only directed at the external, but only really starts when you look at the internal (joy, pleasure, traumas, taboos, awkwardness, abuse, trust etc.). Look up the word "interoception".

georgemcbay 23d ago

> AI Is Breaking This Thing We Call Trust An increasingly large percentage of people being lying POS grifters is breaking trust. AI just happens to be a tool that the laziest of that group of people leans on a lot to supercharge their grifts. But our massive and growing trust problem predates and is way deeper than AI.

imawakegnxoxo 23d ago
Yeah, I think what they really mean is people, especially those with power, broke this thing called trust and now they're trying to use """ai""" as a scapegoat. It'd be along the same lines as saying "I was drunk driving 75km/hr over the limit down a busy suburban road during the middle of the day and 5 people died because of it so we CAN NOT TRUST CARS" And everyone stands up and claps
- duskdozer 22d ago
  Both, in a way. The car that can drive 75km/hr existing enabled the person to choose to use it while drunk. As in, there was no trust beforehand, but the means used to "break" it were not as accessible.

plaidfuji 23d ago

> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it. This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some point in the near future, you will just trust that the software code works. Instead you’ll just debate the spec (the prompt). Your thinking can move one level up the system hierarchy. Everyone’s can. We’ve done it before, we’ll do it again, and this won’t even be the last time it happens. Let go of the anxiety and let it happen.

coldtea 23d ago
>This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. Just because something wasn't required at all levels or 100% it doesn't mean it wasn't required or it wasn't important though. I trusted that the machine code worked because it was a mostly mechanical transformation, with little to zero non-determinism involved, and the design decisions for the high level code were still mine and understood. Not the same with an LLM, even when see just as an added level of abstraction. Given some LLM produced code, I also trust the compiler will produce working machine code matching it. Doesn't mean I trust the LLM code in the same way.
meander_water 23d ago
You trust machine code because it was produced by a deterministic system that was written and tested thoroughly by other engineers. AI systems are not the same. You can't guarantee deterministic output. Also, code is still the best language we have to convey our intentions as engineers into function. Natural language is too imprecise and ambiguous. Understanding the code allows us to understand the intention behind the code and identify bugs, plan and architect better. Inspecting prompts gives you a blinkered view of the system meaning you're more likely to make incorrect assumptions leading to serious bugs.
- Terr_ 23d ago
  Right, it's a chain of trust between the layers. The link I choose next to me helps determine the quality of successive links, whether I inspect them or not. When I buy a wooden shack from a carpenter, I'd never expect them to core-sample every plank to ensure it's good wood. That's because a good carpenter habitually buys wood from a good supplier, and will have the experience to detect other suspicious indicators. Similarly, my coworker might not have verified the machine code, but I can be pretty confident they didn't vibe-install a compiler by Ken Thompson that puts backdoors into the binary.
- faangguyindia 23d ago
  Are your teammates deterministic output machines?

deleted 23d ago

[deleted]

lovich 23d ago

The elite broke trust. No one made people like Sam Altman into a compulsive liar. No one made Musk call a service full self driving, and then make it return liability to the user the second it decides an accident is about to happen. Stop removing agency from the people making the top level decisions and reaping all the rewards. Would you claim that someone who took 5 shots in a row and drove home wasn’t responsible for an accident because the car was what physically did the damage?

mitxela 23d ago
There were always a spectrum of people to choose from to become elites. We chose a system that chose the ones who compulsively lied.
- asdff 23d ago
  To be fair, it has probably only been that way forever. At least we live in a time where some people might realize it.
- rcxdude 23d ago
  That choice was also made within a framework of another system. It's systems all the way down.
leobg 21d ago
You are removing agency from everyone who is not visible, even though they might buy trash and make it profitable for thousands of sellers on Amazon to lie, and to fraudsters to lie, etc.. Pointing fingers at those who are the most visible and happened to be richer than you is something that people have always done as it is so convenient. It becomes even more convenient if those people have money, so you can turn the punishment into robbery. Doing good and distributing wealth at the same time. How convenient! Greetings from Germany.
leobg 21d ago
And by the way, I would really love to somewhere find a log file of your personal life and see how often you have promised something, and it didn’t turn out quite to be as perfect as you thought, or took longer than you thought. Or were you perhaps thought that you had achieved a goal when somebody from the outside might disagree. Difference between you and someone in the spotlight is that for them such a log exists; for you it doesn’t.

Havoc 23d ago

I worry for the next generation. You and I have a baseline. A brain that did most of it’s developmental phase while there was still some unified accepted version of truth (even if dubious) Future generations will have to develop in a more splintered reality. A world where there is nothing to hold on to. Not even a dubious commonly accepted narrative

mitxela 23d ago
note this started years before generative AI, arguably around 2014 with the gamergate era
- nonsensical_ 23d ago
  You’ll have to enlighten how GG plays into all this. Sounds like a curious tie-in.
  - mitxela 23d ago
    It was the spark that started the online culture war that metastasized into the real world and brought about the post-truth era we are now living in. Who knows whether it was a symptom or a cause. Either way it marks the beginning.
    - anon1097 22d ago
      I will admit that GG may have been a symptom, but I don't think it was the cause. The cause, as such, was the interblag and incorporation of advertising corrupting touch into everything.
    - scrollaway 22d ago
      It may have been when you noticed it but it wasn’t the spark. If there was an actual spark, it was the 2016 election as this was the first time when all the disinformation, professional troll farms and Russian influence truly came into play on a world changing scale. But there was splintering of online communities and the death of nuance for years prior as well. Twitter contributed a lot by giving voices to everyone, and promoting those who’d shout the loudest. Facebook and its nasty algorithm also greatly contributed to all this (thanks, everyone who worked there, for the world we have now… hope you’re proud and putting all that ad-sponsored salary to good use). Culture war was brewing long before, it just got amplified more and more.

Art9681 23d ago

Every generation has had tech doomers. Every single one. History repeats. Yet we collectively move forward. This time it will be no different. These folks are wasting time when they could be doing something productive.

Grimblewald 23d ago
plenty of ex-civs had techdoomers who were right. When tevh doomers have no salient points to make, its fair to not listen, but this time around i think something is different to historic doomerism. Trust is a fundamental thing, and i cannot fathom how a post-trust society could possibly host life as we know it. Life ad we know it comes from systemic efficiencies that are rooted in trust, requiring it deeply. no trust means extreme loss in system efficiency, in a system already creaking under the straign of supporting current populations. post trust is in my reakoning a serious issue to engage with.
- mitxela 23d ago
  There are plenty of low-trust societies. They have things like armed security guards and barbed wire fences outside every apartment complex. Coughsouthafrica
  - Grimblewald 22d ago
    right, but that surive in isolation, heard immunity to collapse so to speak, but globe wide? pandemonium. we're back to fiefdoms and warring city-states in a generation.
  - bluefirebrand 22d ago
    > There are plenty of low-trust societies And high trust societies have spent the past two decades importing tons of H1B style workers from many of the lower trust countries
    - mitxela 22d ago
      Which ones are those? Sweden?

philbo 23d ago

I think one way to approach this is by deliberately choosing less capable tools (but still AI-assisted). You don't have to delegate huge tasks to agents wholesale, but it requires fighting against mainstream harnesses to do so because they exist to sell you more tokens. Model providers are not incentivised to offer less autonomous flows. I wrote about this frustration a few months ago [1], then realised that if the model providers won't do it perhaps I should try fixing it myself. So, shameless plug klaxon, I wrote an open-source harness that tries to work more like pair programming and less like code review [2]. As usual, my "Show HN" post [3] sank like a stone, so I think perhaps most engineers aren't really interested in this direction. It is possible though, we don't have to just accept whatever tooling the labs throw at us. [1]: https://philbooth.me/blog/agentic-coding-and-mental-models https://philbooth.me/blog/agentic-coding-and-mental-models [2]: https://www.opairdev.org/ https://www.opairdev.org/ [3]: https://news.ycombinator.com/item?id=49625009 https://news.ycombinator.com/item?id=49625009

eigencoder 22d ago
Have you tried `pi`? What do you think of it?
- philbo 21d ago
  I have. I thought it was okay, but it wasn't really the radical departure from other coding harnesses that I was looking for. Perhaps I could have made it that with more customization, I'm not sure. My own harness doesn't provide the agent with shell access, so everything has to go through focused tools with limited functionality. There's also a navigator mode where the agent has no writable tools at all, instead it monitors the project for changes you make in your regular editor and comments on them in real-time. That stuff seemed easier to implement from the ground up.

danjl 22d ago

The assumptions we made about code always led to the biggest issues. If you assumed a junior engineer, or even a senior engineer not familiar with the local practices, would not make certain mistakes, then those mistakes always slipped through. If anything, the AI makes us be _better_ reviewers, since we need to check more of our assumptions at the door. I used to check the compiler assembly output, in case I could optimize it either by hand, or by changing my code to use a do-while instead of a for loop. I don't do that anymore. The same thing is happening with AI-generated code. I used to have to check it all very carefully, and regenerate code when the AI chose to add a whole new mechanism to the code base rather than re-using an existing mechanism, to keep the code solid. That happens less and less, and soon, it won't make sense to read over the code at all. Maybe in some specific circumstances that will be necessary, but less and less overall. And that's ok. We have moved to higher and higher levels of abstraction over the decades that I've been coding, and each time we can take on bigger problems and invent new things.

xenospn 21d ago

AI is not breaking trust. People using AI are breaking our trust.

hulitu 16d ago

No not AI. People who believe in it. But the main purpose of AI is to give answers. Not good/right answers, but just answers. That's why it is used to manufacture consent.