4 ms·

I've had multiple people copy and paste AI conversations and results in GitHub issues, emails, etc., and there are I think a growing number of people who blindl

I've had multiple people copy and paste AI conversations and results in GitHub issues, emails, etc., and there are I think a growing number of people who blindly trust the results of any of these models... including the 'results summary' posted at the top of Google search results. Almost every summary I have read through contains at least one glaring mistake, but if it's something I know nothing about, I could see how easy it would be to just trust it, since 95% of it seems true/accurate. Trust, but verify is all the more relevant today. Except I would discount the trust, even.

add-sub-mul-div 1y ago
We all think ourselves as understanding the tradeoffs of this tech and that we know how to use it responsibly. And we here may be right. But the typical person wants to do the least amount of effort and thinking possible. Our society will evolve to reflect this, it won't be great, and it will affect all of us no matter how personally responsible some of us remain.
- iotku 1y ago
  I consider myself pretty technically literate, and not the worst at programming (though certainly far from the very best). Even so I can spend plenty of time arguing with LLMs which will give me plausible looking but extremely broken answers to some of the programming problems. In the programming domain I can at least run something and see it doesn't compile or work as I expect, but you can't verify that a written statement about someone/something is the correct interpretation without knowing the correct answer ahead of time. To muddy the waters further, things work just well enough on common knowledge that it's easy to believe it could be right about uncommon knowledge which you don't know how to verify. (Or else you wouldn't be asking it in the first place)
  - add-sub-mul-div 1y ago
    Even with code, "seeing" a block of code working isn't a guarantee there's not a subtle bug that will expose itself in a week, in a month, in a year under the right conditions.
    - giantrobot 1y ago
      I've pointed this out a lot and I often get replies along the lines of "people make mistakes too". While this is true, LLMs lack institutional memory leading to decisions. Even good reasoning models can't reliably tell you why they wrote some code they did when asked to review it. They can't even reliably run tests since they'll hardcode passing values for tests. The same code out of an intern or junior programmer you can at least walk through their reasoning on a code review. Even better if they tend to learn and not make that same mistake again. LLMs will happily screw up randomly on every repeated prompt. The hardest code you encounter is code written by someone else. You don't have the same mental model or memories as the original author. So you need to build all that context and then reason through the code. If an LLM is writing a lot of your code you're missing out on all the context you'd normally build writing it.
Aurornis 1y ago
> I've had multiple people copy and paste AI conversations and results in GitHub issues, emails, etc., A growing number of Discords, open source projects, and other spaces where I participate now have explicit rules against copying and pasting ChatGPT content. When there aren’t rules, many people are quick to discourage LLM copy and paste. “Please don’t do this”. The LLM copy and paste wall of text that may or may not be accurate is extremely frustrating to everyone else. Some people think they’re being helpful by doing it, but it’s quickly becoming a social faux pas.
- lawlessone 1y ago
  see it with comments here sometimes , "i asked chatgpt about Y" , really annoying, we all could have asked chatgpt, we didn't.
  - ljm 1y ago
    Have had some conversations where the other person goes into chatgpt to answer a question while I’m in the process of explaining a solution, and then says “GPT says this, look…” Use an agent to help you code or whatever all you want, I don’t care about that. At least listen when I’m trying to share some specific knowledge instead of fobbing me off with GPT. If we’re both stumped, go nuts. But at least put some effort into the prompt to get a better response.
  - buu700 1y ago
    I don't have an issue quoting LLMs in and of itself, but the context and how you present it both matter. "ChatGPT says X" seems roughly equivalent to "some random blog I found claims X". There's a difference between sharing something as a starting point for investigation and passing off unverified information (from any source) as your own well researched/substantiated work which you're willing to stake your professional reputation on standing by. Of course, quoting an LLM is also pretty different from merely collaborating with an LLM on writing content that's substantially your own words or ideas, which no one should care about one way or another, at least in most contexts.
- tavavex 1y ago
freeopinion 1y ago
prompt> use javascript to convert a unix timestamp to a date in 'YYYY-MM-DD' format using Temporal answer> Temporal.Instant.fromEpochSeconds(timestamp).toPlainDate() Trust but verify?
- nielsbot 1y ago
  what does this mean in this convo?
  - freeopinion 1y ago
    If you were considering purchasing a Biology text book, and spot read two chapters, what if you found the following?: In the first chapter it claimed that most adult humans have 20 teeth. In the second chapter you read that female humans have 22 chromosomes and male humans have 23. You find these claims in the 24 pages you sample. Do you buy the book? Companies are paying huge sums to AI companies with worse track records. Would you put the book in your reference library if somebody gave it to you for free? Services like Google or DuckDuckGo put their AI-generated content at the top of search results with these inaccuracies. [edit: replace paragraph that somehow got deleted, fix typo]
  - freeopinion 1y ago
    Google distinguished itself early with techniques like PageRank that put more relevant content at the top of their search results. Is it too late for a rival to distinguish itself with techniques like "Don't put garbage AI at the top of search results"?
- eszed 1y ago
  I mean... Yes? That looks correct to me°, but it's been a minute since I worked with Temporal, so I'd run it myself and examine the output before I cut and paste. Or have I missed your point? --- °Missing a TZ assertion, but I don't remember what happens by default. Zulu time? I'd hope so, but that reinforces my point.
  - nosianu 1y ago
leeoniya 1y ago
> but if it's something I know nothing about, I could see how easy it would be to just trust it, since 95% of it seems true/accurate. https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect
userbinator 1y ago
"Trust, but verify" is an oxymoron. AI is not to be trusted for information.
- nielsbot 1y ago
  AI = convincing garbage generator
abustamam 1y ago
Whenever I use AI in social settings to fact check or do research or get advice, I always trust but verify, but also disclaim it so that people know to trust but verify. I think this is a good habit to get people into, even in casual conversations. Even if someone didn't directly get their info from AI and got it online, the content could have still been generated by AI. Like you said, the trust part of trust but verify is quickly dwindling.
account42 1y ago
> I've had multiple people copy and paste AI conversations and results in GitHub issues, emails, etc., and there are I think a growing number of people who blindly trust the results of any of these models... including the 'results summary' posted at the top of Google search results. I like the term "echoborg" for these people. I hope it catches on.
fennecbutt 1y ago
It's just mass stupidity really. Technology is just a lever for what already existed. The same people blindly trusting AI nonsense are the same people who trusted nonsense from social media or talking heads on unreputable news channels. Like, who actually reads the output of The Sun, etc? Those people do, have always done and will continue to do so. And they vote, yaaay democracy - if your voter base lives in a fantasy world of fake news and false science is democracy still sancrosact?

add-sub-mul-div 1y ago

We all think ourselves as understanding the tradeoffs of this tech and that we know how to use it responsibly. And we here may be right. But the typical person wants to do the least amount of effort and thinking possible. Our society will evolve to reflect this, it won't be great, and it will affect all of us no matter how personally responsible some of us remain.

iotku 1y ago
I consider myself pretty technically literate, and not the worst at programming (though certainly far from the very best). Even so I can spend plenty of time arguing with LLMs which will give me plausible looking but extremely broken answers to some of the programming problems. In the programming domain I can at least run something and see it doesn't compile or work as I expect, but you can't verify that a written statement about someone/something is the correct interpretation without knowing the correct answer ahead of time. To muddy the waters further, things work just well enough on common knowledge that it's easy to believe it could be right about uncommon knowledge which you don't know how to verify. (Or else you wouldn't be asking it in the first place)
- add-sub-mul-div 1y ago
  Even with code, "seeing" a block of code working isn't a guarantee there's not a subtle bug that will expose itself in a week, in a month, in a year under the right conditions.
  - giantrobot 1y ago
    I've pointed this out a lot and I often get replies along the lines of "people make mistakes too". While this is true, LLMs lack institutional memory leading to decisions. Even good reasoning models can't reliably tell you why they wrote some code they did when asked to review it. They can't even reliably run tests since they'll hardcode passing values for tests. The same code out of an intern or junior programmer you can at least walk through their reasoning on a code review. Even better if they tend to learn and not make that same mistake again. LLMs will happily screw up randomly on every repeated prompt. The hardest code you encounter is code written by someone else. You don't have the same mental model or memories as the original author. So you need to build all that context and then reason through the code. If an LLM is writing a lot of your code you're missing out on all the context you'd normally build writing it.

Aurornis 1y ago

> I've had multiple people copy and paste AI conversations and results in GitHub issues, emails, etc., A growing number of Discords, open source projects, and other spaces where I participate now have explicit rules against copying and pasting ChatGPT content. When there aren’t rules, many people are quick to discourage LLM copy and paste. “Please don’t do this”. The LLM copy and paste wall of text that may or may not be accurate is extremely frustrating to everyone else. Some people think they’re being helpful by doing it, but it’s quickly becoming a social faux pas.

lawlessone 1y ago
see it with comments here sometimes , "i asked chatgpt about Y" , really annoying, we all could have asked chatgpt, we didn't.
- ljm 1y ago
  Have had some conversations where the other person goes into chatgpt to answer a question while I’m in the process of explaining a solution, and then says “GPT says this, look…” Use an agent to help you code or whatever all you want, I don’t care about that. At least listen when I’m trying to share some specific knowledge instead of fobbing me off with GPT. If we’re both stumped, go nuts. But at least put some effort into the prompt to get a better response.
- buu700 1y ago
  I don't have an issue quoting LLMs in and of itself, but the context and how you present it both matter. "ChatGPT says X" seems roughly equivalent to "some random blog I found claims X". There's a difference between sharing something as a starting point for investigation and passing off unverified information (from any source) as your own well researched/substantiated work which you're willing to stake your professional reputation on standing by. Of course, quoting an LLM is also pretty different from merely collaborating with an LLM on writing content that's substantially your own words or ideas, which no one should care about one way or another, at least in most contexts.
tavavex 1y ago

freeopinion 1y ago

prompt> use javascript to convert a unix timestamp to a date in 'YYYY-MM-DD' format using Temporal answer> Temporal.Instant.fromEpochSeconds(timestamp).toPlainDate() Trust but verify?

nielsbot 1y ago
what does this mean in this convo?
- freeopinion 1y ago
  If you were considering purchasing a Biology text book, and spot read two chapters, what if you found the following?: In the first chapter it claimed that most adult humans have 20 teeth. In the second chapter you read that female humans have 22 chromosomes and male humans have 23. You find these claims in the 24 pages you sample. Do you buy the book? Companies are paying huge sums to AI companies with worse track records. Would you put the book in your reference library if somebody gave it to you for free? Services like Google or DuckDuckGo put their AI-generated content at the top of search results with these inaccuracies. [edit: replace paragraph that somehow got deleted, fix typo]
- freeopinion 1y ago
  Google distinguished itself early with techniques like PageRank that put more relevant content at the top of their search results. Is it too late for a rival to distinguish itself with techniques like "Don't put garbage AI at the top of search results"?
eszed 1y ago
I mean... Yes? That looks correct to me°, but it's been a minute since I worked with Temporal, so I'd run it myself and examine the output before I cut and paste. Or have I missed your point? --- °Missing a TZ assertion, but I don't remember what happens by default. Zulu time? I'd hope so, but that reinforces my point.
- nosianu 1y ago

leeoniya 1y ago

> but if it's something I know nothing about, I could see how easy it would be to just trust it, since 95% of it seems true/accurate. https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect

userbinator 1y ago

"Trust, but verify" is an oxymoron. AI is not to be trusted for information.

nielsbot 1y ago
AI = convincing garbage generator

abustamam 1y ago

Whenever I use AI in social settings to fact check or do research or get advice, I always trust but verify, but also disclaim it so that people know to trust but verify. I think this is a good habit to get people into, even in casual conversations. Even if someone didn't directly get their info from AI and got it online, the content could have still been generated by AI. Like you said, the trust part of trust but verify is quickly dwindling.

account42 1y ago

> I've had multiple people copy and paste AI conversations and results in GitHub issues, emails, etc., and there are I think a growing number of people who blindly trust the results of any of these models... including the 'results summary' posted at the top of Google search results. I like the term "echoborg" for these people. I hope it catches on.

fennecbutt 1y ago

It's just mass stupidity really. Technology is just a lever for what already existed. The same people blindly trusting AI nonsense are the same people who trusted nonsense from social media or talking heads on unreputable news channels. Like, who actually reads the output of The Sun, etc? Those people do, have always done and will continue to do so. And they vote, yaaay democracy - if your voter base lives in a fantasy world of fake news and false science is democracy still sancrosact?