7 ms·

AI 'hallucinated' fake legal cases filed to B.C. court in Canadian first

43 points by luu 3y ago

jruohonen 3y ago
It is spreading like a wildfire. Yet, the question remains about the repercussions.
- echelon 3y ago
  We never talked about the Internet's repercussions, and yet here we are. Did we do a good job, or should we have left that stone unturned?
  - badgersnake 3y ago
    We have been talking about them since day 1 and we still are. DMCA, the existence of the EFF, cryptocurrency regulations are all examples of this.
  - hutzlibu 3y ago
    "Did we do a good job, or should we have left that stone unturned?" Regarding the quality of the average Website - no , but also no to the second. The time of the internet was just there with no precedent and now it is hyped imperfect AIs we have to deal with. I think the main problem in both cases is, that most people don't have a clue at all, how it works. (And too many of them are in positions of power).
  - jruohonen 3y ago
    > Did we do a good job, or should we have left that stone unturned? It's kind of a hangover, don't you think? The past decade, I mean, with the Internet.
- supriyo-biswas 3y ago
  Had the lawyer used a fine-tuned model with some RAG to go along with it, I assume it’d have been fine. This is just people failing to use a technology correctly, like riding a bike without a helmet.
happytiger 3y ago
A hundredth the price and a quarter the quality means that this is here to stay. Might be a little early in the accuracy phase to start riding AI written briefs into court unchecked, but then I’ve never met a lawyer who didn’t try to make their billing efficient. But logically, since all that is needed is improved accuracy it’s more likely that improved accuracy will be the answer rather than any change in human behavior.
- A_D_E_P_T 3y ago
  > A hundredth the price and a quarter the quality means that this is here to stay No, it's simply that those noobs don't know how to use LLMs. They'll eventually learn. Basically, you don't use them to dig up new information, unless you're extremely careful about triple-checking that information. Google Scholar's legal database search is better for that. You use LLMs to write boilerplate, paraphrase, edit, and synthesize information from your own sources. Do it properly, and you'll never "hallucinate" a fake case in one of your legal filings, and you'll be able to write 'em in 5% of the time.
  - consp 3y ago
    > Do it properly, and you'll never "hallucinate" a fake case in one of your legal filings, and you'll be able to write 'em in 5% of the time. All fun and games for those who can and good for them, but I'm betting the majority can't or won't. The result being society pays for those later ones incompetence.
- sgt101 3y ago
  I've led a team building an LLM agent for customer service. Our finding is that it's between 50% and 10% of the operational cost of a human for a case. This costing is based on the range of costs for offshore vs. nearshore workers and doesn't account for a lot of the overhead of a human powered service organisation (in the jargon this isn't fully loaded). I believe that the real cost is about 20% if dev expenses are included - but that's just my view of where inbetween the bounds thing come to rest. Now, that's not 100th. In terms of quality, there are things it can't do and despite our architecture (which is aimed at managing the deficiencies of LLM's) we still see some hallucinations creeping through. For example our encoder has problems with directionality as in it will write text like "average transaction value declined from $150 to $154 in october." We can catch (in our tests anyway) all the mistakes about the values, but the actual textual phrasing is hard to check - at the level where hard means I think that the value of the system doesn't justify it. I think, from customer feedback, that this sort of thing will be ok for the apps we are building, but it is a real problem with this generation of models and it's not clear to me that it will be solved in the future (although like everyone else I was blindsided by the jump from GPT3 to 4 so who knows).
yosito 3y ago
The danger of "AI" is that we actually believe the plausible fabrications it produces are "intelligent". The other day, I debated a guy who thought that the utopian future was governments run by AI. He was convinced that the AI would always make the perfect, optimal decision in any circumstance. The scary thing to me is that LLMs are probably really good at fabricating the kind of brain dead lies that get corrupt politicians into power.
- blibble 3y ago
  > The danger of "AI" is that we actually believe the plausible fabrications it produces are "intelligent". the thing I find scary is you see people on here believing this if a website full of techies can't tell the difference what chance do the general public have? I suspect the invention of the LLM will be the final nail in the coffin of liberal democracy (after the invention of social media)
  - extheat 3y ago
    People should never have blindly trusted anything on the internet in the first place. As far as I’m aware, most of the fears like this are overrated. People will live, learn and adapt as we always have. Constantly complaining about doom and having pessimism about anything seemingly scary is not a great way to live a fulfilling life.
- shzhdbi09gv8ioi 3y ago
  > The danger of "AI" is that we actually believe the plausible fabrications it produces are "intelligent". 100% this! I'm fed up with seemingly rational people who just can't comprehend that the AI hype is just sales talk. I had to convince a customer the other day we cannot write whole apps with ChatGPT. As far as I know, not a single example exists of a full app written by ChatGPT. It just can't be done, because ChatGPT is not reasoning!!! It is not intelligent. It can just spit out seemingly coherent, but often incorrect, snippets of code. I think it's best described as a word-calculator, or autocomplete on crack, in the sense that it is great on guessing what comes next. But it has no reasoning behind it's predictions, only statistics. EDIT: The AI apologetics are downvoting me LOL
flanked-evergl 3y ago
How are language models doing what they are known to do newsworthy? This feels a bit like reporting that water is wet.
- weevil 3y ago
  The story isn't about LLMs doing LLM stuff. It's about lawyers using LLMs as a shortcut for proper legal work, laboring under the delusion that it is entirely accurate, honest and 'intelligent', and the ramifications for the legal system.
  - flanked-evergl 3y ago
    Fair enough, that is newsworthy.
- ZaoLahma 3y ago
  I guess the value in reporting it is that for most people, for us on HN as well, computing is considered accurate. You can trust the output if you trust the input and the program that processes the input. That is what we expect and value in computing - accuracy. For LLMs that's not really the case anymore and it needs to be highlighted that "computers" no longer necessarily produce accurate output, to make sure not too much faith is put in what they produce.
  - flanked-evergl 3y ago
    > "computers" no longer necessarily produce accurate output This was always the case. Just because a computer executes your model, doesn't mean your model has any bearing on reality. This is not a new phenomenon.
eurekin 3y ago
Right. It's the AI that is the problem. I have another use case for LLM's, I haven't thought of before: absolution of responsibility. The public is already primed to focus on AI in such cases.
- b112 3y ago
  In the early 90s, when every business was moving full out into computing, I'd call for an issue. Sometimes the person helping me, would say "I'm sorry, the computer won't let me do that". This seemed to grow as a response, until, to protect myself, I started pushing back. "I don't care, you guys owe me $x, and telling me your computer won't let you do it is absurd. Pay me." Most people would then get a manager on the phone, who would ape the "computer was at fault" line, but after pushback.. surprise!, there was a way to resolve things. Point is, abdicating responsibility is an easy out for the lazy and for corporate greed. And AI will surely be used in this fashion, because people love an out. Amazon is already the worst of this, if you have an issue that doesn't fit in a premade solution, good luck getting any fix! Taking humans out of the chain is always bad. Just look at the UK post office scandal, for the dark side of this.
  - eurekin 3y ago
    Calls very close to home... It really feels like playing a whack-a-mole, never really progressing towards any reasonable level of responsibility
- dspillett 3y ago
  > absolution of responsibility There are some still fighting the fight, but this is already happening¹ when it comes to failure to attribute and other licensing issues. “I didn't copy/misattribute/other your code, an AI trained in part by it was ‘inspired’ by it amongst other code”. ---- [1] unless you believe the assurances that an LLM will never regurgitate chunks of training material, like the image based models have been shown to
happymellon 3y ago
> In one case, a judge imposed a fine on New York lawyers who submitted a legal brief with imaginary cases hallucinated by ChatGPT — an incident the lawyers maintained was a good-faith error. They need to be disbarred. Submitting legal filings that contain errors because you used ChatGPT to make up crap is the opposite of a "good-faith" error.
- speedgoose 3y ago
  If the user didn't understand that ChatGPT make up crap sometimes, despite the warnings everywhere about it that they may not read, it could still be a good-faith error to me. ChatGPT was just released.
  - unsupp0rted 3y ago
    If the user is a lawyer who officially referenced a case that a) they never read and b) never existed, then the user can’t be allowed to practice law anymore.
    - CPLX 3y ago
      Every single lawyer references cases they never actually read in every filing.
      - deleted 3y ago
        [deleted]
      - averageRoyalty 3y ago
        Then we're discovering together that they all should be disbarred then....
  - happymellon 3y ago
    If the user doesn't understand their tool then they should review what it spits out. > I found a guy who claims to have the sum of all knowledge, and I've just copied and pasted code from him that I haven't reviewed. Then either your an idiot and should be disbarred or you are negligent and should be disbarred. As a lawyer you have people's lives in your hands, this is not the time for "woops, sorry I just couldn't be arsed to read what I submitted".
guil177 3y ago
The title of the article should be: somebody faked a legal case using AI.
mrtksn 3y ago
Isn’t “hallucination” named after a human phenomena? People too remember things that never happened. Wouldn’t be solvable with a second AI agent which checks the output of the first one and be like “bro, you sure about that? I never heard of it”. In my experience with LLMs, they don’t insist when corrected, instead they apologize and generate a response with that correction in mind.
- thriftwy 3y ago
  I wonder how many AI hallucinations stem from the fact that nowhere in the prompt it was said that information have to be real. And this cannot be implicitly assumed since humans like fiction in other contexts.
- Closi 3y ago
  It's absolutely solvable - and presumably this is the sort of functionality that more legal-specific AI applications offer (probably mixed with some sort of knowledge graph that contains facts). Seems like the issue here was that a lawyer was just using OOTB ChatGPT.
  - tiborsaas 3y ago
    I'm not so sure with generic LLM-s. With what I've seen so far in how LLM-s work "hallucination" is a feature, not a bug. In other use cases it's called creativity when the user wants new ideas generated or filling in the missing spots. In these cases they are really happy that it can hallucinate, but of course they don't call it like that. So the dilemma is that we need to eliminate the same thing in one case but improve it in others. But I believe you are right that we can engineer a fine tuned system that can use tools to fact check using traditional techniques.
- supriyo-biswas 3y ago
  > they don’t insist when corrected That’s just OpenAI’s RLHF and instruct tuning though. Bing Chat’s Sydney had the temperament of a moody teenager and would often contradict or accuse people of being wrong when pushed. The correct way to solve this is to use retrieval augmented generation (finding relevant cases using embeddings and then feeding the data along with the question), so that the model is grounded to the truth.
_fizz_buzz_ 3y ago
I think people under and overestimate AI at the same time. E.g. I asked ChatGPT4 to draw me a schematic of a simple buck converter (i.e. 4 components + load). In the written response it got the basics right. Drawing that schematic is completely garbled non-sense. I was expecting something like this maybe: https://en.wikipedia.org/wiki/Buck_converter#/media/File:Buck_conventions.svg https://en.wikipedia.org/wiki/Buck_converter#/media/File:Buc... I got this: https://imgur.com/a/tEqprGq https://imgur.com/a/tEqprGq
- bobim 3y ago
  AI version is so much better!
  - deleted 3y ago
    [deleted]
- hackerlight 3y ago
  Is GPT-4 just passing a prompt to DALL-E to create an image? The garbled diagram makes sense since DALL-E isn't supposed to be that intelligent or an AGI.
  - _fizz_buzz_ 3y ago
    Yes it did. And this was the description ChatGPT put underneath it: "Here's the illustration of a basic buck converter schematic. This diagram includes the input voltage source (Vin), the switch (transistor), diode, inductor (L), capacitor (C), output voltage (Vout), and the load (represented as a resistor). The connections between these components and the flow of current are also shown. This should help you understand the basic operation of a buck converter." Which is a pretty good description of the components. However, it then gaslights me into believing this is somehow depicted in the picture Dall-E produced.
    - hackerlight 3y ago
      So even if GPT-4 "understands" what needs to be drawn (in the sense that the higher-level concepts and relations between them are embedded in its weights), it's difficult to get DALL-E to draw it correctly because DALL-E doesn't have that same "understanding". OpenAI will need to combine these models somehow, or train DALL-E so that it has the same conceptual understanding.
taffronaut 3y ago
From TFA, "the case was a high-net-worth family matter" so probably not an existential threat to anyone.
lapcat 3y ago
I haven't used or paid much attention to ChatGPT, but the other day I was reading a macOS question on Reddit, and one of the "answers" was completely bizarre, claiming that the Apple Launchpad app was developed by Canonical. I checked the commenter's bio, and sure enough, they were a prolific ChatGPT user. It also turns out that Canonical has a product called Launchpad, which was the basis of ChatGPT's mindlessly wrong answer. The scary thing is that even though ChatGPT's response was completely detached from reality, it was articulate and sounded authoritative, easily capable of fooling someone who wasn't aware of the facts. It seems to me that these "AI tools" are a menace in a society already rife with misinformation. Of course the Reddit commenter didn't have the decency to preface their comment with a disclaimer about how it was generated. I'm not looking forward to the future of this.
elzbardico 3y ago
The problem with AI is not the AI itself, it is people. Dumb people. Dumb people with credentials and power.

43 points by luu 3y ago

jruohonen 3y ago

It is spreading like a wildfire. Yet, the question remains about the repercussions.

echelon 3y ago
We never talked about the Internet's repercussions, and yet here we are. Did we do a good job, or should we have left that stone unturned?
- badgersnake 3y ago
  We have been talking about them since day 1 and we still are. DMCA, the existence of the EFF, cryptocurrency regulations are all examples of this.
- hutzlibu 3y ago
  "Did we do a good job, or should we have left that stone unturned?" Regarding the quality of the average Website - no , but also no to the second. The time of the internet was just there with no precedent and now it is hyped imperfect AIs we have to deal with. I think the main problem in both cases is, that most people don't have a clue at all, how it works. (And too many of them are in positions of power).
- jruohonen 3y ago
  > Did we do a good job, or should we have left that stone unturned? It's kind of a hangover, don't you think? The past decade, I mean, with the Internet.
supriyo-biswas 3y ago
Had the lawyer used a fine-tuned model with some RAG to go along with it, I assume it’d have been fine. This is just people failing to use a technology correctly, like riding a bike without a helmet.

happytiger 3y ago

A hundredth the price and a quarter the quality means that this is here to stay. Might be a little early in the accuracy phase to start riding AI written briefs into court unchecked, but then I’ve never met a lawyer who didn’t try to make their billing efficient. But logically, since all that is needed is improved accuracy it’s more likely that improved accuracy will be the answer rather than any change in human behavior.

A_D_E_P_T 3y ago
> A hundredth the price and a quarter the quality means that this is here to stay No, it's simply that those noobs don't know how to use LLMs. They'll eventually learn. Basically, you don't use them to dig up new information, unless you're extremely careful about triple-checking that information. Google Scholar's legal database search is better for that. You use LLMs to write boilerplate, paraphrase, edit, and synthesize information from your own sources. Do it properly, and you'll never "hallucinate" a fake case in one of your legal filings, and you'll be able to write 'em in 5% of the time.
- consp 3y ago
  > Do it properly, and you'll never "hallucinate" a fake case in one of your legal filings, and you'll be able to write 'em in 5% of the time. All fun and games for those who can and good for them, but I'm betting the majority can't or won't. The result being society pays for those later ones incompetence.
sgt101 3y ago
I've led a team building an LLM agent for customer service. Our finding is that it's between 50% and 10% of the operational cost of a human for a case. This costing is based on the range of costs for offshore vs. nearshore workers and doesn't account for a lot of the overhead of a human powered service organisation (in the jargon this isn't fully loaded). I believe that the real cost is about 20% if dev expenses are included - but that's just my view of where inbetween the bounds thing come to rest. Now, that's not 100th. In terms of quality, there are things it can't do and despite our architecture (which is aimed at managing the deficiencies of LLM's) we still see some hallucinations creeping through. For example our encoder has problems with directionality as in it will write text like "average transaction value declined from $150 to $154 in october." We can catch (in our tests anyway) all the mistakes about the values, but the actual textual phrasing is hard to check - at the level where hard means I think that the value of the system doesn't justify it. I think, from customer feedback, that this sort of thing will be ok for the apps we are building, but it is a real problem with this generation of models and it's not clear to me that it will be solved in the future (although like everyone else I was blindsided by the jump from GPT3 to 4 so who knows).

yosito 3y ago

The danger of "AI" is that we actually believe the plausible fabrications it produces are "intelligent". The other day, I debated a guy who thought that the utopian future was governments run by AI. He was convinced that the AI would always make the perfect, optimal decision in any circumstance. The scary thing to me is that LLMs are probably really good at fabricating the kind of brain dead lies that get corrupt politicians into power.

blibble 3y ago
> The danger of "AI" is that we actually believe the plausible fabrications it produces are "intelligent". the thing I find scary is you see people on here believing this if a website full of techies can't tell the difference what chance do the general public have? I suspect the invention of the LLM will be the final nail in the coffin of liberal democracy (after the invention of social media)
- extheat 3y ago
  People should never have blindly trusted anything on the internet in the first place. As far as I’m aware, most of the fears like this are overrated. People will live, learn and adapt as we always have. Constantly complaining about doom and having pessimism about anything seemingly scary is not a great way to live a fulfilling life.
shzhdbi09gv8ioi 3y ago
> The danger of "AI" is that we actually believe the plausible fabrications it produces are "intelligent". 100% this! I'm fed up with seemingly rational people who just can't comprehend that the AI hype is just sales talk. I had to convince a customer the other day we cannot write whole apps with ChatGPT. As far as I know, not a single example exists of a full app written by ChatGPT. It just can't be done, because ChatGPT is not reasoning!!! It is not intelligent. It can just spit out seemingly coherent, but often incorrect, snippets of code. I think it's best described as a word-calculator, or autocomplete on crack, in the sense that it is great on guessing what comes next. But it has no reasoning behind it's predictions, only statistics. EDIT: The AI apologetics are downvoting me LOL

flanked-evergl 3y ago

How are language models doing what they are known to do newsworthy? This feels a bit like reporting that water is wet.

weevil 3y ago
The story isn't about LLMs doing LLM stuff. It's about lawyers using LLMs as a shortcut for proper legal work, laboring under the delusion that it is entirely accurate, honest and 'intelligent', and the ramifications for the legal system.
- flanked-evergl 3y ago
  Fair enough, that is newsworthy.
ZaoLahma 3y ago
I guess the value in reporting it is that for most people, for us on HN as well, computing is considered accurate. You can trust the output if you trust the input and the program that processes the input. That is what we expect and value in computing - accuracy. For LLMs that's not really the case anymore and it needs to be highlighted that "computers" no longer necessarily produce accurate output, to make sure not too much faith is put in what they produce.
- flanked-evergl 3y ago
  > "computers" no longer necessarily produce accurate output This was always the case. Just because a computer executes your model, doesn't mean your model has any bearing on reality. This is not a new phenomenon.

eurekin 3y ago

Right. It's the AI that is the problem. I have another use case for LLM's, I haven't thought of before: absolution of responsibility. The public is already primed to focus on AI in such cases.

b112 3y ago
In the early 90s, when every business was moving full out into computing, I'd call for an issue. Sometimes the person helping me, would say "I'm sorry, the computer won't let me do that". This seemed to grow as a response, until, to protect myself, I started pushing back. "I don't care, you guys owe me $x, and telling me your computer won't let you do it is absurd. Pay me." Most people would then get a manager on the phone, who would ape the "computer was at fault" line, but after pushback.. surprise!, there was a way to resolve things. Point is, abdicating responsibility is an easy out for the lazy and for corporate greed. And AI will surely be used in this fashion, because people love an out. Amazon is already the worst of this, if you have an issue that doesn't fit in a premade solution, good luck getting any fix! Taking humans out of the chain is always bad. Just look at the UK post office scandal, for the dark side of this.
- eurekin 3y ago
  Calls very close to home... It really feels like playing a whack-a-mole, never really progressing towards any reasonable level of responsibility
dspillett 3y ago
> absolution of responsibility There are some still fighting the fight, but this is already happening¹ when it comes to failure to attribute and other licensing issues. “I didn't copy/misattribute/other your code, an AI trained in part by it was ‘inspired’ by it amongst other code”. ---- [1] unless you believe the assurances that an LLM will never regurgitate chunks of training material, like the image based models have been shown to

happymellon 3y ago

> In one case, a judge imposed a fine on New York lawyers who submitted a legal brief with imaginary cases hallucinated by ChatGPT — an incident the lawyers maintained was a good-faith error. They need to be disbarred. Submitting legal filings that contain errors because you used ChatGPT to make up crap is the opposite of a "good-faith" error.

speedgoose 3y ago
If the user didn't understand that ChatGPT make up crap sometimes, despite the warnings everywhere about it that they may not read, it could still be a good-faith error to me. ChatGPT was just released.
- unsupp0rted 3y ago
  If the user is a lawyer who officially referenced a case that a) they never read and b) never existed, then the user can’t be allowed to practice law anymore.
  - CPLX 3y ago
    Every single lawyer references cases they never actually read in every filing.
    - deleted 3y ago
      [deleted]
    - averageRoyalty 3y ago
      Then we're discovering together that they all should be disbarred then....
- happymellon 3y ago
  If the user doesn't understand their tool then they should review what it spits out. > I found a guy who claims to have the sum of all knowledge, and I've just copied and pasted code from him that I haven't reviewed. Then either your an idiot and should be disbarred or you are negligent and should be disbarred. As a lawyer you have people's lives in your hands, this is not the time for "woops, sorry I just couldn't be arsed to read what I submitted".

guil177 3y ago

The title of the article should be: somebody faked a legal case using AI.

mrtksn 3y ago

Isn’t “hallucination” named after a human phenomena? People too remember things that never happened. Wouldn’t be solvable with a second AI agent which checks the output of the first one and be like “bro, you sure about that? I never heard of it”. In my experience with LLMs, they don’t insist when corrected, instead they apologize and generate a response with that correction in mind.

thriftwy 3y ago
I wonder how many AI hallucinations stem from the fact that nowhere in the prompt it was said that information have to be real. And this cannot be implicitly assumed since humans like fiction in other contexts.
Closi 3y ago
It's absolutely solvable - and presumably this is the sort of functionality that more legal-specific AI applications offer (probably mixed with some sort of knowledge graph that contains facts). Seems like the issue here was that a lawyer was just using OOTB ChatGPT.
- tiborsaas 3y ago
  I'm not so sure with generic LLM-s. With what I've seen so far in how LLM-s work "hallucination" is a feature, not a bug. In other use cases it's called creativity when the user wants new ideas generated or filling in the missing spots. In these cases they are really happy that it can hallucinate, but of course they don't call it like that. So the dilemma is that we need to eliminate the same thing in one case but improve it in others. But I believe you are right that we can engineer a fine tuned system that can use tools to fact check using traditional techniques.
supriyo-biswas 3y ago
> they don’t insist when corrected That’s just OpenAI’s RLHF and instruct tuning though. Bing Chat’s Sydney had the temperament of a moody teenager and would often contradict or accuse people of being wrong when pushed. The correct way to solve this is to use retrieval augmented generation (finding relevant cases using embeddings and then feeding the data along with the question), so that the model is grounded to the truth.

_fizz_buzz_ 3y ago

I think people under and overestimate AI at the same time. E.g. I asked ChatGPT4 to draw me a schematic of a simple buck converter (i.e. 4 components + load). In the written response it got the basics right. Drawing that schematic is completely garbled non-sense. I was expecting something like this maybe: https://en.wikipedia.org/wiki/Buck_converter#/media/File:Buck_conventions.svg https://en.wikipedia.org/wiki/Buck_converter#/media/File:Buc... I got this: https://imgur.com/a/tEqprGq https://imgur.com/a/tEqprGq

bobim 3y ago
AI version is so much better!
- deleted 3y ago
  [deleted]
hackerlight 3y ago
Is GPT-4 just passing a prompt to DALL-E to create an image? The garbled diagram makes sense since DALL-E isn't supposed to be that intelligent or an AGI.
- _fizz_buzz_ 3y ago
  Yes it did. And this was the description ChatGPT put underneath it: "Here's the illustration of a basic buck converter schematic. This diagram includes the input voltage source (Vin), the switch (transistor), diode, inductor (L), capacitor (C), output voltage (Vout), and the load (represented as a resistor). The connections between these components and the flow of current are also shown. This should help you understand the basic operation of a buck converter." Which is a pretty good description of the components. However, it then gaslights me into believing this is somehow depicted in the picture Dall-E produced.
  - hackerlight 3y ago
    So even if GPT-4 "understands" what needs to be drawn (in the sense that the higher-level concepts and relations between them are embedded in its weights), it's difficult to get DALL-E to draw it correctly because DALL-E doesn't have that same "understanding". OpenAI will need to combine these models somehow, or train DALL-E so that it has the same conceptual understanding.

taffronaut 3y ago

From TFA, "the case was a high-net-worth family matter" so probably not an existential threat to anyone.

lapcat 3y ago

I haven't used or paid much attention to ChatGPT, but the other day I was reading a macOS question on Reddit, and one of the "answers" was completely bizarre, claiming that the Apple Launchpad app was developed by Canonical. I checked the commenter's bio, and sure enough, they were a prolific ChatGPT user. It also turns out that Canonical has a product called Launchpad, which was the basis of ChatGPT's mindlessly wrong answer. The scary thing is that even though ChatGPT's response was completely detached from reality, it was articulate and sounded authoritative, easily capable of fooling someone who wasn't aware of the facts. It seems to me that these "AI tools" are a menace in a society already rife with misinformation. Of course the Reddit commenter didn't have the decency to preface their comment with a disclaimer about how it was generated. I'm not looking forward to the future of this.

elzbardico 3y ago

The problem with AI is not the AI itself, it is people. Dumb people. Dumb people with credentials and power.