5 ms·

And this seems actually wildly reasonable. It’s actually pretty scammy to take people’s money (or whatever your business model is) for legal or medical advice j

by xp84 1y ago

And this seems actually wildly reasonable. It’s actually pretty scammy to take people’s money (or whatever your business model is) for legal or medical advice just to give them whatever AI shits out. If I wanted ChatGPT’s opinion (as I actually often do) I’d just ask ChatGPT for free or nearly-free. Anyone repackaging ChatGPT’s output as fit for a specialized purpose is scamming their customers.

left-struck 1y ago
Your comment actually hints at this towards the end but yeah this doesn’t just apply to medical and legal topics.
VBprogrammer 1y ago
You are failing to recognise all of the hard work which goes into "prompt engineering" to get AI to magically work!
- oblio 1y ago
  It's the hardest work of all, it's basically voodoo, or put more generously, alchemy. Thankfully we have progressed so this time it will probably take less than 1000 years to progress to full blown chemistry :-)
  - DiscourseFan 1y ago
    Its more and more of a science everyday
    - oblio 1y ago
      Yes, a more correct comparison would be early medicine: a science, but still filled with leeches and lancets. Oh, and another thing, we still aren't able to quantify if AI coding is a net benefit. In my use cases the biggest benefit I get from it is as an enhanced code search, basically. Which has value, but I'm not sure it's a "$1 trillion per year business sector" kind of value.
amelius 1y ago
Wait, so we cannot use the API anymore?
- fukka42 1y ago
  To scam people? No.
miki123211 1y ago
What if Chat GPT is used as a "power tool" for actual legal / medical professionals? E.G. Here's the set of documents uploaded by the customer. Here are my most important highlights (notice that document #68 is an official letter from last year informing the person that form 68d doesn't need to be filled in their specific situation). Here are my takeaways. Here's my recommendation. Apporve or change?
- agentcoops 1y ago
  As long as their service offering is better than me typing the question into my phone I’m fine with it.
- bee_rider 1y ago
  From the article > The AI research company updated its usage policies on Oct. 29 to clarify that users of ChatGPT can’t use the service for “tailored advice that requires a license, such as legal or medical advice, without appropriate involvement by a licensed professional.” I think your use case would probably be fine (although I am not a licensed professional, hahaha). I think they mostly want a licensed professional in the “driver seat,” or at least there to eat the responsibility. It is sort of funny that coding agents became a thing, in this context. The responsibility-sink is irrelevant I guess because code writing doesn’t generate any responsibility as a byproduct.
- alistairSH 1y ago
  In the legal context, AI is still prone to errors. And the legal system (in the US) doesn't have much in the way of guardrails. Here's a case of several judges getting caught using AI to write error-filled judgements[1]. So it's not just lazy lawyers (several news articles over the past 3 years or so), but the courts themselves! 1 - https://www.reuters.com/sustainability/society-equity/two-federal-judges-say-use-ai-led-errors-us-court-rulings-2025-10-23/ https://www.reuters.com/sustainability/society-equity/two-fe...
- dotancohen 1y ago
  I would expect that to work exactly how I code with it. I would examine the inferred output, decide if it is fit for purpose, and if so then commit it. If not, I would either refine my prompt or make the fixes to the inferred code myself. Would this hypothetical legal professional not verify that the inferred output (which is really a prediction) is correct?
agentcoops 1y ago
Indeed. Similarly, I like now having ChatGPT as the absolute lowest bar of service I’m willing to accept. If you aren’t a better therapist, lawyer or programmer than it then why would I think about paying you.
- sabatonfan 1y ago
  Having chatgpt as your therapist is actually wild. Its yes-man abilities etc. will not help you and its always helpful to talk to another human being in the loop during therapy so that you know you are actually being sane. Don't ever compromise money on your mental health. If you need to, either join offline anonymous support groups or join some good forums/discord servers/reddit about therapy if you can not even afford that. They are good enough and I think personally for me, whats therapatic is to try to understand and help the other person who was struggling just as I once was y'know But saying to use chatgpt as a therapist is just something that I hate a lot simply because in my opinion, it might actually even be harmful but I am not sure.
  - agentcoops 1y ago
    I didn’t say to use ChatGPT as a therapist——I said don’t use a human therapist who is worse than it.
- freejazz 1y ago
  The exact kind of customer I would not want to have anyway.
altcognito 1y ago
> Anyone repackaging ChatGPT’s output as fit for a specialized purpose is scamming their customers. This is genuinely hilarious given what LLMs are.

by xp84 1y ago

left-struck 1y ago

Your comment actually hints at this towards the end but yeah this doesn’t just apply to medical and legal topics.

VBprogrammer 1y ago

You are failing to recognise all of the hard work which goes into "prompt engineering" to get AI to magically work!

oblio 1y ago
It's the hardest work of all, it's basically voodoo, or put more generously, alchemy. Thankfully we have progressed so this time it will probably take less than 1000 years to progress to full blown chemistry :-)
- DiscourseFan 1y ago
  Its more and more of a science everyday
  - oblio 1y ago
    Yes, a more correct comparison would be early medicine: a science, but still filled with leeches and lancets. Oh, and another thing, we still aren't able to quantify if AI coding is a net benefit. In my use cases the biggest benefit I get from it is as an enhanced code search, basically. Which has value, but I'm not sure it's a "$1 trillion per year business sector" kind of value.

amelius 1y ago

Wait, so we cannot use the API anymore?

fukka42 1y ago
To scam people? No.

miki123211 1y ago

What if Chat GPT is used as a "power tool" for actual legal / medical professionals? E.G. Here's the set of documents uploaded by the customer. Here are my most important highlights (notice that document #68 is an official letter from last year informing the person that form 68d doesn't need to be filled in their specific situation). Here are my takeaways. Here's my recommendation. Apporve or change?

agentcoops 1y ago
As long as their service offering is better than me typing the question into my phone I’m fine with it.
bee_rider 1y ago
From the article > The AI research company updated its usage policies on Oct. 29 to clarify that users of ChatGPT can’t use the service for “tailored advice that requires a license, such as legal or medical advice, without appropriate involvement by a licensed professional.” I think your use case would probably be fine (although I am not a licensed professional, hahaha). I think they mostly want a licensed professional in the “driver seat,” or at least there to eat the responsibility. It is sort of funny that coding agents became a thing, in this context. The responsibility-sink is irrelevant I guess because code writing doesn’t generate any responsibility as a byproduct.
alistairSH 1y ago
In the legal context, AI is still prone to errors. And the legal system (in the US) doesn't have much in the way of guardrails. Here's a case of several judges getting caught using AI to write error-filled judgements[1]. So it's not just lazy lawyers (several news articles over the past 3 years or so), but the courts themselves! 1 - https://www.reuters.com/sustainability/society-equity/two-federal-judges-say-use-ai-led-errors-us-court-rulings-2025-10-23/ https://www.reuters.com/sustainability/society-equity/two-fe...
dotancohen 1y ago
I would expect that to work exactly how I code with it. I would examine the inferred output, decide if it is fit for purpose, and if so then commit it. If not, I would either refine my prompt or make the fixes to the inferred code myself. Would this hypothetical legal professional not verify that the inferred output (which is really a prediction) is correct?

agentcoops 1y ago

Indeed. Similarly, I like now having ChatGPT as the absolute lowest bar of service I’m willing to accept. If you aren’t a better therapist, lawyer or programmer than it then why would I think about paying you.

sabatonfan 1y ago
Having chatgpt as your therapist is actually wild. Its yes-man abilities etc. will not help you and its always helpful to talk to another human being in the loop during therapy so that you know you are actually being sane. Don't ever compromise money on your mental health. If you need to, either join offline anonymous support groups or join some good forums/discord servers/reddit about therapy if you can not even afford that. They are good enough and I think personally for me, whats therapatic is to try to understand and help the other person who was struggling just as I once was y'know But saying to use chatgpt as a therapist is just something that I hate a lot simply because in my opinion, it might actually even be harmful but I am not sure.
- agentcoops 1y ago
  I didn’t say to use ChatGPT as a therapist——I said don’t use a human therapist who is worse than it.
freejazz 1y ago
The exact kind of customer I would not want to have anyway.

altcognito 1y ago

> Anyone repackaging ChatGPT’s output as fit for a specialized purpose is scamming their customers. This is genuinely hilarious given what LLMs are.