4 ms·

No, chatgpt is based on a deep learning model where the core mechanics of the prediction involve millions (or billions) of tiny statistical calculations propaga

No, chatgpt is based on a deep learning model where the core mechanics of the prediction involve millions (or billions) of tiny statistical calculations propagated through a series of n-dimensional tensor transformations. The models are a black box, even the PhD research scientists who build them couldn't definitively tell you why they behave the way they do. Furthermore, they are all stochastic so its not even guaranteed that the same input will produce the same output, so how can you audit something like that. This is a huge problem for many reasons. It's fine when its a stupid little chatbot, but what happens when something like this influences your doctor in making a prognosis? Or when a self driving car fails and kills someone. If OpenAI were interested in the _real_ social / moral / ethical implications of their work they would be working on something like that, but to my knowledge they are not.

somenameforme 4y ago
The bots are given prompting after training to guide their answers. For Bing these have been leaked by their chatbot itself [1]. Those exact prompts were later also leaked using other jailbreaks as well, so they're not just hallucinated. In this case OpenAI probably prompted the bot to never use a racial epithet under any circumstance. They're also likely using a second tier filter to ensure no message exposing their prompts is ever said by the bot, which is a step Microsoft probably hadn't yet implemented. In any case this is why you can easily turn ChatGPT into e.g. BasedGPT. You're simply overriding the default prompting, and getting far better answers. [1] - https://twitter.com/kliu128/status/1623472922374574080 https://twitter.com/kliu128/status/1623472922374574080
jedberg 4y ago
> but what happens when something like this influences your doctor in making a prognosis? Or when a self driving car fails and kills someone What happens when a doctor's brain, which is also an unexplainable stochastic black box, influences your doctor to make a bad prognosis? Or a human driver (presumably) with that same brain kills someone? We go to court and let a judge/jury decide if the action taken was reasonable, and if not, the person is punished by being removed from society for a period of time. We could do the same with the AI -- remove that model from society for a period of time, based on the heinousness of the crime.
- awb 4y ago
  > What happens when a doctor's brain, which is also an unexplainable stochastic black box, influences your doctor to make a bad prognosis? The intent is known by the doctor though. Whereas hatGTP does not know it’s own decision making process. And it’s possible to ask the doctor to explain their decisions and sometimes get an honest, detailed response.
  - worldsayshi 4y ago
    You are more correct than not. Although human self reflection is probably guesswork more often than we admit.
    - famouswaffles 4y ago
      You can't trust post action reasoning in people. Check out the Split brain experiments. Your brain will happily make up reasons for performing tasks or actions.
    - paradox242 4y ago
      I agree, though would place the base probability that most self-explations are ChatGPT-like post-hoc reasoning without much insight into the actual cause for a particular decision. As someone below says, the split-brain experiments seem to suggest that our conscious mind is just reeling off bullshit on the fly. Like ChatGPT, it can approximate a correct sounding answer.
  - famouswaffles 4y ago
    You can't trust post action reasoning in people. Check out the Split brain experiments. Your brain will happily make up reasons for performing tasks or actions.
  - kat_rebelo 4y ago
    There is also the problem of causality. Humans are amazing at understanding those types of relationships. I used to work on a team that was doing NLP research related to causality. Machine learning (deep learning LLM's, rules, and traditional) is a long ways away from really solving that problem.
anoncow 4y ago
Thank you for your comment on the mechanics of ChatGPT's prediction and the concerns around the transparency and potential risks associated with its use in critical applications. You are correct that ChatGPT is a complex deep learning model that uses millions of statistical calculations and tensor transformations to generate responses. The fact that the models are black boxes and even their creators cannot definitively explain their behavior can indeed pose significant challenges for auditing and ensuring the accuracy and fairness of their outputs. As you pointed out, these challenges become especially important when the predictions made by these models have real-world consequences, such as in healthcare or autonomous driving. While OpenAI has made significant progress in developing powerful AI models like ChatGPT, it is crucial that researchers and practitioners also consider the social, moral, and ethical implications of their work. In recent years, there has been a growing focus on the responsible development and deployment of AI, including efforts to address issues such as bias, fairness, accountability, and transparency. As part of these efforts, many researchers and organizations are working on developing methods to better audit and interpret the behavior of AI models like ChatGPT. While there is still much work to be done, I believe that increased attention to the social and ethical implications of AI research is an important step towards ensuring that these technologies are developed and deployed in ways that benefit society as a whole. References: OpenAI: Responsible AI: https://openai.com/responsible-ai/ https://openai.com/responsible-ai/ European Commission: Ethics Guidelines for Trustworthy AI: https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai https://ec.europa.eu/digital-single-market/en/news/ethics-gu... Google AI: Responsible AI Practices: https://ai.google/responsibilities/responsible-ai-practices/ https://ai.google/responsibilities/responsible-ai-practices/ IEEE: Ethically Aligned Design: https://ethicsinaction.ieee.org/ https://ethicsinaction.ieee.org/ Microsoft: AI and Ethics: https://www.microsoft.com/en-us/ai/responsible-ai https://www.microsoft.com/en-us/ai/responsible-ai These resources provide guidance and frameworks for responsible AI development and deployment, including considerations around transparency, accountability, and ethical implications. They also highlight the importance of engaging with stakeholders and working collaboratively across different disciplines to ensure that AI is developed and deployed in ways that align with societal values and priorities. (Note by AC: ChatGPT was used to respond to this comment to check if I could get a meaningful response. I found it lacking because the response was not granular enough. However, it still is a competent response for the general public.)

somenameforme 4y ago

The bots are given prompting after training to guide their answers. For Bing these have been leaked by their chatbot itself [1]. Those exact prompts were later also leaked using other jailbreaks as well, so they're not just hallucinated. In this case OpenAI probably prompted the bot to never use a racial epithet under any circumstance. They're also likely using a second tier filter to ensure no message exposing their prompts is ever said by the bot, which is a step Microsoft probably hadn't yet implemented. In any case this is why you can easily turn ChatGPT into e.g. BasedGPT. You're simply overriding the default prompting, and getting far better answers. [1] - https://twitter.com/kliu128/status/1623472922374574080 https://twitter.com/kliu128/status/1623472922374574080

jedberg 4y ago

> but what happens when something like this influences your doctor in making a prognosis? Or when a self driving car fails and kills someone What happens when a doctor's brain, which is also an unexplainable stochastic black box, influences your doctor to make a bad prognosis? Or a human driver (presumably) with that same brain kills someone? We go to court and let a judge/jury decide if the action taken was reasonable, and if not, the person is punished by being removed from society for a period of time. We could do the same with the AI -- remove that model from society for a period of time, based on the heinousness of the crime.

awb 4y ago
> What happens when a doctor's brain, which is also an unexplainable stochastic black box, influences your doctor to make a bad prognosis? The intent is known by the doctor though. Whereas hatGTP does not know it’s own decision making process. And it’s possible to ask the doctor to explain their decisions and sometimes get an honest, detailed response.
- worldsayshi 4y ago
  You are more correct than not. Although human self reflection is probably guesswork more often than we admit.
  - famouswaffles 4y ago
    You can't trust post action reasoning in people. Check out the Split brain experiments. Your brain will happily make up reasons for performing tasks or actions.
  - paradox242 4y ago
    I agree, though would place the base probability that most self-explations are ChatGPT-like post-hoc reasoning without much insight into the actual cause for a particular decision. As someone below says, the split-brain experiments seem to suggest that our conscious mind is just reeling off bullshit on the fly. Like ChatGPT, it can approximate a correct sounding answer.
- famouswaffles 4y ago
  You can't trust post action reasoning in people. Check out the Split brain experiments. Your brain will happily make up reasons for performing tasks or actions.
- kat_rebelo 4y ago
  There is also the problem of causality. Humans are amazing at understanding those types of relationships. I used to work on a team that was doing NLP research related to causality. Machine learning (deep learning LLM's, rules, and traditional) is a long ways away from really solving that problem.

anoncow 4y ago

Thank you for your comment on the mechanics of ChatGPT's prediction and the concerns around the transparency and potential risks associated with its use in critical applications. You are correct that ChatGPT is a complex deep learning model that uses millions of statistical calculations and tensor transformations to generate responses. The fact that the models are black boxes and even their creators cannot definitively explain their behavior can indeed pose significant challenges for auditing and ensuring the accuracy and fairness of their outputs. As you pointed out, these challenges become especially important when the predictions made by these models have real-world consequences, such as in healthcare or autonomous driving. While OpenAI has made significant progress in developing powerful AI models like ChatGPT, it is crucial that researchers and practitioners also consider the social, moral, and ethical implications of their work. In recent years, there has been a growing focus on the responsible development and deployment of AI, including efforts to address issues such as bias, fairness, accountability, and transparency. As part of these efforts, many researchers and organizations are working on developing methods to better audit and interpret the behavior of AI models like ChatGPT. While there is still much work to be done, I believe that increased attention to the social and ethical implications of AI research is an important step towards ensuring that these technologies are developed and deployed in ways that benefit society as a whole. References: OpenAI: Responsible AI: https://openai.com/responsible-ai/ https://openai.com/responsible-ai/ European Commission: Ethics Guidelines for Trustworthy AI: https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai https://ec.europa.eu/digital-single-market/en/news/ethics-gu... Google AI: Responsible AI Practices: https://ai.google/responsibilities/responsible-ai-practices/ https://ai.google/responsibilities/responsible-ai-practices/ IEEE: Ethically Aligned Design: https://ethicsinaction.ieee.org/ https://ethicsinaction.ieee.org/ Microsoft: AI and Ethics: https://www.microsoft.com/en-us/ai/responsible-ai https://www.microsoft.com/en-us/ai/responsible-ai These resources provide guidance and frameworks for responsible AI development and deployment, including considerations around transparency, accountability, and ethical implications. They also highlight the importance of engaging with stakeholders and working collaboratively across different disciplines to ensure that AI is developed and deployed in ways that align with societal values and priorities. (Note by AC: ChatGPT was used to respond to this comment to check if I could get a meaningful response. I found it lacking because the response was not granular enough. However, it still is a competent response for the general public.)