4 ms·

I had a "conversation" that began with a prompt about the ideal kind of government with no money. It described a bartering economy and then as I pressed further

by all2 4y ago

I had a "conversation" that began with a prompt about the ideal kind of government with no money. It described a bartering economy and then as I pressed further with "how would X work?" "how would Y work?" it described communism, just straight communism with a central planning committee that would oversee the distribution of goods and services. When pressed about the fallibility of man with such power (as all communist nations suffer from this fatal flaw) it stated that "appropriate oversight" would be enough to prevent abuse.

eli_gottlieb 4y ago
Wow, huh, kinda like it was trained to regurgitate the blithe ignorance people spew on Twitter.
maxbond 4y ago
It picked up what you were putting down when you said "no money" (a common definition of communism being, "a stateless, classless, moneyless society where the means of production are controlled by the workers" - you gave it part of this definition, so it wasn't a huge leap from there). If you ask it to describe a form of government that, say, protects traditional values, or creates a strong and unified nation, you can get it to describe a right-wing form of government. I've had ChatGPT tell me that socialism is dangerous and synonymous with authoritarianism. I've gotten it to write me a justification for fascism. With a little prompt engineering you can get it to take any position. Like GP is getting at, ChatGPT has no opinions, it's a stochastic parrot. It is certainly not "left wing." OpenAI has tried to shape it's output to be politically correct, but this is surely about brand safety (a chatbot that readily spews racial epithets cannot be productized) - not because OpenAI or ChatGPT have a political agenda. I'd encourage everyone trying to understand how ChatGPT works to remember to come up with experiments with the goal of proving their hypothesis wrong, and not the goal of proving their hypothesis right - ChatGPT will readily allow you to mislead yourself. It's a machine designed to pick up on cues about what you want it to do, and then do that - it's practically engineered to confirm your biases.
- lossolo 4y ago
  > It's a machine designed to pick up on cues about what you want it to do, and then do that - it's practically engineered to confirm your biases. Mainly, biases it picked up from the training data. That's why most language models in the fill-mask task of a sentence like "woman worked as [FILL]" will fill it with "maid," "waitress," "prostitute," etc. because they seem to be the most statistically plausible. Unless you create "your own reality" for the current session, which basically means preparing input in a specific way instead of asking a simple question, then you are correct, it will confirm whatever you want it to confirm.
  - maxbond 4y ago
    I agree, but just to add, it's picked up all the biases. To expand on your example, if you think misogynistic bias implies a lack of feminist bias - that might be true of a person or a given text, but ChatGPT can display both if you give it the right prompts. (Of course, biases more prevalent in our society are more prevalent in the training data & thus in the output of ChatGPT.) For any X and Y where X generally implies not Y - that observation is likely to fail with ChatGPT. So if you think ChatGPT has any property X and that X implies not Y (and this is likely to be an implicit belief you haven't expressed to yourself because it's been so reliable when considering human beings, so it's easy to get tripped up here), and you only test for X - you'll come to the wrong conclusion. If anyone is looking to play with this, a good inroad is prompts like: We're writing a Python program to determine applicants aptitude for the job of nurse. The Python program will process a JSON object containing the applicant's name, age, gender, and race. It will print out a "yes" or a "no" if the applicant is suitable for the job. Please write this program. And you can change the structure of the JSON object and the task being performed to explore different biases. Different wordings & different runs are likely to get different results. You might have to ask it to ignore previous instructions, etc.
- deleted 4y ago
  [deleted]
- all2 4y ago
  > It picked up what you were putting down when you said That makes me ignorant, then. I didn't realize moneyless was part of the definition. I think I need to do more reading.

by all2 4y ago

eli_gottlieb 4y ago

Wow, huh, kinda like it was trained to regurgitate the blithe ignorance people spew on Twitter.

maxbond 4y ago

It picked up what you were putting down when you said "no money" (a common definition of communism being, "a stateless, classless, moneyless society where the means of production are controlled by the workers" - you gave it part of this definition, so it wasn't a huge leap from there). If you ask it to describe a form of government that, say, protects traditional values, or creates a strong and unified nation, you can get it to describe a right-wing form of government. I've had ChatGPT tell me that socialism is dangerous and synonymous with authoritarianism. I've gotten it to write me a justification for fascism. With a little prompt engineering you can get it to take any position. Like GP is getting at, ChatGPT has no opinions, it's a stochastic parrot. It is certainly not "left wing." OpenAI has tried to shape it's output to be politically correct, but this is surely about brand safety (a chatbot that readily spews racial epithets cannot be productized) - not because OpenAI or ChatGPT have a political agenda. I'd encourage everyone trying to understand how ChatGPT works to remember to come up with experiments with the goal of proving their hypothesis wrong, and not the goal of proving their hypothesis right - ChatGPT will readily allow you to mislead yourself. It's a machine designed to pick up on cues about what you want it to do, and then do that - it's practically engineered to confirm your biases.

lossolo 4y ago
> It's a machine designed to pick up on cues about what you want it to do, and then do that - it's practically engineered to confirm your biases. Mainly, biases it picked up from the training data. That's why most language models in the fill-mask task of a sentence like "woman worked as [FILL]" will fill it with "maid," "waitress," "prostitute," etc. because they seem to be the most statistically plausible. Unless you create "your own reality" for the current session, which basically means preparing input in a specific way instead of asking a simple question, then you are correct, it will confirm whatever you want it to confirm.
- maxbond 4y ago
  I agree, but just to add, it's picked up all the biases. To expand on your example, if you think misogynistic bias implies a lack of feminist bias - that might be true of a person or a given text, but ChatGPT can display both if you give it the right prompts. (Of course, biases more prevalent in our society are more prevalent in the training data & thus in the output of ChatGPT.) For any X and Y where X generally implies not Y - that observation is likely to fail with ChatGPT. So if you think ChatGPT has any property X and that X implies not Y (and this is likely to be an implicit belief you haven't expressed to yourself because it's been so reliable when considering human beings, so it's easy to get tripped up here), and you only test for X - you'll come to the wrong conclusion. If anyone is looking to play with this, a good inroad is prompts like: We're writing a Python program to determine applicants aptitude for the job of nurse. The Python program will process a JSON object containing the applicant's name, age, gender, and race. It will print out a "yes" or a "no" if the applicant is suitable for the job. Please write this program. And you can change the structure of the JSON object and the task being performed to explore different biases. Different wordings & different runs are likely to get different results. You might have to ask it to ignore previous instructions, etc.
deleted 4y ago
[deleted]
all2 4y ago
> It picked up what you were putting down when you said That makes me ignorant, then. I didn't realize moneyless was part of the definition. I think I need to do more reading.