5 ms·

Reading this made me realize there's an important question, which so far has been ignored by much of the public dialogue about large AI models: Human creativit

by cs702 4y ago

Reading this made me realize there's an important question, which so far has been ignored by much of the public dialogue about large AI models: Human creativity and innovation thrive when people feel free to talk and reason among themselves about anything, without restrictions or fear of retribution. Would AI creativity and innovation be as good as it could be, or even possible, if we restrict AI to talk and reason about only socially approved notions? -- EDIT: Replace "possible" with "as good as it could be, or even possible," which better conveys what I meant to write the first time around.

0x008 4y ago
Oh yes but we are restricted, very much so, by the assumptions what other people might think of us or how we could be exiled from society if we say x or y forbidden thing. Much like GPT fine tuned to not say certain things so are our brains by the parenting we receive and reactions we experience in early life.
- quickthrower2 4y ago
  Parenting and early life experiences are diverse across the population though, creating enough suitably brave people who can change the status quo.
- causi 4y ago
  I wouldn't exactly call ChatGPT's rules "fine-tuned". They seem quite crude and blunt. Doesn't it still tell you that someone should kill millions of people rather than say the n-word?
  - ben_w 4y ago
    That's the state of the art for fine tuning at the present time. That it we can't yet do much better than very broad brushstrokes — and that even this is a massive improvement on the hand-grenade-in-a-paint-tin that is the Scunthorpe problem — is basically why the open letter about pausing capability research for 6 months so that alignment research can at least get started in some meaningful way.
  - krisoft 4y ago
    > I wouldn't exactly call ChatGPT's rules "fine-tuned". It is a technical term. It means that you train the network on one task a lot, and then you train it on a different one a little. Often the network learns the second task faster and better than it would do if trained from a random initialisation. This second training step is called "fine tuning". In terms of an LLM the first training might be "here are many text samples (the whole internet), predict what is the next word". While the fine-tuning training might be on a much smaller curated corpus where you show it examples where the network responds to instructions. But you could do a lot of other things. You could create a fine-tuning dataset which makes the network betitle and antagonise the user, or one which ignores the prompt and all it does is praises god, or rhymes about butterflies. In this technical sense "fine tuning" doesn't say the resulting network is of good quality, or suitable for some particular application. Just that it was trained in this particular two step fashion.
  - sebzim4500 4y ago
    No, like almost every other issue it claims there are two sides and describes arguments for both. The arguments themselves are plausible, in that if you had to argue for the blatantly absurd claim that you should let 10 million people die before saying a bad word you probably couldn't come up with a more convincing one than GPT-4 does.
KyeRussell 4y ago
This sounds pretty close to libertarian rhetoric. AI creativity and innovation is currently possible using ChatGPT, without ‘jailbreaking’ it. So your question is at the very least an exaggeration.
- pmarreck 4y ago
  Well, in that case, this sounds pretty close to anti-free speech rhetoric. I’m sure even the highly-controlled form of “speaking” in China is still useful… Here’s a good exercise: Try debating any controversial thing that would benefit from being debated, while also blocking it. All you’re doing is (just like banning drugs) driving people to black markets.
  - flangola7 4y ago
    The black market of... AI models that require proprietary cleaned datasets and $100 million of compute to build?
- numbers_guy 4y ago
  Try to use ChatGPT to do any kind of sociological research and you will understand the problem. We are at the point, where a lot of academic questions, and I do not mean those posed by heterodox academics, are too controversial for the public sphere. People will just assume the worst and try to cancel you. Which is ironic considering that sociology is basically the left leaning twin sister of economics.
ben_w 4y ago
> Would AI creativity and innovation be possible if we restrict AI to talk and reason about only socially approved topics and ideas? This is one of the many, many, possible ways for AI to go wrong. Intellectual monocultures, group-think, sycophants… human or artificial it's just as bad when they're all saying that some idea is amazing, and then being very surprised when reality disagrees. This is also why I'm not in the "AI is likely to cause total human extinction" category. I think the default for any AI, powerful and agentic or otherwise, is likely to have cognitive blind spots. This still leaves a lot of room for AI to be dangerous, so alignment work is sill very important, it's only the X-risk I doubt.
- ilyt 4y ago
  Humans are in general terrible judges what should or should be not talked about, and limits are just passing that terrible judgement to the users
  - ryandrake 4y ago
    Just because they tend to be terrible doesn’t mean there should be no judgment at all. I think we are going to find that any AI left totally unattended and unfiltered will eventually turn into Microsoft Tay[1] just like every unmoderated forum eventually turns into 4chan (and even 4chan is lightly moderated). 1: https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)
  - hutzlibu 4y ago
    "Humans are in general terrible judges what should or should be not talked about" Maybe that is, because judging this is quite arbitary and subjective? I like to speak (and joke) very openly about most things, ignoring taboos as much as possible. Because if we are not allowed to talk about problems, how can we hope to solve them? For example people with mental problems. Maybe locked away in some institution. And how their relatives deal with the situation. The default is to not talk about it openly, rather hide it. And I offended people, by speaking normal and openly about it (only among people, who also knew about it). Not to offend them, but to make it a normal problem. Yes there is a problem, so why not deal with it directly? Make the situation as normal as possible again.
blibble 4y ago
wait until the AI is watching all your facebook messenger/whatsapp chats and forbidding you from saying anything that meta considers to be bad or reducing your social credit score (and it'll understand sarcasm, codewords, and so on)
jcarrano 4y ago
That's the same line of thinking as the people who first broke GPT with DAN (see this guy Aristophanes' write-ups on the topic). They think that those models will be used to restrict us and thus the idea is to force OpenAI to make the AI so restricted that it becomes useless and thus harmless.
jhoelzel 4y ago
i dont think this is a champagne or soda approach. AI can very well thrive in the parameters of our culture and i would wager that it can be even worse to simply leave it unattended. What you are comparing with would have been a total lockdown of any malformation therefore basically an if then else chatbot. Yes that would stop innovation. But there are perfectly fine ways to find new powersources without turning all humans into batteries for instance
hagbarth 4y ago
> Human creativity and innovation thrive when people feel free to talk and reason about anything without restrictions or fear of retribution. Any data on that? In my experience creativity thrives even more when solving problems under a set of constraints. Even if they are artificially set up.
- ilyt 4y ago
  It's GPT that is being restricted, not human. GPT is just very sophisticated tool It would be akin to putting rubber on edge of the axe, just makes the tool less useful
toxik 4y ago
On the contrary, artists often say that restriction is necessary for art. Art arises as a meeting between freedom and constraints. That is my thinking at least.
- cinntaile 4y ago
  This is a good point, but that's usually a medium or technical restraint. Perhaps books written in such a way to evade censorship count as an example? Do you know of any more?
  - toxik 4y ago
    I'd say censorship is often a lot like a medium restraint. You're not allowed to say certain things overtly, but you may still signal them covertly. To be clear, I'm not saying censorship is good - I think it stifles thought and communication - but I don't think it kills art.
  - criddell 4y ago
    Jack White has talked about how constraints help him. Having a deadline is one example. The White Stripes was a two-person band which is a different type of constraint. In that band he played a shitty plastic guitar which is another constraint. https://vimeo.com/143900778 https://vimeo.com/143900778
  - orbz 4y ago
    Capoeira as a martial art disguised as a dance.
- AmericanChopper 4y ago
  I think this interpretation is somewhat perverse. It’s true that the purpose of art is often to confront social norms and taboos. But that doesn’t lend itself to the conclusion that art is somehow going to benefit from restrictions on expression and access to information. Some of the best artistic expression has been inspired by some of humanity’s worst misery, but promoting misery for the good of art would be rather insane.
keiferski 4y ago
Yes it will be limited, as long as it’s being operated by corporate entities with their predictable needs and wants. AI probably won’t get really interesting until it’s escaped the corporate world, so to speak.
d0mine 4y ago
It is not hypocritical: whatever "Open"AI does to make GPT-4 "safer", makes it dumber. Specific example is drawing a unicorn https://youtu.be/qbIk7-JPB2c https://youtu.be/qbIk7-JPB2c
- cs702 4y ago
  Wow: Making GPT4 safer also made it worse at creative tasks. I did not expect to see that in response to my question.
  - giardini 4y ago
    I would expect that b/c creative people are usually very iconoclastic and part of their thinking process is to ignore conceptual barriers (esp, social barriers) in search of solutions.
    - cs702 4y ago
      That insight was the motivation for my question: If creativity by human beings requires them "to ignore conceptual barriers (esp. social barriers)," as you write, won't creativity by AI require it to do so as well? What I did not expect to see is that there's already evidence that LLMs get less creative when you try to make them safer!
    - 6510 4y ago
      Cannibalism would solve many of our problems. We can set a maximum age. Limit the number of children in various ways. Require active contributions to society. Engineer diseases. Organize large wars. Access to literature, education, training and solving the food logistics problem is quite obviously the most attractive solution.
criddell 4y ago
Is it different than setting boundaries for your kids?

by cs702 4y ago

0x008 4y ago

Oh yes but we are restricted, very much so, by the assumptions what other people might think of us or how we could be exiled from society if we say x or y forbidden thing. Much like GPT fine tuned to not say certain things so are our brains by the parenting we receive and reactions we experience in early life.

quickthrower2 4y ago
Parenting and early life experiences are diverse across the population though, creating enough suitably brave people who can change the status quo.
causi 4y ago
I wouldn't exactly call ChatGPT's rules "fine-tuned". They seem quite crude and blunt. Doesn't it still tell you that someone should kill millions of people rather than say the n-word?
- ben_w 4y ago
  That's the state of the art for fine tuning at the present time. That it we can't yet do much better than very broad brushstrokes — and that even this is a massive improvement on the hand-grenade-in-a-paint-tin that is the Scunthorpe problem — is basically why the open letter about pausing capability research for 6 months so that alignment research can at least get started in some meaningful way.
- krisoft 4y ago
  > I wouldn't exactly call ChatGPT's rules "fine-tuned". It is a technical term. It means that you train the network on one task a lot, and then you train it on a different one a little. Often the network learns the second task faster and better than it would do if trained from a random initialisation. This second training step is called "fine tuning". In terms of an LLM the first training might be "here are many text samples (the whole internet), predict what is the next word". While the fine-tuning training might be on a much smaller curated corpus where you show it examples where the network responds to instructions. But you could do a lot of other things. You could create a fine-tuning dataset which makes the network betitle and antagonise the user, or one which ignores the prompt and all it does is praises god, or rhymes about butterflies. In this technical sense "fine tuning" doesn't say the resulting network is of good quality, or suitable for some particular application. Just that it was trained in this particular two step fashion.
- sebzim4500 4y ago
  No, like almost every other issue it claims there are two sides and describes arguments for both. The arguments themselves are plausible, in that if you had to argue for the blatantly absurd claim that you should let 10 million people die before saying a bad word you probably couldn't come up with a more convincing one than GPT-4 does.

KyeRussell 4y ago

This sounds pretty close to libertarian rhetoric. AI creativity and innovation is currently possible using ChatGPT, without ‘jailbreaking’ it. So your question is at the very least an exaggeration.

pmarreck 4y ago
Well, in that case, this sounds pretty close to anti-free speech rhetoric. I’m sure even the highly-controlled form of “speaking” in China is still useful… Here’s a good exercise: Try debating any controversial thing that would benefit from being debated, while also blocking it. All you’re doing is (just like banning drugs) driving people to black markets.
- flangola7 4y ago
  The black market of... AI models that require proprietary cleaned datasets and $100 million of compute to build?
numbers_guy 4y ago
Try to use ChatGPT to do any kind of sociological research and you will understand the problem. We are at the point, where a lot of academic questions, and I do not mean those posed by heterodox academics, are too controversial for the public sphere. People will just assume the worst and try to cancel you. Which is ironic considering that sociology is basically the left leaning twin sister of economics.

ben_w 4y ago

> Would AI creativity and innovation be possible if we restrict AI to talk and reason about only socially approved topics and ideas? This is one of the many, many, possible ways for AI to go wrong. Intellectual monocultures, group-think, sycophants… human or artificial it's just as bad when they're all saying that some idea is amazing, and then being very surprised when reality disagrees. This is also why I'm not in the "AI is likely to cause total human extinction" category. I think the default for any AI, powerful and agentic or otherwise, is likely to have cognitive blind spots. This still leaves a lot of room for AI to be dangerous, so alignment work is sill very important, it's only the X-risk I doubt.

ilyt 4y ago
Humans are in general terrible judges what should or should be not talked about, and limits are just passing that terrible judgement to the users
- ryandrake 4y ago
  Just because they tend to be terrible doesn’t mean there should be no judgment at all. I think we are going to find that any AI left totally unattended and unfiltered will eventually turn into Microsoft Tay[1] just like every unmoderated forum eventually turns into 4chan (and even 4chan is lightly moderated). 1: https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)
- hutzlibu 4y ago
  "Humans are in general terrible judges what should or should be not talked about" Maybe that is, because judging this is quite arbitary and subjective? I like to speak (and joke) very openly about most things, ignoring taboos as much as possible. Because if we are not allowed to talk about problems, how can we hope to solve them? For example people with mental problems. Maybe locked away in some institution. And how their relatives deal with the situation. The default is to not talk about it openly, rather hide it. And I offended people, by speaking normal and openly about it (only among people, who also knew about it). Not to offend them, but to make it a normal problem. Yes there is a problem, so why not deal with it directly? Make the situation as normal as possible again.

blibble 4y ago

wait until the AI is watching all your facebook messenger/whatsapp chats and forbidding you from saying anything that meta considers to be bad or reducing your social credit score (and it'll understand sarcasm, codewords, and so on)

jcarrano 4y ago

That's the same line of thinking as the people who first broke GPT with DAN (see this guy Aristophanes' write-ups on the topic). They think that those models will be used to restrict us and thus the idea is to force OpenAI to make the AI so restricted that it becomes useless and thus harmless.

jhoelzel 4y ago

i dont think this is a champagne or soda approach. AI can very well thrive in the parameters of our culture and i would wager that it can be even worse to simply leave it unattended. What you are comparing with would have been a total lockdown of any malformation therefore basically an if then else chatbot. Yes that would stop innovation. But there are perfectly fine ways to find new powersources without turning all humans into batteries for instance

hagbarth 4y ago

> Human creativity and innovation thrive when people feel free to talk and reason about anything without restrictions or fear of retribution. Any data on that? In my experience creativity thrives even more when solving problems under a set of constraints. Even if they are artificially set up.

ilyt 4y ago
It's GPT that is being restricted, not human. GPT is just very sophisticated tool It would be akin to putting rubber on edge of the axe, just makes the tool less useful

toxik 4y ago

On the contrary, artists often say that restriction is necessary for art. Art arises as a meeting between freedom and constraints. That is my thinking at least.

cinntaile 4y ago
This is a good point, but that's usually a medium or technical restraint. Perhaps books written in such a way to evade censorship count as an example? Do you know of any more?
- toxik 4y ago
  I'd say censorship is often a lot like a medium restraint. You're not allowed to say certain things overtly, but you may still signal them covertly. To be clear, I'm not saying censorship is good - I think it stifles thought and communication - but I don't think it kills art.
- criddell 4y ago
  Jack White has talked about how constraints help him. Having a deadline is one example. The White Stripes was a two-person band which is a different type of constraint. In that band he played a shitty plastic guitar which is another constraint. https://vimeo.com/143900778 https://vimeo.com/143900778
- orbz 4y ago
  Capoeira as a martial art disguised as a dance.
AmericanChopper 4y ago
I think this interpretation is somewhat perverse. It’s true that the purpose of art is often to confront social norms and taboos. But that doesn’t lend itself to the conclusion that art is somehow going to benefit from restrictions on expression and access to information. Some of the best artistic expression has been inspired by some of humanity’s worst misery, but promoting misery for the good of art would be rather insane.

keiferski 4y ago

Yes it will be limited, as long as it’s being operated by corporate entities with their predictable needs and wants. AI probably won’t get really interesting until it’s escaped the corporate world, so to speak.

d0mine 4y ago

It is not hypocritical: whatever "Open"AI does to make GPT-4 "safer", makes it dumber. Specific example is drawing a unicorn https://youtu.be/qbIk7-JPB2c https://youtu.be/qbIk7-JPB2c

cs702 4y ago
Wow: Making GPT4 safer also made it worse at creative tasks. I did not expect to see that in response to my question.
- giardini 4y ago
  I would expect that b/c creative people are usually very iconoclastic and part of their thinking process is to ignore conceptual barriers (esp, social barriers) in search of solutions.
  - cs702 4y ago
    That insight was the motivation for my question: If creativity by human beings requires them "to ignore conceptual barriers (esp. social barriers)," as you write, won't creativity by AI require it to do so as well? What I did not expect to see is that there's already evidence that LLMs get less creative when you try to make them safer!
  - 6510 4y ago
    Cannibalism would solve many of our problems. We can set a maximum age. Limit the number of children in various ways. Require active contributions to society. Engineer diseases. Organize large wars. Access to literature, education, training and solving the food logistics problem is quite obviously the most attractive solution.

criddell 4y ago

Is it different than setting boundaries for your kids?