4 ms·

This current moment is super impressive, but the output of the models is still mediocre. I worry that over the next 12-24 months, we'll be overwhelmed with auto

This current moment is super impressive, but the output of the models is still mediocre. I worry that over the next 12-24 months, we'll be overwhelmed with autogenerated content (docs, apps, games, news) that gets a B- or C. Once the novelty wears off, no one wants to play your [terrible] autogenerated asteroid clone. There are a lot of startups that are two LLMs in a trench coat, and they will not be able to improve faster than people's expectations increase.

SketchySeaBeast 4y ago
Your BAD autogenerated asteroid clone. "Give it time" people will say, but how does one prompt ChatGPT to make a game fun, or a story actually interesting?
- turnsout 4y ago
  ^ Yes, edited my post! I've spent HOURS tweaking prompts and going back and forth with GPT3–4 to get more creative and interesting output, but it's not there yet. This current generation will just never give you legitimately good song lyrics, let alone a game with an interesting and compelling mechanic.
  - redeux 4y ago
    Having played around with the same scenario, I’ve come to the naive conclusion that the lack of creative output is because the current generation of GPT is only able to regurgitate and not create. This makes everything feel very derivative. On the other hand, if you act as the creative mind and use GPT to accelerate your output, it becomes more useful.
    - SketchySeaBeast 4y ago
      Am I off base, or is "create" sort of amorphous in this context? It's already creating novel sentences and paragraphs. What is difference between that and real creation? Personally it seems to me that if you feed a statistical model of a ton of input it's going to feed back the average of that input, leading to uninspiring phrases. Regarding game creation, we'd need to tell it about game mechanics, and I have no idea how it'd determine what's interesting or how it would innovate.
      - turnsout 4y ago
        I think that's it—these models are definitely capable of creating output not seen in the input, which we would define as human "creativity," and they can even combine concepts in novel ways, which is loosely what we call "innovation." But they are quite biased to create plausible output, which severely restricts their ability to create something interesting. You can dial up the temperature, but this quickly pushes the output from "bland" to "totally unhinged."
  - Joeri 4y ago
    Creativity is making unexpected connections, applying a twist that doesn’t logically follow from what came before. This is the opposite of what these models are designed to do, which is to give the most likely next word. On the other hand, raising the temperature should allow unlikely sequences to be output, provided the model is trained on enough unlikely sequences as input. Also, most creative work does not require much creativity, so just a little human guidance may go a long way.
- wongarsu 4y ago
  There are a lot of bad hobby games out there already. Telling human programmers "make this fun" doesn't reliably work, because being able to design good games is a skill with very little overlap with programming. If anything it has overlap with UX design, and we all know the stereotypes about programmer-built UIs. It's certainly possible to learn both skills, but taking away the need to be good at programming won't decrease the average quality of games made.
  - SketchySeaBeast 4y ago
    I'm not saying humans consistently make good games, but we do know that humans can make good games. The question is can ChatGPT. If it's supposed to reach a the ranks of a general intelligence, it should be able to. I'm not sure how it can.
    - flangola7 4y ago
      Ask it to list verbosely step by step essential elements of what makes a game fun, then iterate upon each one of those.
aezart 4y ago
This is my worry as well. Google results are already full of content farm/AI SEO junk, and I don't understand people who see that and think "let's do that, but more". There's already more good content than I could possibly consume in my lifetime, and all this automatically generated stuff is just going to make it harder to find.
taneq 4y ago
> they will not be able to improve faster than people's expectations increase. I've never read a better explanation for why "we will never have AGI." As the objective quality of AI output increases, we simply value it less.
- rchaud 4y ago
  Quality is objective, value is not. Can Midjourney copy DeviantArt/ArtStation to levels far beyond mine? Yes. Does the market value DeviantArt/Artstation output to any level? No.
rchaud 4y ago
I agree. The use cases are still far too limited to be of use at the scale beyond Canva or something. They will pay to automate the creation of social media graphics and emails, because nobody wants to work on those. But then what?

SketchySeaBeast 4y ago

Your BAD autogenerated asteroid clone. "Give it time" people will say, but how does one prompt ChatGPT to make a game fun, or a story actually interesting?

turnsout 4y ago
^ Yes, edited my post! I've spent HOURS tweaking prompts and going back and forth with GPT3–4 to get more creative and interesting output, but it's not there yet. This current generation will just never give you legitimately good song lyrics, let alone a game with an interesting and compelling mechanic.
- redeux 4y ago
  Having played around with the same scenario, I’ve come to the naive conclusion that the lack of creative output is because the current generation of GPT is only able to regurgitate and not create. This makes everything feel very derivative. On the other hand, if you act as the creative mind and use GPT to accelerate your output, it becomes more useful.
  - SketchySeaBeast 4y ago
    Am I off base, or is "create" sort of amorphous in this context? It's already creating novel sentences and paragraphs. What is difference between that and real creation? Personally it seems to me that if you feed a statistical model of a ton of input it's going to feed back the average of that input, leading to uninspiring phrases. Regarding game creation, we'd need to tell it about game mechanics, and I have no idea how it'd determine what's interesting or how it would innovate.
    - turnsout 4y ago
      I think that's it—these models are definitely capable of creating output not seen in the input, which we would define as human "creativity," and they can even combine concepts in novel ways, which is loosely what we call "innovation." But they are quite biased to create plausible output, which severely restricts their ability to create something interesting. You can dial up the temperature, but this quickly pushes the output from "bland" to "totally unhinged."
- Joeri 4y ago
  Creativity is making unexpected connections, applying a twist that doesn’t logically follow from what came before. This is the opposite of what these models are designed to do, which is to give the most likely next word. On the other hand, raising the temperature should allow unlikely sequences to be output, provided the model is trained on enough unlikely sequences as input. Also, most creative work does not require much creativity, so just a little human guidance may go a long way.
wongarsu 4y ago
There are a lot of bad hobby games out there already. Telling human programmers "make this fun" doesn't reliably work, because being able to design good games is a skill with very little overlap with programming. If anything it has overlap with UX design, and we all know the stereotypes about programmer-built UIs. It's certainly possible to learn both skills, but taking away the need to be good at programming won't decrease the average quality of games made.
- SketchySeaBeast 4y ago
  I'm not saying humans consistently make good games, but we do know that humans can make good games. The question is can ChatGPT. If it's supposed to reach a the ranks of a general intelligence, it should be able to. I'm not sure how it can.
  - flangola7 4y ago
    Ask it to list verbosely step by step essential elements of what makes a game fun, then iterate upon each one of those.

aezart 4y ago

This is my worry as well. Google results are already full of content farm/AI SEO junk, and I don't understand people who see that and think "let's do that, but more". There's already more good content than I could possibly consume in my lifetime, and all this automatically generated stuff is just going to make it harder to find.

taneq 4y ago

> they will not be able to improve faster than people's expectations increase. I've never read a better explanation for why "we will never have AGI." As the objective quality of AI output increases, we simply value it less.

rchaud 4y ago
Quality is objective, value is not. Can Midjourney copy DeviantArt/ArtStation to levels far beyond mine? Yes. Does the market value DeviantArt/Artstation output to any level? No.

rchaud 4y ago

I agree. The use cases are still far too limited to be of use at the scale beyond Canva or something. They will pay to automate the creation of social media graphics and emails, because nobody wants to work on those. But then what?