4 ms·

Actually, all you need to do is to apply structured randomness to get diversity from a LLM. For example in TinyStories paper, a precursor of the Phi models: >

Actually, all you need to do is to apply structured randomness to get diversity from a LLM. For example in TinyStories paper, a precursor of the Phi models: > We collected a vocabulary consisting of about 1500 basic words, which try to mimic the vocabulary of a typical 3-4 year-old child, separated into nouns, verbs, and adjectives. In each generation, 3 words are chosen randomly (one verb, one noun, and one adjective). The model is instructed to generate a story that somehow combines these random words into the story You can do the same for generating worlds, just prepare good ingredients and sample at random.

miltonlost 2y ago
A story is not just words crammed together that sound plausible. Is the AI going to know about pacing? About character motivations? About interconnecting disparate plots? That paper sounds like it has a scientist’s conception that a story is just words, and not complex trade offs between the start of a story and its end and middle, complexity and planning that won’t come from any sort of next-token generation. These are “stories” in the most vacuous definition possible, one that is just “and then this happened” like a child’s conception of plot
- wewtyflakes 2y ago
  > Is the AI going to know about pacing? About character motivations? About interconnecting disparate plots? For LLMs like GPT-4, this all seems reasonable to account for and assume the LLM is capable of processing, given appropriate guidance/frameworks (of which may be just classical programming).
- Philpax 2y ago
  > Is the AI going to know about pacing? About character motivations? About interconnecting disparate plots? Yes. This is an active research area. See https://github.com/yingpengma/Awesome-Story-Generation https://github.com/yingpengma/Awesome-Story-Generation, which is not up to date.

miltonlost 2y ago

A story is not just words crammed together that sound plausible. Is the AI going to know about pacing? About character motivations? About interconnecting disparate plots? That paper sounds like it has a scientist’s conception that a story is just words, and not complex trade offs between the start of a story and its end and middle, complexity and planning that won’t come from any sort of next-token generation. These are “stories” in the most vacuous definition possible, one that is just “and then this happened” like a child’s conception of plot

wewtyflakes 2y ago
> Is the AI going to know about pacing? About character motivations? About interconnecting disparate plots? For LLMs like GPT-4, this all seems reasonable to account for and assume the LLM is capable of processing, given appropriate guidance/frameworks (of which may be just classical programming).
Philpax 2y ago
> Is the AI going to know about pacing? About character motivations? About interconnecting disparate plots? Yes. This is an active research area. See https://github.com/yingpengma/Awesome-Story-Generation https://github.com/yingpengma/Awesome-Story-Generation, which is not up to date.