5 ms·

The hard part of world building is not contradicting yourself too much. Right now, "remember all the little details and their implications" seems like it would

The hard part of world building is not contradicting yourself too much. Right now, "remember all the little details and their implications" seems like it would be challenging for language models.

Taek 4y ago
I've had success using a multistage inference approach. Establish a set of rules, then have the LLM ask itself whether its current suggested world building complies with all the rules. Then have it tweak its own story. Finally, after all that, present the output to the user. It means inference takes 4x as long because you are doing multiple rounds of inference before actually presenting output but the final output you get is way more stable/consistent/accurate
- williamcotton 4y ago
  That seems to be similar to this approach, at least at a cursory level: Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback https://arxiv.org/abs/2302.12813 https://arxiv.org/abs/2302.12813 The “facts” being derived not from Wikipedia but rather the rules of the fictional world.
- sitkack 4y ago
  I have had it make new languages based off existing languages but change the weights on how thing are used and what semantics are allowed or fixed. Are you building in a recursive layer by asking it to take on a role while also targeting its output at a constructed user persona?
  - Taek 4y ago
    I hadn't thought about targeting a constructed user persona, that's a neat idea. But otherwise yes, its a recursive model that runs inference a bunch of times to improve the quality of its output.
CatWChainsaw 4y ago
The real world is full of contradictions. If anything they would make your built world more lifelike.
- actionfromafar 4y ago
  Yeah, just this morning, the coffee in my cup transformed to orange juice while I looked away.
  - wongarsu 4y ago
    So either a) you live in a world full of magic (the mysterious, unpredictable kind) or b) you have a small child at home But there is a challenge in building fiction that even things that can reasonably happen will often be rejected by the reader if they are strange and unexplained. The saying "truth is stranger than fiction" actually tells you more about fiction than about truth.
  - Dudeman112 4y ago
    I mean, George Lucas transformed his magic into counting magical mitochondria while people looked away and that universe is still as successful as ever Popular fiction doesn't give a shit about contradicting itself
    - discreteevent 4y ago
      The root of this thread is a comment about "not contradicting yourself too much" yet there are a couple of comments within the thread that seem to imply that because we can find some examples of contradiction in fiction then uncontrolled/unlimited contradiction should be fine. This "I can think of a counterexample - let's make it a rule" seems to appear more frequently now but maybe that's just me.
      - Dudeman112 4y ago
        >in fiction then uncontrolled/unlimited contradiction should be fine I'm not saying it's fine in general. I'm saying it doesn't matter for pop fiction Popular fiction gets retconned all the time. I'm fundamentally disagreeing with "not contradicting yourself too much" mattering at all for fiction that's successful with the masses If you ever feel tempted to think that quality writing matters for popular success, just remember that lewd Twilight fanfic has sold more than a hundred million copies
localplume 4y ago
eh I think its not as black-or-white. The Elder Scrolls lore is inconsistent but that is chalked up to unreliable narrators. Used in the right amount, I think slight contradiction in worldbuilding is good, and it also reduces the burden on the worldbuilder for every tiny detail to be factually consistent.
ianbicking 4y ago
The first level of consistency is: consistent with the world around us. People have two legs and two arms. You eat off plates. GPT is good at this. But if you want to build Ant World it'll be hard. The second level of consistency is: consistent with a stereotype. So "standard fantasy world" is going to be consistent with fantasy tropes. (Mostly, except with a strong tourist industry and the occasional café; the modern world leaks in.) The third level of consistency is: consistent with what you explicitly state about the world. That's the city backstory. If you want magic to come from the sun and you can catch magic in little sun basins and then use that to power your dishwasher, then you can state that... but it may have a hard time remembering that, and a harder time extrapolating all the effects. But GPT won't entirely forget it, and may even come up with some interesting ideas. The real challenge though is: consistency with the author's as-yet-unfulfilled desires. That's where an interactive tool that allows co-creation with the AI might offer something, but it WILL be a struggle because the author must develop, articulate, refine, and reform their ideas, and the AI may help or resist, but often will simply make it clear the gaps that exist between desire and vision.

The hard part of world building is not contradicting yourself too much. Right now, "remember all the little details and their implications" seems like it would be challenging for language models.

Taek 4y ago

I've had success using a multistage inference approach. Establish a set of rules, then have the LLM ask itself whether its current suggested world building complies with all the rules. Then have it tweak its own story. Finally, after all that, present the output to the user. It means inference takes 4x as long because you are doing multiple rounds of inference before actually presenting output but the final output you get is way more stable/consistent/accurate

williamcotton 4y ago
That seems to be similar to this approach, at least at a cursory level: Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback https://arxiv.org/abs/2302.12813 https://arxiv.org/abs/2302.12813 The “facts” being derived not from Wikipedia but rather the rules of the fictional world.
sitkack 4y ago
I have had it make new languages based off existing languages but change the weights on how thing are used and what semantics are allowed or fixed. Are you building in a recursive layer by asking it to take on a role while also targeting its output at a constructed user persona?
- Taek 4y ago
  I hadn't thought about targeting a constructed user persona, that's a neat idea. But otherwise yes, its a recursive model that runs inference a bunch of times to improve the quality of its output.

CatWChainsaw 4y ago

The real world is full of contradictions. If anything they would make your built world more lifelike.

actionfromafar 4y ago
Yeah, just this morning, the coffee in my cup transformed to orange juice while I looked away.
- wongarsu 4y ago
  So either a) you live in a world full of magic (the mysterious, unpredictable kind) or b) you have a small child at home But there is a challenge in building fiction that even things that can reasonably happen will often be rejected by the reader if they are strange and unexplained. The saying "truth is stranger than fiction" actually tells you more about fiction than about truth.
- Dudeman112 4y ago
  I mean, George Lucas transformed his magic into counting magical mitochondria while people looked away and that universe is still as successful as ever Popular fiction doesn't give a shit about contradicting itself
  - discreteevent 4y ago
    The root of this thread is a comment about "not contradicting yourself too much" yet there are a couple of comments within the thread that seem to imply that because we can find some examples of contradiction in fiction then uncontrolled/unlimited contradiction should be fine. This "I can think of a counterexample - let's make it a rule" seems to appear more frequently now but maybe that's just me.
    - Dudeman112 4y ago
      >in fiction then uncontrolled/unlimited contradiction should be fine I'm not saying it's fine in general. I'm saying it doesn't matter for pop fiction Popular fiction gets retconned all the time. I'm fundamentally disagreeing with "not contradicting yourself too much" mattering at all for fiction that's successful with the masses If you ever feel tempted to think that quality writing matters for popular success, just remember that lewd Twilight fanfic has sold more than a hundred million copies

localplume 4y ago

eh I think its not as black-or-white. The Elder Scrolls lore is inconsistent but that is chalked up to unreliable narrators. Used in the right amount, I think slight contradiction in worldbuilding is good, and it also reduces the burden on the worldbuilder for every tiny detail to be factually consistent.

ianbicking 4y ago

The first level of consistency is: consistent with the world around us. People have two legs and two arms. You eat off plates. GPT is good at this. But if you want to build Ant World it'll be hard. The second level of consistency is: consistent with a stereotype. So "standard fantasy world" is going to be consistent with fantasy tropes. (Mostly, except with a strong tourist industry and the occasional café; the modern world leaks in.) The third level of consistency is: consistent with what you explicitly state about the world. That's the city backstory. If you want magic to come from the sun and you can catch magic in little sun basins and then use that to power your dishwasher, then you can state that... but it may have a hard time remembering that, and a harder time extrapolating all the effects. But GPT won't entirely forget it, and may even come up with some interesting ideas. The real challenge though is: consistency with the author's as-yet-unfulfilled desires. That's where an interactive tool that allows co-creation with the AI might offer something, but it WILL be a struggle because the author must develop, articulate, refine, and reform their ideas, and the AI may help or resist, but often will simply make it clear the gaps that exist between desire and vision.