4 ms·

Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves? Last week I tried to get an LLM (one of the re

Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves? Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. When I told it to stop doing cyberpunk scifi stuff it went completely to wild west.

o11c 2y ago
You should not ever expect an LLM to actually do what you want without handholding, and randomness in particular is one of the places it fails badly. This is probably fundamental. That said, this is also not helped by the fact that all of the default interfaces lack many essential features, so you have to build the interface yourself. Neither "clear the context on every attempt" nor "reuse the context repeatedly" will give good results, but having one context producing just one-line summaries, then fresh contexts expanding each one will do slightly less badly. (If you actually want the LLM to do something useful, there are many more things that need to be added beyond this)
- dotancohen 2y ago
  Sounds to me like you might want to reduce the Top P - that will prevent the really unlikely next tokens from ever being selected, while still providing nice randomness in the remaining next tokens so you continue to get diverse stories.
janalsncm 2y ago
Generate a list of 5000 possible topics you’d like it to talk about. Randomly pick one and inject that into your prompt.
coder543 2y ago
Someone mentioned generating millions of (very short) stories with an LLM a few weeks ago: https://news.ycombinator.com/item?id=42577644 https://news.ycombinator.com/item?id=42577644 They linked to an interactive explorer that nicely shows the diversity of the dataset, and the HF repo links to the GitHub repo that has the code that generated the stories: https://github.com/lennart-finke/simple_stories_generate https://github.com/lennart-finke/simple_stories_generate So, it seems there are ways to get varied stories.
- fi-le 2y ago
  I was wondering where the traffic came from, thanks for mentioning it!
TMWNN 2y ago
> Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves? > Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. 100% relevant: "Someday" <https://en.wikipedia.org/wiki/Someday_(short_story) https://en.wikipedia.org/wiki/Someday_(short_story)> by Isaac Asimov, 1956
greenavocado 2y ago
Set temperature to 1.0
behohippy 2y ago
It's a 3b model so the creativity is pretty limited. What helped for me was prompting for specific stories in specific styles. I have a python script that randomizes the prompt and the writing style, including asking for specific author styles.
jaggs 2y ago
https://old.reddit.com/r/LocalLLaMA/comments/1i615u1/the_first_time_ive_felt_a_llm_wrote_well_not_just/m88c6el/ https://old.reddit.com/r/LocalLLaMA/comments/1i615u1/the_fir...

o11c 2y ago

You should not ever expect an LLM to actually do what you want without handholding, and randomness in particular is one of the places it fails badly. This is probably fundamental. That said, this is also not helped by the fact that all of the default interfaces lack many essential features, so you have to build the interface yourself. Neither "clear the context on every attempt" nor "reuse the context repeatedly" will give good results, but having one context producing just one-line summaries, then fresh contexts expanding each one will do slightly less badly. (If you actually want the LLM to do something useful, there are many more things that need to be added beyond this)

dotancohen 2y ago
Sounds to me like you might want to reduce the Top P - that will prevent the really unlikely next tokens from ever being selected, while still providing nice randomness in the remaining next tokens so you continue to get diverse stories.

janalsncm 2y ago

Generate a list of 5000 possible topics you’d like it to talk about. Randomly pick one and inject that into your prompt.

coder543 2y ago

Someone mentioned generating millions of (very short) stories with an LLM a few weeks ago: https://news.ycombinator.com/item?id=42577644 https://news.ycombinator.com/item?id=42577644 They linked to an interactive explorer that nicely shows the diversity of the dataset, and the HF repo links to the GitHub repo that has the code that generated the stories: https://github.com/lennart-finke/simple_stories_generate https://github.com/lennart-finke/simple_stories_generate So, it seems there are ways to get varied stories.

fi-le 2y ago
I was wondering where the traffic came from, thanks for mentioning it!

TMWNN 2y ago

> Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves? > Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. 100% relevant: "Someday" <https://en.wikipedia.org/wiki/Someday_(short_story) https://en.wikipedia.org/wiki/Someday_(short_story)> by Isaac Asimov, 1956

greenavocado 2y ago

Set temperature to 1.0

behohippy 2y ago

It's a 3b model so the creativity is pretty limited. What helped for me was prompting for specific stories in specific styles. I have a python script that randomizes the prompt and the writing style, including asking for specific author styles.

jaggs 2y ago

https://old.reddit.com/r/LocalLLaMA/comments/1i615u1/the_first_time_ive_felt_a_llm_wrote_well_not_just/m88c6el/ https://old.reddit.com/r/LocalLLaMA/comments/1i615u1/the_fir...