8 ms·

LLMs are not suitable for brainstorming

68 points by bcstyle 2y ago

p1esk 2y ago
GPT4 is great for brainstorming. It helped me come up with an idea for my last paper.
- gavmor 2y ago
  The author might accuse you of merely "better-than-average level" thinking
ashiban 2y ago
Secret prompt - add 'using TRIZ methodology' to your brainstorming prompts
- electrondood 2y ago
  Never heard of TRIZ before and am now falling down a great rabbit hole. Thanks!
- little_name 2y ago
  Just googled, TIL. It's very much based on the idea of replicating patterns though, so I think it doesn't contradict with the original post. Was thinking if you can apply TRIZ method to invent Transformer before 2017 - hard to imagine it working on me
christina97 2y ago
I mean they are. I’ve had great success brainstorming various things with them. People use LLMs for brainstorming every day.
deleted 2y ago
[deleted]
jasonjmcghee 2y ago
Idk. In the vast majority of topics that intellectually stimulate me, I'm far below the average specialist. That means LLMs have a lot to offer me.
- afro88 2y ago
  Came here to say this. The article assumes that you're already an expert on what you're asking the LLM to help you brainstorm for. A better title would be "LLMs are not suitable for brainstorming in topics you know deeply" I'll also add that there's a big difference between "give me 10 startup ideas" and having a conversation with it to explore different startup ideas (for example). Through conversation you can build off what each other say and explore the space.
schmidt_fifty 2y ago
> LLMs won’t behave in a more creative and independent way (as we hoped), but more susceptible to issues like hallucination. What is creativity if not what people call hallucination? Horribly named phenomenon, btw.
- kedean 2y ago
  I tried asking an LLM for assistance with Chef Policyfiles today (more specific than that of course), and the response was as if it was for a non-existent product rather than Chef. It even included a code block that it claimed I could run on repl.it if I had an account, which resembled a resource block in chef (policies are not a kind of resource). If a coworker confidently gave me code that is not only wrong, but relies on things that don't exist, I wouldn't call it creative. If they tried to claim that no I'm wrong, it is real, I might even say they hallucinated it.
  - schmidt_fifty 2y ago
    [dead]
resource_waste 2y ago
>However, here I would like to argue that (especially in cutting edge scenarios) LLMs are not a good tool to do truly effective brainstorming. Great title, they baited and switched. 95% of the time I don't need effective brainstorming, I need a bunch of ideas, let me pick the best, and move on. If its a real engineering problem, then I need >truly effective brainstorming.
- defrost 2y ago
  I recently spoke with a cousin about their current project decommissioning an oil and gas platform over the next four years (ie. real engineering (in reverse?)). Their comment was that ChatGPT can't write final reports but it's extremely useful for seeding brainstorming sessions with real human engineers. It can generate a list of health, safety, and environmental concerns related to the project that can be distributed and act as a "starter dot point list" that engineers can agree to, expand on, reject as being just silly, and add to as tengential thoughts arise. I'd argue that makes it useful in the context of real engineering brainstorming .. and that it's being used that way on multi million dollar projects in a billion dollar domain.
  - behringer 2y ago
    Yeah for sure, lists of generalized questions is one of ChatGPTs strongest abilities, in my experience.
- joegibbs 2y ago
  There's a lot of this with current AI - someone will write a piece titled something like "LLMs are not suitable for brainstorming" or "GPT4 is useless for programmers" or "AI image generation has no use cases" - and what they say in the body is that they're not perfect for brainstorming, GPT4 can't write an entire app perfectly from one prompt, and Midjourney doesn't let you take a picture from your mind's eye and put it on paper.
- firewolf34 2y ago
  Seriously, it's the same argument that people give for "ChatGPT can't give me good code, I don't know why", just rephrased. The deluge of "GPT is not useful for X" articles meant to bait the average critic, despite it being used en masse for "subproblems in X-space" already... They're asking the wrong type of work from it. If you need some boilerplate or a transformation, it's going to give you a fantastic template to work with. If you need it, on the other hand, to engineer out a highly-specific and nuanced solution with an esoteric codebase to a complex problem, maybe not so much. The former is wide, the latter is narrow. It's going to take maybe a bit more breakdown of the scope into proper subproblems before you'll get a good answer; and that's something you can do yourself, or have an agent perform across multiple queries maybe (though I'll admit, more work needs to be done for the whole multi-agent workflows to be truly useful).
Nevermark 2y ago
I find LLMs to be useful tools for brainstorming. I have often inadvertently found myself brainstorming just by explaining something challenging I am working on to a colleague with no expertise in my area. Just explaining forces new perspectives and new ideas. Similarly, LLMs are captive audiences for throwing out ideas and playing with them. The interaction creates something more than just talking to myself or a whiteboard. Not all brainstorming partners need to be first order contributors to have a very positive impact. Different kinds of interactions can spark entirely different kinds of ideas.
- jasonjmcghee 2y ago
  I agree with this, but also get so irritated with the endless praise. Like, tell me I'm wrong and tell me why. If I present a bad idea, tell me that. Or even like "huh? explain" Looking forward to ramping up the honesty parameter. Hoping OpenAI's new voice model trivializes this so I don't need to prompt engineer.
  - fassssst 2y ago
    Curious what happens if you set custom instructions to correct that. Does it work?
  - pr337h4m 2y ago
    A simple "Be brutally honest" at the end works surprisingly well.
  - vundercind 2y ago
    Me: “I’d like to start a business harvesting cat shit to fight global warming. People would pay me to take their cat shit and bury it because maybe it has carbon in it or something IDK I’m not a scientist. I would give them carbon credits in the form of a cat-shit-sequestration crypto token my business created. I’d pick up the cat shit by bicycle to save carbon emissions. Please help me develop a business plan.” Chargpt: “that’s a great idea for a business! Here are some suggestions…” Not a real exchange, but god, it does feel that way sometimes. One reason you can’t trust the damn thing is it’s too positive.
infinite-hugs 2y ago
I’ll add this here for the fake internet points but also as a general PSA. Most people are not aware that “Brainstorming” although popularized as a process since its origins in the 1960s… is actually Step 2. Most people are unaware that Step 1 is called “Questionstorming”. At the heart of the process is leveraging divergent and convergent modes of thinking which is done both to generate questions (and select the most promising ones) and answers… you’re welcome :D
- gavanwilhite 2y ago
  Fantastic context. I love hourglass shaped brainstorming, and the “questions” prompt is a solid one!
  - sitkack 2y ago
    With LLMs, having questions is more important than having answers. When you run out of questions, the exploration stops.
    - nathanasmith 2y ago
      I just keep prompting with "continue" when that happens.
      - sitkack 2y ago
        "continue" is now the most important word in the english language right after "No" right now. This causes the LLM to orbit around some point in the latent space, but doesn't cause it to explore. You have to tell it to pick a direction and there is no better direction than a question.
dahart 2y ago
> The reason [LLMs are not a good tool to do truly effective brainstorming] is LLMs are trained to follow existing patterns in the human-produced corpus, and not natively taught to “brainstorm”. The problem with this argument is that people do the same thing, we’re not that great at brainstorming either. When we brainstorm in groups, we’re just bringing multiple points of view together. The more data LLMs are trained on, the more viewpoints it might be able to bring that you haven’t considered. That said, LLMs and all NNs so far are built to interpolate, and they are bad and have unbounded error when extrapolating outside their training examples. That is a good reason to not expect today’s AI to come up with new ideas.
- trueismywork 2y ago
  We don't need more data, we need independent LLMs with slight randomness.
  - dahart 2y ago
    Aren’t independent LLMs just more data? Some AI researchers are arguing that more data is the primary thing that got us this far, and the primary thing that will improve AI from here. Here’s an example (and a very good talk whether you believe my summary) https://www.youtube.com/live/a13aqr07tJ4?si=FZO5m_XzrfpDhyQP https://www.youtube.com/live/a13aqr07tJ4?si=FZO5m_XzrfpDhyQP As a part-time generative artist for several decades, a user of Monte-Carlo methods on a daily basis, and author of some papers on the topic, I personally believe that randomness is not a good answer to anything creative. Randomness is boring, and average. Randomness only helps you when you have a high quality Markov model constraining what your RNG chooses between, and that’s more or less all LLMs actually are. Adding more randomness to creative works in general makes creative works muddy and lowers quality, it needs to be guided. Randomness is a useful tool, but is widely misunderstood IMO and not very effective at brainstorming style exploration; brainstorming is about solving problems in interesting ways, i.e., there is reasoning behind it, not slightly more random word salad.
- nequo 2y ago
  Setting the temperature to a higher value would emulate this, wouldn't it? Then the model would be more willing to deviate from the most likely next token.
nl 2y ago
This is utterly wrong. The strength of LLMs is that they have very wide knowledge, so while your specialist knowledge might surpass them in a particular area it will know more than you across other topics. When brainstorming this wide knowledge is what you want. The trick is (as always) better prompting to push it hard - so things like "consider parallels in similar situations in other fields" are useful.
platz 2y ago
not suitable = better-than-average?
WhitneyLand 2y ago
Brainstorming startup ideas? How often is that a successful approach at all? I can’t think of a good case study off the top of my head.
- little_name 2y ago
  I don't know a case study but I was personally told by 2 different founders who had 100MM+ exits that their startup ideas were formed (at least partially) from brainstorm sessions. One guy even pointed to me the library where they had the whiteboard session which led to the idea that they exited in the end. The SPC also have a blog that proposes brainstorming from -1 to 1: https://blog.southparkcommons.com/how-to-go-from-minus-1-to-0/ https://blog.southparkcommons.com/how-to-go-from-minus-1-to-...
labrador 2y ago
Counterpoint: LLMs are very suitable for brainstorming if prompted appropriately LLMs are going to give you the consensus reality (average opinion? average facts?) but you can easily steer it into offbeat, controversial and esoteric areas of it's training with the right prompts
- behringer 2y ago
  I spent a good long while trying to come up with novel video-game game play ideas, exactly what one would call "brainstorming" for ideas, and quite frankly it was pretty awful. ChatGPT more or less it converged on simply taking the current subject, adding a generic gameplay element to it and outputting it. It took half the ideas being thrown at it and included a rhythm mechanic... That's not that interesting, and I just couldn't get it to really think outside the box.
  - labrador 2y ago
    That's a good point I forgot to mention. It really depends on your subject matter. I put a clip up supporting why I think LLMs are good for brainstorming from Marc Andreeson about this topic in case your interested "Marc Andreessen says with the right prompting, you can unlock the latent super genius in AI models" https://www.youtube.com/watch?v=N2yN4IG8UYA https://www.youtube.com/watch?v=N2yN4IG8UYA And here's my comment on another site to a pro novelist who suddenly discovered Claude wrote like a genius who posits that Anthropic deliberatly cripples Claude's writing ability because they don't want to scare writers suddenly, they want to ease them into it I disagree. Your supernatural scenes triggered words an analysis from higher quality writers and commenters in the training data. If you were writing about bass fishing it would likely not impress you with it's writing. You can try something like this as an experiment. In other words it's good at some writing and bad at others, depending on the training data. I'd love to be proven wrong, like a gripping story about bass fishing might be interesting.
bawolff 2y ago
> What’s worse is when we ask topics that don’t have consensus currently, the LLMs won’t behave in a more creative and independent way (as we hoped), but more susceptible to issues like hallucination. I mean, what is the difference between creativity and hallucination (honest question)? --- Maybe this means AI would be better at the opposite - after you brainstorm creative out there ideas, AI can tell you if they have been tried in the past and what the consensus view is on why they failed, allowing you to adjust course.
- chefandy 2y ago
  > I mean, what is the difference between creativity and hallucination (honest question)? Well, creative problem solving involves getting new ideas and perspectives by creating novel associations between things we know. Hallucinations are creating things we "know" because they sound like they could be right and basing "ideas" on them. In short, hallucinations are bullshit. Totally open creative problem solving isn't always perfect, but even the ideas that don't really work can reveal something about the concepts involved. And bullshit often takes creativity to create, but that doesn't make it useful as actual creative output. It's not like the second you move beyond empirically provable statements, everything has equal merit. If you're trying to creatively work to a useful end, whether or not the knowledge you're working with is fabricated is pretty consequential. Doing otherwise would be like trying to optimize your code based on utterly fake but plausible algorithms-- sounding right isn't right enough to be useful. IMO, being able to create lies that pass the smell test is LLMs' most dangerous proclivity, especially when they're presented as expertise-in-a-box.
  - bawolff 2y ago
    Its not like humans always have good creative output. A lot of human creative output is essentially unworkable bullshit that gets discarded quite quickly. I'm not saying llms are good at creativity (i certainly don't think they are) but i kind of feel like its a difference in quality not kind. Like if you asked me to describe what it means for a human to be creative, i would probably write something quite similar to what you wrote above.
michael_nielsen 2y ago
There goes 50+% of my use. "LLMs are no good for [use case X]" often means "I aren't very good at using LLMs for [use case X]". With many powerful tools - violins, say, or carpentry tools - we know that it takes a long time and a lot of learning to achieve competent performance, much less virtuoso performance. Someone who spent ten hours learning the violin and concluded "Violins sound terrible" wouldn't have diagnosed a problem with violins, but with their own mastery. I certainly think current LLMs have some big intrinsic weaknesses, but also that what they are is quite subtle.
- llm_trw 2y ago
  I think it strongly depends on the model. I found that og gpt4 is better for brainstorming than gpt4 turbo, which in tern is better than gtp4o. If you're just using the web based portal you don't get much of a choice which model to use or it's temperature.
sweetheart 2y ago
I'm interpreting "brainstorming" to be any kind of general noodling on an issue to try to find new/novel/interesting solutions. In that case, I think every significant moment of true inspiration I had (of which there have been like, maybe 3 ever), they were always the result of seemingly random, completely unrelated things popping into my mind that, for whatever reason, clicked perfectly into the problem I was mulling over. To me, this means that manufacturing "true" inspiration doesn't require a tool that can deviate from "standard" human thinking patterns. I think it just means that you would want a tool that helps expose you to as many new and unknown fields/concepts/ideas in as little time as possible. So I think in that way LLMs are an amazing tool for helping one to brainstorm.
deleted 2y ago
[deleted]
locallyidle36 2y ago
Strongly disagree - brainstorming isn’t about asking someone to give you creative answers, it’s a team sport about triggering unique thoughts that neither person would have had on their own. Although I might be biased, I love using AI for brainstorming and started building a tool for it https://youtu.be/t5gfETbUzy0 https://youtu.be/t5gfETbUzy0
chiefalchemist 2y ago
Disagree. LLM might be mostly bad at it, but that doesn't make them unsuitable. They don't have a career to worry about. They don't care what peers or the boss thinks. Etc. They can spit out ideas - LOTS of them - that real humans then use as a starting point to carry on with. For more details see "Co-intelligence" by Ethan Mollick.
6510 2y ago
Wait, does that mean hallucinations should be embraced and improved uppon?
__loam 2y ago
LLMs seem to be unsuitable for a lot of work.
BOOSTERHIDROGEN 2y ago
I'm curious for someone like Ilya does he use LLM for daily brainstorming ?
- mirekrusin 2y ago
  Ilya's LLM is using Ilya for its own benefits.
bcstyle 2y ago
Author here. Was not expecting this quick post being picked up by HN - thanks for all the comments! I want to acknowledge that the original title is inaccurate, as many of you have pointed out. It should be "(current) LLMs don't brainstorm novel things really well" rather than just brainstorming. It's not intended to be a clickbait though - I was kind of mixing two definitions of brainstorming unintentionally. When we refer to the group activity that aims to collect all angles from participants (and common wisdoms), LLMs are really good, and it's something I do on a regular basis. However when it comes to the hope of reaching novel ideas that don't exist before (which some of us will consider what distinguishes brainstorming from group discussion or research study), I would say today's LLMs don't do well. I've seen such issue in business and arts domains, and also someone here mentioned similar experience in video game design. I would argue that (so far) for any idea LLMs tell us, there exists at least one instance of a similar pattern in the training data (either exact or in a high level). If this is what you need, then great. But some problems require more than that. And I would argue that a lot of important innovations in history didn't follow this pattern. I'm aware of reports on LLMs helping research (e.g. the works shared by Terry Tao), but I don't think they contradict the point here. Will be super happy to be proven wrong though!
- sitkack 2y ago
  > Was not expecting this quick post being picked up by HN - thanks for all the comments! You submitted the post?! It got “picked up” so quickly because it is wrong. I think what your post does show is how effectively you can farm HN for supporting arguments. You provide zero evidence for your claims, nor do you give us any transcripts of your attempts. Every post that matches the structure of yours is usually a summary of how the author has low skill in using LLMs. Though I am absolutely delighted in the comments and the nih paper referenced was a joy to read.
aoeusnth1 2y ago
I think effective brainstorming is actually a matter of prompt engineering, which is to say, breaking the model out of its box and into unexplored territory. See https://twitter.com/repligate https://twitter.com/repligate for some extreme examples of this.
- deleted 2y ago
  [deleted]
infogulch 2y ago
I've found LLMs to be great inspiration and very helpful for writing. My basic process is: 1. Ask agent to write a few paragraphs. 2. Notice that it's terrible and it would be easier to rewrite it myself than fix it. 3. Actually be motivated to write.
paulmd 2y ago
Sure they are. Just not infallible/perfect, but yes, they certainly are capable of mixing up inputs and coming up with something that’s not in the input set. No, they won’t come up with a totally new set of concepts and axioms that are totally unconnected to any human set… and you don’t want them to. Because those are the actual hallucinations. New ideas need to be novel, but they still need to fit into the rest of the set. It’s remarkable just how much people race to be pessimistic about LLMs, even when the pessimistic assertions are obviously and factually incorrect. Like yes we know LLMs have “hallucinations”, which is a loaded term to begin with, because the pessimists literally never shut up about it, even though it’s a fairly trivial overall problem given the immense overall utility. Some people are clearly so very very emotionally invested in there being nothing productive or useful about AI if it’s not AGI or if it needs the slightest amount of correction or oversight.

68 points by bcstyle 2y ago

p1esk 2y ago

GPT4 is great for brainstorming. It helped me come up with an idea for my last paper.

gavmor 2y ago
The author might accuse you of merely "better-than-average level" thinking

ashiban 2y ago

Secret prompt - add 'using TRIZ methodology' to your brainstorming prompts

electrondood 2y ago
Never heard of TRIZ before and am now falling down a great rabbit hole. Thanks!
little_name 2y ago
Just googled, TIL. It's very much based on the idea of replicating patterns though, so I think it doesn't contradict with the original post. Was thinking if you can apply TRIZ method to invent Transformer before 2017 - hard to imagine it working on me

christina97 2y ago

I mean they are. I’ve had great success brainstorming various things with them. People use LLMs for brainstorming every day.

deleted 2y ago

[deleted]

jasonjmcghee 2y ago

Idk. In the vast majority of topics that intellectually stimulate me, I'm far below the average specialist. That means LLMs have a lot to offer me.

afro88 2y ago
Came here to say this. The article assumes that you're already an expert on what you're asking the LLM to help you brainstorm for. A better title would be "LLMs are not suitable for brainstorming in topics you know deeply" I'll also add that there's a big difference between "give me 10 startup ideas" and having a conversation with it to explore different startup ideas (for example). Through conversation you can build off what each other say and explore the space.

schmidt_fifty 2y ago

> LLMs won’t behave in a more creative and independent way (as we hoped), but more susceptible to issues like hallucination. What is creativity if not what people call hallucination? Horribly named phenomenon, btw.

kedean 2y ago
I tried asking an LLM for assistance with Chef Policyfiles today (more specific than that of course), and the response was as if it was for a non-existent product rather than Chef. It even included a code block that it claimed I could run on repl.it if I had an account, which resembled a resource block in chef (policies are not a kind of resource). If a coworker confidently gave me code that is not only wrong, but relies on things that don't exist, I wouldn't call it creative. If they tried to claim that no I'm wrong, it is real, I might even say they hallucinated it.
- schmidt_fifty 2y ago
  [dead]

resource_waste 2y ago

>However, here I would like to argue that (especially in cutting edge scenarios) LLMs are not a good tool to do truly effective brainstorming. Great title, they baited and switched. 95% of the time I don't need effective brainstorming, I need a bunch of ideas, let me pick the best, and move on. If its a real engineering problem, then I need >truly effective brainstorming.

defrost 2y ago
I recently spoke with a cousin about their current project decommissioning an oil and gas platform over the next four years (ie. real engineering (in reverse?)). Their comment was that ChatGPT can't write final reports but it's extremely useful for seeding brainstorming sessions with real human engineers. It can generate a list of health, safety, and environmental concerns related to the project that can be distributed and act as a "starter dot point list" that engineers can agree to, expand on, reject as being just silly, and add to as tengential thoughts arise. I'd argue that makes it useful in the context of real engineering brainstorming .. and that it's being used that way on multi million dollar projects in a billion dollar domain.
- behringer 2y ago
  Yeah for sure, lists of generalized questions is one of ChatGPTs strongest abilities, in my experience.
joegibbs 2y ago
There's a lot of this with current AI - someone will write a piece titled something like "LLMs are not suitable for brainstorming" or "GPT4 is useless for programmers" or "AI image generation has no use cases" - and what they say in the body is that they're not perfect for brainstorming, GPT4 can't write an entire app perfectly from one prompt, and Midjourney doesn't let you take a picture from your mind's eye and put it on paper.
firewolf34 2y ago
Seriously, it's the same argument that people give for "ChatGPT can't give me good code, I don't know why", just rephrased. The deluge of "GPT is not useful for X" articles meant to bait the average critic, despite it being used en masse for "subproblems in X-space" already... They're asking the wrong type of work from it. If you need some boilerplate or a transformation, it's going to give you a fantastic template to work with. If you need it, on the other hand, to engineer out a highly-specific and nuanced solution with an esoteric codebase to a complex problem, maybe not so much. The former is wide, the latter is narrow. It's going to take maybe a bit more breakdown of the scope into proper subproblems before you'll get a good answer; and that's something you can do yourself, or have an agent perform across multiple queries maybe (though I'll admit, more work needs to be done for the whole multi-agent workflows to be truly useful).

Nevermark 2y ago

I find LLMs to be useful tools for brainstorming. I have often inadvertently found myself brainstorming just by explaining something challenging I am working on to a colleague with no expertise in my area. Just explaining forces new perspectives and new ideas. Similarly, LLMs are captive audiences for throwing out ideas and playing with them. The interaction creates something more than just talking to myself or a whiteboard. Not all brainstorming partners need to be first order contributors to have a very positive impact. Different kinds of interactions can spark entirely different kinds of ideas.

jasonjmcghee 2y ago
I agree with this, but also get so irritated with the endless praise. Like, tell me I'm wrong and tell me why. If I present a bad idea, tell me that. Or even like "huh? explain" Looking forward to ramping up the honesty parameter. Hoping OpenAI's new voice model trivializes this so I don't need to prompt engineer.
- fassssst 2y ago
  Curious what happens if you set custom instructions to correct that. Does it work?
- pr337h4m 2y ago
  A simple "Be brutally honest" at the end works surprisingly well.
- vundercind 2y ago
  Me: “I’d like to start a business harvesting cat shit to fight global warming. People would pay me to take their cat shit and bury it because maybe it has carbon in it or something IDK I’m not a scientist. I would give them carbon credits in the form of a cat-shit-sequestration crypto token my business created. I’d pick up the cat shit by bicycle to save carbon emissions. Please help me develop a business plan.” Chargpt: “that’s a great idea for a business! Here are some suggestions…” Not a real exchange, but god, it does feel that way sometimes. One reason you can’t trust the damn thing is it’s too positive.

infinite-hugs 2y ago

I’ll add this here for the fake internet points but also as a general PSA. Most people are not aware that “Brainstorming” although popularized as a process since its origins in the 1960s… is actually Step 2. Most people are unaware that Step 1 is called “Questionstorming”. At the heart of the process is leveraging divergent and convergent modes of thinking which is done both to generate questions (and select the most promising ones) and answers… you’re welcome :D

gavanwilhite 2y ago
Fantastic context. I love hourglass shaped brainstorming, and the “questions” prompt is a solid one!
- sitkack 2y ago
  With LLMs, having questions is more important than having answers. When you run out of questions, the exploration stops.
  - nathanasmith 2y ago
    I just keep prompting with "continue" when that happens.
    - sitkack 2y ago
      "continue" is now the most important word in the english language right after "No" right now. This causes the LLM to orbit around some point in the latent space, but doesn't cause it to explore. You have to tell it to pick a direction and there is no better direction than a question.

dahart 2y ago

> The reason [LLMs are not a good tool to do truly effective brainstorming] is LLMs are trained to follow existing patterns in the human-produced corpus, and not natively taught to “brainstorm”. The problem with this argument is that people do the same thing, we’re not that great at brainstorming either. When we brainstorm in groups, we’re just bringing multiple points of view together. The more data LLMs are trained on, the more viewpoints it might be able to bring that you haven’t considered. That said, LLMs and all NNs so far are built to interpolate, and they are bad and have unbounded error when extrapolating outside their training examples. That is a good reason to not expect today’s AI to come up with new ideas.

trueismywork 2y ago
We don't need more data, we need independent LLMs with slight randomness.
- dahart 2y ago
  Aren’t independent LLMs just more data? Some AI researchers are arguing that more data is the primary thing that got us this far, and the primary thing that will improve AI from here. Here’s an example (and a very good talk whether you believe my summary) https://www.youtube.com/live/a13aqr07tJ4?si=FZO5m_XzrfpDhyQP https://www.youtube.com/live/a13aqr07tJ4?si=FZO5m_XzrfpDhyQP As a part-time generative artist for several decades, a user of Monte-Carlo methods on a daily basis, and author of some papers on the topic, I personally believe that randomness is not a good answer to anything creative. Randomness is boring, and average. Randomness only helps you when you have a high quality Markov model constraining what your RNG chooses between, and that’s more or less all LLMs actually are. Adding more randomness to creative works in general makes creative works muddy and lowers quality, it needs to be guided. Randomness is a useful tool, but is widely misunderstood IMO and not very effective at brainstorming style exploration; brainstorming is about solving problems in interesting ways, i.e., there is reasoning behind it, not slightly more random word salad.
nequo 2y ago
Setting the temperature to a higher value would emulate this, wouldn't it? Then the model would be more willing to deviate from the most likely next token.

nl 2y ago

This is utterly wrong. The strength of LLMs is that they have very wide knowledge, so while your specialist knowledge might surpass them in a particular area it will know more than you across other topics. When brainstorming this wide knowledge is what you want. The trick is (as always) better prompting to push it hard - so things like "consider parallels in similar situations in other fields" are useful.

platz 2y ago

not suitable = better-than-average?

WhitneyLand 2y ago

Brainstorming startup ideas? How often is that a successful approach at all? I can’t think of a good case study off the top of my head.

little_name 2y ago
I don't know a case study but I was personally told by 2 different founders who had 100MM+ exits that their startup ideas were formed (at least partially) from brainstorm sessions. One guy even pointed to me the library where they had the whiteboard session which led to the idea that they exited in the end. The SPC also have a blog that proposes brainstorming from -1 to 1: https://blog.southparkcommons.com/how-to-go-from-minus-1-to-0/ https://blog.southparkcommons.com/how-to-go-from-minus-1-to-...

labrador 2y ago

Counterpoint: LLMs are very suitable for brainstorming if prompted appropriately LLMs are going to give you the consensus reality (average opinion? average facts?) but you can easily steer it into offbeat, controversial and esoteric areas of it's training with the right prompts

behringer 2y ago
I spent a good long while trying to come up with novel video-game game play ideas, exactly what one would call "brainstorming" for ideas, and quite frankly it was pretty awful. ChatGPT more or less it converged on simply taking the current subject, adding a generic gameplay element to it and outputting it. It took half the ideas being thrown at it and included a rhythm mechanic... That's not that interesting, and I just couldn't get it to really think outside the box.
- labrador 2y ago
  That's a good point I forgot to mention. It really depends on your subject matter. I put a clip up supporting why I think LLMs are good for brainstorming from Marc Andreeson about this topic in case your interested "Marc Andreessen says with the right prompting, you can unlock the latent super genius in AI models" https://www.youtube.com/watch?v=N2yN4IG8UYA https://www.youtube.com/watch?v=N2yN4IG8UYA And here's my comment on another site to a pro novelist who suddenly discovered Claude wrote like a genius who posits that Anthropic deliberatly cripples Claude's writing ability because they don't want to scare writers suddenly, they want to ease them into it I disagree. Your supernatural scenes triggered words an analysis from higher quality writers and commenters in the training data. If you were writing about bass fishing it would likely not impress you with it's writing. You can try something like this as an experiment. In other words it's good at some writing and bad at others, depending on the training data. I'd love to be proven wrong, like a gripping story about bass fishing might be interesting.

bawolff 2y ago

> What’s worse is when we ask topics that don’t have consensus currently, the LLMs won’t behave in a more creative and independent way (as we hoped), but more susceptible to issues like hallucination. I mean, what is the difference between creativity and hallucination (honest question)? --- Maybe this means AI would be better at the opposite - after you brainstorm creative out there ideas, AI can tell you if they have been tried in the past and what the consensus view is on why they failed, allowing you to adjust course.

chefandy 2y ago
> I mean, what is the difference between creativity and hallucination (honest question)? Well, creative problem solving involves getting new ideas and perspectives by creating novel associations between things we know. Hallucinations are creating things we "know" because they sound like they could be right and basing "ideas" on them. In short, hallucinations are bullshit. Totally open creative problem solving isn't always perfect, but even the ideas that don't really work can reveal something about the concepts involved. And bullshit often takes creativity to create, but that doesn't make it useful as actual creative output. It's not like the second you move beyond empirically provable statements, everything has equal merit. If you're trying to creatively work to a useful end, whether or not the knowledge you're working with is fabricated is pretty consequential. Doing otherwise would be like trying to optimize your code based on utterly fake but plausible algorithms-- sounding right isn't right enough to be useful. IMO, being able to create lies that pass the smell test is LLMs' most dangerous proclivity, especially when they're presented as expertise-in-a-box.
- bawolff 2y ago
  Its not like humans always have good creative output. A lot of human creative output is essentially unworkable bullshit that gets discarded quite quickly. I'm not saying llms are good at creativity (i certainly don't think they are) but i kind of feel like its a difference in quality not kind. Like if you asked me to describe what it means for a human to be creative, i would probably write something quite similar to what you wrote above.

michael_nielsen 2y ago

There goes 50+% of my use. "LLMs are no good for [use case X]" often means "I aren't very good at using LLMs for [use case X]". With many powerful tools - violins, say, or carpentry tools - we know that it takes a long time and a lot of learning to achieve competent performance, much less virtuoso performance. Someone who spent ten hours learning the violin and concluded "Violins sound terrible" wouldn't have diagnosed a problem with violins, but with their own mastery. I certainly think current LLMs have some big intrinsic weaknesses, but also that what they are is quite subtle.

llm_trw 2y ago
I think it strongly depends on the model. I found that og gpt4 is better for brainstorming than gpt4 turbo, which in tern is better than gtp4o. If you're just using the web based portal you don't get much of a choice which model to use or it's temperature.

sweetheart 2y ago

I'm interpreting "brainstorming" to be any kind of general noodling on an issue to try to find new/novel/interesting solutions. In that case, I think every significant moment of true inspiration I had (of which there have been like, maybe 3 ever), they were always the result of seemingly random, completely unrelated things popping into my mind that, for whatever reason, clicked perfectly into the problem I was mulling over. To me, this means that manufacturing "true" inspiration doesn't require a tool that can deviate from "standard" human thinking patterns. I think it just means that you would want a tool that helps expose you to as many new and unknown fields/concepts/ideas in as little time as possible. So I think in that way LLMs are an amazing tool for helping one to brainstorm.

deleted 2y ago

[deleted]

locallyidle36 2y ago

Strongly disagree - brainstorming isn’t about asking someone to give you creative answers, it’s a team sport about triggering unique thoughts that neither person would have had on their own. Although I might be biased, I love using AI for brainstorming and started building a tool for it https://youtu.be/t5gfETbUzy0 https://youtu.be/t5gfETbUzy0

chiefalchemist 2y ago

Disagree. LLM might be mostly bad at it, but that doesn't make them unsuitable. They don't have a career to worry about. They don't care what peers or the boss thinks. Etc. They can spit out ideas - LOTS of them - that real humans then use as a starting point to carry on with. For more details see "Co-intelligence" by Ethan Mollick.

6510 2y ago

Wait, does that mean hallucinations should be embraced and improved uppon?

__loam 2y ago

LLMs seem to be unsuitable for a lot of work.

BOOSTERHIDROGEN 2y ago

I'm curious for someone like Ilya does he use LLM for daily brainstorming ?

mirekrusin 2y ago
Ilya's LLM is using Ilya for its own benefits.

bcstyle 2y ago

Author here. Was not expecting this quick post being picked up by HN - thanks for all the comments! I want to acknowledge that the original title is inaccurate, as many of you have pointed out. It should be "(current) LLMs don't brainstorm novel things really well" rather than just brainstorming. It's not intended to be a clickbait though - I was kind of mixing two definitions of brainstorming unintentionally. When we refer to the group activity that aims to collect all angles from participants (and common wisdoms), LLMs are really good, and it's something I do on a regular basis. However when it comes to the hope of reaching novel ideas that don't exist before (which some of us will consider what distinguishes brainstorming from group discussion or research study), I would say today's LLMs don't do well. I've seen such issue in business and arts domains, and also someone here mentioned similar experience in video game design. I would argue that (so far) for any idea LLMs tell us, there exists at least one instance of a similar pattern in the training data (either exact or in a high level). If this is what you need, then great. But some problems require more than that. And I would argue that a lot of important innovations in history didn't follow this pattern. I'm aware of reports on LLMs helping research (e.g. the works shared by Terry Tao), but I don't think they contradict the point here. Will be super happy to be proven wrong though!

sitkack 2y ago
> Was not expecting this quick post being picked up by HN - thanks for all the comments! You submitted the post?! It got “picked up” so quickly because it is wrong. I think what your post does show is how effectively you can farm HN for supporting arguments. You provide zero evidence for your claims, nor do you give us any transcripts of your attempts. Every post that matches the structure of yours is usually a summary of how the author has low skill in using LLMs. Though I am absolutely delighted in the comments and the nih paper referenced was a joy to read.

aoeusnth1 2y ago

I think effective brainstorming is actually a matter of prompt engineering, which is to say, breaking the model out of its box and into unexplored territory. See https://twitter.com/repligate https://twitter.com/repligate for some extreme examples of this.

deleted 2y ago
[deleted]

infogulch 2y ago

I've found LLMs to be great inspiration and very helpful for writing. My basic process is: 1. Ask agent to write a few paragraphs. 2. Notice that it's terrible and it would be easier to rewrite it myself than fix it. 3. Actually be motivated to write.

paulmd 2y ago

Sure they are. Just not infallible/perfect, but yes, they certainly are capable of mixing up inputs and coming up with something that’s not in the input set. No, they won’t come up with a totally new set of concepts and axioms that are totally unconnected to any human set… and you don’t want them to. Because those are the actual hallucinations. New ideas need to be novel, but they still need to fit into the rest of the set. It’s remarkable just how much people race to be pessimistic about LLMs, even when the pessimistic assertions are obviously and factually incorrect. Like yes we know LLMs have “hallucinations”, which is a loaded term to begin with, because the pessimists literally never shut up about it, even though it’s a fairly trivial overall problem given the immense overall utility. Some people are clearly so very very emotionally invested in there being nothing productive or useful about AI if it’s not AGI or if it needs the slightest amount of correction or oversight.