5 ms·

How does a group help anything? If you put 1000 dumb people together, they don't magically become smart?

by bboygravity 2y ago

sunshinerag 2y ago
Wait what … how does democracy work then?
- nfw2 2y ago
  the benefit of democracy is primarily that it prevents governments from doing bad things, less so that it empowers more effective governance
  - mathgeek 2y ago
    It can do either, and can fail to do either. It’s the people having power that enables the outcomes, not the system itself. Democracy just grants the power to a broader set of people.
- optimalsolver 2y ago
  It doesn't.
- coldtea 2y ago
  Democracy is not about being smart or dumb. It's about everyboty having a say in decisions of government that affects them. The failure of democracy as a system is not when people make dumb decisions (experts and high-IQ people have made some of the most stupid and catastrophic decisions in history), but when people's collective decisions are not being respected.
littlestymaar 2y ago
> If you put 1000 dumb people together, they don't magically become smart? 1000 is probably too high, but groups of people are in fact more intelligent than individuals (though for humans it is likely because recognizing a correct answer is easier than finding it in the first place)
- nfw2 2y ago
  depends on the circumstances. lin-manuel miranda can probably write a better musical by himself than a team of 20 people with equal input would. also, the bottlenecks that teamwork helps solve (eg the high cost of gaining expertise and low throughput of reasoning capacity) may not be that relevant in the ai age
  - littlestymaar 2y ago
    > by himself than a team of 20 people with equal input would. Sure, but the result would still be far better than the average of the output of the 20 individuals taken alone. > also, the bottlenecks that teamwork helps solve (eg the high cost of gaining expertise and low throughput of reasoning capacity) may not be that relevant in the ai age It's always tempting to anthropomorphize these systems and conclude that what works for us would work for them, but yes we don't really know if it would bring anything to AI.
- TheOtherHobbes 2y ago
  Functional groups which work well together, include open sharing of research and ideas, persistence of best output, are dedicated to realism, and are more focussed on problem solving than status display, will be smarter. The group works like a filter which generates multiple solutions and selects, remembers, and abstracts the best. Dysfunctional groups which do the opposite will be catastrophically stupid. There have been plenty of dysfunctional groups in history.
- torginus 2y ago
  I wonder if there's research on this, like if you took a group of individuals who scored the same on an IQ test, then got them to solve one together, how would the score improve? Is there a way of selecting people to cover each other's intellectual blind spots?
IshKebab 2y ago
If you put 1000 people who can't talk together they will create language so they can communicate. He's saying if we put LLMs together and don't force them to use English to communicate then they'll create their own language which may be superior for LLMs to English. May be true but who knows. I wonder if anyone has somehow tested the Sapir-Whorf hypothesis for LLMs somehow by training them on different languages and comparing task performance. I guess it's too difficult to get a large equivalent training set in different languages.
- wodderam 2y ago
  It feels like an exercise in anthropomorphization to me. Sapir-Whorf hypothesis is generally not considered to be reality. It makes intuitive sense but is wrong. There are hours of podcasts with Chomsky talking about LLMs. The gist of which is that LLMS are extracting surface level statistical structure of language that will be good for routine coding and not much else. It is easy to infer that Chomsky would believe this idea to be utter nonsense. I believe even the idea of getting a 1000 people together and we agree to label a rock "rock", a tree "tree", a bird "bird" is not even how human language works. Something that is completely counter intuitive. Reading the paper, no one believes a hidden markov model is creating some kind of new thought process in the hidden state. I certainly though could have no idea what I am talking about with all this and have pieced together parts that make no sense while this is a breakthrough path to AGI.
  - pjerem 2y ago
    Well maybe not 1000 people but to our knowledge, the human brain is actually made of physically independent zones that barely communicate together except with the zone that take all the outputs together and tries to do something coherent with all the garbage. Idk if this could work with LLMs, especially because all the brain zones are somehow specialized into something while two LLMs are just identical machines. But we also know that the specialization isn’t that hardcoded : we know that people losing half their brain (after a stroke) can still relearn things that were managed in the "dead" part. I don’t know, please correct my errors, I was just thinking aloud to say that multiple independent agents working together may be how "intelligence" already works in the biological world so why not for AIs ?
deleted 2y ago
[deleted]
JFingleton 2y ago
> If you put 1000 dumb people together, they don't magically become smart? Do they not become smart*er* though?
- computably 2y ago
  "Smarter" is too vague. A group can compensate for individual weaknesses or even converge on a hard-to-make prediction given sufficiently uncorrelated outputs; basically the idea behind ensemble models / wisdom of the crowds. But a group of 1000 dumb apes would never achieve categorically-above-ape intelligence, probably not even "genius" ape intelligence. Groups of unintelligent agents come with downsides as well, like the ant death spiral.
  - coldtea 2y ago
    >But a group of 1000 dumb apes would never achieve categorically-above-ape intelligence And yet, here we are. A group of 1000 apes is large enough to have offspring and, given time, go through evolution.
mromanuk 2y ago
Because group estimation is superior to individual estimations: The phenomenon is called wisdom of the crowds. When a group of people independently estimate something, individual errors tend to cancel each other out, leading to a surprisingly accurate collective result. This works because of: Diversity of opinions: Different perspectives bring a range of estimates. Independence: Errors aren't systematically biased as long as individuals estimate without external influence. Error averaging: Overestimation and underestimations balance out when averaged. Law of large numbers: More participants increase accuracy by minimizing random errors. It was demonstrated by Francis Galton in 1906, where a crowd's average guess of a bull's weight was almost spot-on. (estimates must be independent and reasonably informed for this to work.)
senectus1 2y ago
they kinda do.. Its how City's work. People learn by being around others being both successful and unsuccessful.
coldtea 2y ago
Isn't that the very case behind the "wisdom of crowds" thing?
- amelius 2y ago
  Looking at the current state of democracies around the world, my hopes are not on "wisdom of the crowds".
  - bee_rider 2y ago
    If you think the democracies are doing bad, you should see the autocracies!
    - amelius 2y ago
      You mean the thing democracies are turning into, thanks to social (crowd wisdom) media?
      - bee_rider 2y ago
        I don’t think social media really is crowd wisdom at all. It is built to pander to our worst impulses (I think, knowingly and openly, right? The algorithm selects for engagement, not learning and growing), and I’d be surprised if it isn’t producing a feedback loop as well (perhaps as an unintentional side effect). The wisdom of the crowds hypothesis relies on a random sampling, we’re intentionally applying a skew toward the angry and shallow.
      - coldtea 2y ago
        No, he means the thing democracies had turned to, when hardly differentiating parties turned into a practocal "uniparty" in economic, corporate, and foreign policy, and ruled by pissing on what the people voted for, which the current populist backlash is a reaction against, as elites (and supporters) lament as "too much democracy" and scorn the ignorant plebes (case in point) and pine for censorship and "expert" rule.
konart 2y ago
Not magically. Our great ancestors were pretty dumb, but they were getting smarter and better because of sharing their knowledge.
- ulbu 2y ago
  they were not one bit dumber than you.
  - EliBullockPapa 2y ago
    Average intelligence measures have risen substantially since early 1900s https://en.wikipedia.org/wiki/Flynn_effect https://en.wikipedia.org/wiki/Flynn_effect
- pigpop 2y ago
  yes they got "smarter" by compiling a corpus of knowledge which future generations could train on. sarcasm aside, throwing away the existing corpus in favor of creating a new one from scratch seems misguided. this paper isn't about creating a new language, they are omitting the sampler that chooses a single token in favor of sending the entire end state back in to the model like a superposition of tokens. that's the breadth first search part, they don't collapse the choice down to a single token before continuing so it effectively operates on all of the possible tokens each step until it decides it's done. it would be interesting to try this with similar models that had slightly different post training if you could devise a good way to choose the best answer or combine the outputs effectively or feed the output of a downstream model back in to the initial model, etc. but I'm not sure if there'd necessarily be any benefit to this over using a single specialized model.

How does a group help anything? If you put 1000 dumb people together, they don't magically become smart?

sunshinerag 2y ago

Wait what … how does democracy work then?

nfw2 2y ago
the benefit of democracy is primarily that it prevents governments from doing bad things, less so that it empowers more effective governance
- mathgeek 2y ago
  It can do either, and can fail to do either. It’s the people having power that enables the outcomes, not the system itself. Democracy just grants the power to a broader set of people.
optimalsolver 2y ago
It doesn't.
coldtea 2y ago
Democracy is not about being smart or dumb. It's about everyboty having a say in decisions of government that affects them. The failure of democracy as a system is not when people make dumb decisions (experts and high-IQ people have made some of the most stupid and catastrophic decisions in history), but when people's collective decisions are not being respected.

littlestymaar 2y ago

> If you put 1000 dumb people together, they don't magically become smart? 1000 is probably too high, but groups of people are in fact more intelligent than individuals (though for humans it is likely because recognizing a correct answer is easier than finding it in the first place)

nfw2 2y ago
depends on the circumstances. lin-manuel miranda can probably write a better musical by himself than a team of 20 people with equal input would. also, the bottlenecks that teamwork helps solve (eg the high cost of gaining expertise and low throughput of reasoning capacity) may not be that relevant in the ai age
- littlestymaar 2y ago
  > by himself than a team of 20 people with equal input would. Sure, but the result would still be far better than the average of the output of the 20 individuals taken alone. > also, the bottlenecks that teamwork helps solve (eg the high cost of gaining expertise and low throughput of reasoning capacity) may not be that relevant in the ai age It's always tempting to anthropomorphize these systems and conclude that what works for us would work for them, but yes we don't really know if it would bring anything to AI.
TheOtherHobbes 2y ago
Functional groups which work well together, include open sharing of research and ideas, persistence of best output, are dedicated to realism, and are more focussed on problem solving than status display, will be smarter. The group works like a filter which generates multiple solutions and selects, remembers, and abstracts the best. Dysfunctional groups which do the opposite will be catastrophically stupid. There have been plenty of dysfunctional groups in history.
torginus 2y ago
I wonder if there's research on this, like if you took a group of individuals who scored the same on an IQ test, then got them to solve one together, how would the score improve? Is there a way of selecting people to cover each other's intellectual blind spots?

IshKebab 2y ago

If you put 1000 people who can't talk together they will create language so they can communicate. He's saying if we put LLMs together and don't force them to use English to communicate then they'll create their own language which may be superior for LLMs to English. May be true but who knows. I wonder if anyone has somehow tested the Sapir-Whorf hypothesis for LLMs somehow by training them on different languages and comparing task performance. I guess it's too difficult to get a large equivalent training set in different languages.

wodderam 2y ago
It feels like an exercise in anthropomorphization to me. Sapir-Whorf hypothesis is generally not considered to be reality. It makes intuitive sense but is wrong. There are hours of podcasts with Chomsky talking about LLMs. The gist of which is that LLMS are extracting surface level statistical structure of language that will be good for routine coding and not much else. It is easy to infer that Chomsky would believe this idea to be utter nonsense. I believe even the idea of getting a 1000 people together and we agree to label a rock "rock", a tree "tree", a bird "bird" is not even how human language works. Something that is completely counter intuitive. Reading the paper, no one believes a hidden markov model is creating some kind of new thought process in the hidden state. I certainly though could have no idea what I am talking about with all this and have pieced together parts that make no sense while this is a breakthrough path to AGI.
- pjerem 2y ago
  Well maybe not 1000 people but to our knowledge, the human brain is actually made of physically independent zones that barely communicate together except with the zone that take all the outputs together and tries to do something coherent with all the garbage. Idk if this could work with LLMs, especially because all the brain zones are somehow specialized into something while two LLMs are just identical machines. But we also know that the specialization isn’t that hardcoded : we know that people losing half their brain (after a stroke) can still relearn things that were managed in the "dead" part. I don’t know, please correct my errors, I was just thinking aloud to say that multiple independent agents working together may be how "intelligence" already works in the biological world so why not for AIs ?

deleted 2y ago

[deleted]

JFingleton 2y ago

> If you put 1000 dumb people together, they don't magically become smart? Do they not become smart*er* though?

computably 2y ago
"Smarter" is too vague. A group can compensate for individual weaknesses or even converge on a hard-to-make prediction given sufficiently uncorrelated outputs; basically the idea behind ensemble models / wisdom of the crowds. But a group of 1000 dumb apes would never achieve categorically-above-ape intelligence, probably not even "genius" ape intelligence. Groups of unintelligent agents come with downsides as well, like the ant death spiral.
- coldtea 2y ago
  >But a group of 1000 dumb apes would never achieve categorically-above-ape intelligence And yet, here we are. A group of 1000 apes is large enough to have offspring and, given time, go through evolution.

mromanuk 2y ago

Because group estimation is superior to individual estimations: The phenomenon is called wisdom of the crowds. When a group of people independently estimate something, individual errors tend to cancel each other out, leading to a surprisingly accurate collective result. This works because of: Diversity of opinions: Different perspectives bring a range of estimates. Independence: Errors aren't systematically biased as long as individuals estimate without external influence. Error averaging: Overestimation and underestimations balance out when averaged. Law of large numbers: More participants increase accuracy by minimizing random errors. It was demonstrated by Francis Galton in 1906, where a crowd's average guess of a bull's weight was almost spot-on. (estimates must be independent and reasonably informed for this to work.)

senectus1 2y ago

they kinda do.. Its how City's work. People learn by being around others being both successful and unsuccessful.

coldtea 2y ago

Isn't that the very case behind the "wisdom of crowds" thing?

amelius 2y ago
Looking at the current state of democracies around the world, my hopes are not on "wisdom of the crowds".
- bee_rider 2y ago
  If you think the democracies are doing bad, you should see the autocracies!
  - amelius 2y ago
    You mean the thing democracies are turning into, thanks to social (crowd wisdom) media?
    - bee_rider 2y ago
      I don’t think social media really is crowd wisdom at all. It is built to pander to our worst impulses (I think, knowingly and openly, right? The algorithm selects for engagement, not learning and growing), and I’d be surprised if it isn’t producing a feedback loop as well (perhaps as an unintentional side effect). The wisdom of the crowds hypothesis relies on a random sampling, we’re intentionally applying a skew toward the angry and shallow.
    - coldtea 2y ago
      No, he means the thing democracies had turned to, when hardly differentiating parties turned into a practocal "uniparty" in economic, corporate, and foreign policy, and ruled by pissing on what the people voted for, which the current populist backlash is a reaction against, as elites (and supporters) lament as "too much democracy" and scorn the ignorant plebes (case in point) and pine for censorship and "expert" rule.

konart 2y ago

Not magically. Our great ancestors were pretty dumb, but they were getting smarter and better because of sharing their knowledge.

ulbu 2y ago
they were not one bit dumber than you.
- EliBullockPapa 2y ago
  Average intelligence measures have risen substantially since early 1900s https://en.wikipedia.org/wiki/Flynn_effect https://en.wikipedia.org/wiki/Flynn_effect
pigpop 2y ago
yes they got "smarter" by compiling a corpus of knowledge which future generations could train on. sarcasm aside, throwing away the existing corpus in favor of creating a new one from scratch seems misguided. this paper isn't about creating a new language, they are omitting the sampler that chooses a single token in favor of sending the entire end state back in to the model like a superposition of tokens. that's the breadth first search part, they don't collapse the choice down to a single token before continuing so it effectively operates on all of the possible tokens each step until it decides it's done. it would be interesting to try this with similar models that had slightly different post training if you could devise a good way to choose the best answer or combine the outputs effectively or feed the output of a downstream model back in to the initial model, etc. but I'm not sure if there'd necessarily be any benefit to this over using a single specialized model.