16 ms·

Knowledge from small number of debates outperforms wisdom of large crowds (2017)

235 points by Dowwie 8y ago

sharemywin 8y ago
The aggregation of many independent estimates can outperform the most accurate individual judgment. This centenarian finding, popularly known as the wisdom of crowds, has been applied to problems ranging from the diagnosis of cancer to financial forecasting. It is widely believed that social influence undermines collective wisdom by reducing the diversity of opinions within the crowd. Here, we show that if a large crowd is structured in small independent groups, deliberation and social influence within groups improve the crowd's collective accuracy. We asked a live crowd (N=5180) to respond to general-knowledge questions (e.g., what is the height of the Eiffel Tower?). Participants first answered individually, then deliberated and made consensus decisions in groups of five, and finally provided revised individual estimates. We found that averaging consensus decisions was substantially more accurate than aggregating the initial independent opinions. Remarkably, combining as few as four consensus choices outperformed the wisdom of thousands of individuals.
- sharemywin 8y ago
  wonder how that would compare to people providing a confidence to their guess and using a weighted average.
denzil_correa 8y ago
We intuitively do know this and mankind has used different variations of this concept throughout human civilization. Sortition has been used for more than 2000 years to come up with fair representation of the population in governance [0]. A more contemporary example would be the jury system in the US. But, nice to have experimental results on this concept. [0] https://en.wikipedia.org/wiki/Sortition https://en.wikipedia.org/wiki/Sortition
hobofan 8y ago
I would say that this has been known for quite some time in philosophy, but I guess it's good to have some real-life verification for it. This article on Belief Merging and Judgement Aggregation[0] is a good entry point for the field, for anybody that is interested. [0]: https://plato.stanford.edu/entries/belief-merging/ https://plato.stanford.edu/entries/belief-merging/
- deleted 8y ago
  [deleted]
- watersb 8y ago
  Off-topic, but I love the logo at the upper-left corner of the plato.stanford.edu web pages. Stanford has an amazing collection of sculptures by Auguste Rodin, including his iconic "The Thinker". If you have business or just visiting the Silicon Valley area, it's worth exploring the Stanford campus and its Rodin museum. https://en.m.wikipedia.org/wiki/Auguste_Rodin https://en.m.wikipedia.org/wiki/Auguste_Rodin
sytelus 8y ago
You can make a case that democracy is a grand application of “wisdom of large crowd”. But if that can easily and consistently outperformed then do we have better political system than democracy? What are the consequences of this?
- fsloth 8y ago
  Democracy is not a vehicle for correctness or efficiency. It's entire point is to distribute political power to a wide powerbase with a constant churn. This is to disable known pathologies of entrenched power bases and to discourage social stratification. Since modern states don't have an entrenched hereditary aristocracy we don't really have a concept of what democracy shields us from, since we've not had to suffer from it - except when we do, we recognize it and shun it. The social idea was that aristocracy had to be in power since they were the ones with the education. After public schooling became a things, suddenly just being able to read and write is not a distinguishing feature, and having an explicit hereditary elite became a socially untenable construct. That's not to say that a human society is a fair playing field - but it's not as unjust as in past centuries. Similarly to vacciness, when democracy works as intended you see only the "bad parts".
  - agsdfgsd 8y ago
    >Since modern states don't have an entrenched hereditary aristocracy We absolutely do, but because we are bombarded with propaganda telling us they "earned it" by "working hard" and "pulling themselves up by their bootstraps" we pretend it is just a co-incidence that the people born into the top 1% almost always remain there, and people born in the bottom 99% almost always remain there. >After public schooling became a things, suddenly just being able to read and write is not a distinguishing feature It never was. The aristocracy were educated specifically on economics, justice, etc. It wasn't simply a matter of being able to read.
    - fsloth 8y ago
      Sure, I simplified a lot. The current western world order is still much more fair to the common folk than feudalism. The education aspect was not as much about how much more educated the aristocracy was rather that the unwashed masses rose to their level, and it was not anymore sufficient to say that god has decreed duke X a distinguished gentleman and commoner Y a filthy peasant. It is not easy to rise social ladders, but it's way easier with our figment of a fair society, rather than if we believed social status was a fixed quantity given by the creator of the universe. The pithy armchair psychologist in me would claim modern society is much more facilitating towards a growth mindset than a feodal one.
notahacker 8y ago
There's a big unanswered question about whether "debate" is quite so useful at improving accuracy of answers when it consists of an audience with strong priors listening to motivated reasoning rather than people deferring to the people who are most confident they have a decent grasp of the subject of a neutral trivia question. I think it's conceivable the opposite effect might occur if the experiment were be to repeated using polarising political issues, even if the questions themselves were fact-based (economic growth rates, crime rates, immigration figures, global temperature changes etc).
- wisty 8y ago
  The backfire effect seems to be heavily exaggerated - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2819073 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2819073 And just think about how the research is done. "Hello conservative test subject. Here's a 1 page article explaining why Climate Change is a real problem". What impact do you think it will have? How about this. Dear liberal, belief in a just world is psychologically healthy, and belief in an unjust world is linked to mental health issues, where's a wikipedia link - https://en.wikipedia.org/wiki/Just-world_hypothesis#Current_research https://en.wikipedia.org/wiki/Just-world_hypothesis#Current_... It seems likely to me that a couple of pithy arguments are unlikely to change a core belief, but is very likely to create some short-term pushback. If they do change a core belief then I think it will have stronger and longer-lasting impacts than the push-back. I also suspect there's a delay - arguments might take a while to sink in.
- patcon 8y ago
  In the same experiment (but still unpublished) they actually find evidence that a similar CONSENSUS effect is observed in polarized issues with no clear "best" choice. https://www.ted.com/talks/mariano_sigman_and_dan_ariely_how_can_groups_make_good_decisions/transcript?language=en https://www.ted.com/talks/mariano_sigman_and_dan_ariely_how_... The most interesting bit to me is the fact that a rare middle position stakeholder, a "high confident gray", allows groups to reach consensus more often. This potentially has huge applications in system design of the processes of democracy imho
theontheone 8y ago
I think attributing the accuracy to debates is a rather hasty conclusion. I would love to see an experiment where the individual participants rated their confidence from 1 to 10, and only the highest confidence answer in each group was taken. My hunch is this would perform just as well (if not better) than post-discussion.
ajuc 8y ago
Am I correct that they asked the same people the same questions 3 times? It's no surprise that the 3rd time the answers were the best. I'd expect this to happen even if there was no debate (if not to the same degree). > Each participant was provided with pen and an answer sheet linked to their seat number. The event’s speaker (author M.S.) conducted the crowd from the stage (Fig. 1A). In the first stage of the experiment, the speaker asked eight questions (Supplementary Table 1) and gave participants 20 seconds to respond to each of them (stage i1, left panel in Fig. 1A). Then, participants were instructed to organize in to groups of five based on a numerical code in their answer sheet (see Methods). The speaker repeated four of the eight questions and gave each group one minute to reach a consensus (stage c, middle panel in Fig. 1A ). Finally, the eight questions were presented again from stage and participants had 20 seconds to write down their individual estimate, which gave them a chance to revise their opinions and change their minds (stage i2, right panel in Fig. 1A). Participants also reported their confidence in their individual responses in a scale from 0 to 10.
- thenaturalist 8y ago
  I can’t follow your reasoning. How does asking you the exact same question twice improve your accuracy? If you don’t know how high the Eiffel Tower is, why would you know it the second time? Conversely, you likely created a mental anchor when giving an estimate the first time and would be hard pressed to provide an answer contradicting your first estimate - even if it might be more accurate. There were no answers provided between asking the questions as far as I understand this excerpt. In the group only 4/8 questions were asked and here a difference in accuracy can be made. Maybe you’re mixed with travelers who recently visited Paris, historians or people who due to other circumstances or pure luck provide more accurate estimates, effectively influencing the anchor of your first estimate.
  - ajuc 8y ago
    Given more time I can remember something, or I can notice that the last question gives some hints or even unrelated associations that help to answer the first question. I don't think "if you don't know the answer in 20 seconds you won't ever know it" is true.
ivanmaeder 8y ago
I've only read the summary but Philip Tetlock comes to this conclusion based on his work with "The Good Judgment Project" (described in his book "Superforecasting"). The GJP is a kind of experiment he's been running for a few years in an attempt to learn how to improve predictions. https://en.wikipedia.org/wiki/The_Good_Judgment_Project https://en.wikipedia.org/wiki/The_Good_Judgment_Project From what I remember: overall, teams did better than individuals and wisdom of the crowds because they were able to feed off and combine each other's points of view and separate knowledge. However! It's important for teams to not let groupthink dominate—individuals within teams needed to challenge each other.
amelius 8y ago
Perhaps we can save democracy by replacing the voting mechanism by placing people in groups of 10, and letting them reach consensus before making a vote.
- aisengard 8y ago
  I mean, this is essentially how the House of Representatives works.
- deytempo 8y ago
  You mean like trial by jury. I like it.
  - mchannon 8y ago
    Trial by jury gets one set of 10 (12, whatever), whereas the consensus model described would have n sets of 10. I seriously doubt the wisdom of a crowd goes up when the total crowd size shrinks to 10. I suspect it plummets. This does call into question the notion of a multiple-jury trial (maybe videoconferenced in, where the individual juries do not interact with each other). Based on this study's findings, it would appear that juries might get their verdicts wrong less frequently with this system. Imagine if the first OJ trial or Trayvon Martin trial had been before multiple juries; would society have been better off if the results were less dependent on the biases of juries small enough to be cherry-picked?
    - sova 8y ago
      To summarize the findings of this research article for you: Wisdom of the Crowd vs one person = good Wisdom of the crowd vs tiny groups 4+ = bad
    - deytempo 8y ago
      In this case a jury is not really about everyone in it being wise, it’s about there being a better chance that an informed decision will take place due to those in the group discussing the decision before voting rather than just voting for whatever CNN recommends. With the current system you just have people doing whatever they are going to do without having to explain it to anyone else.
baud147258 8y ago
It reminds me of that assertion that the IQ of a crowd is the lowest IQ divided by the number of people in the crowd. Seems the study validate this with that smaller groups performs better
- prewett 8y ago
  Except that the previous study refuted that idea: the average answer was better than individual answers, which is definitely not IQ_crowd = IQ_min / n.
hkt 8y ago
Deliberative democracy is great. Put people in enriched decisionmaking environments for better outcomes, and then get greater democratic legitimacy from doing so than even some representatives? Win win. This is the opposite of populism too: when we voted Brexit in the UK, it was an uncontrolled, information-scarce (and falsehood-rich) environment. A room where people were organised and where they asked for their own experts and talked to one another would never have delivered that result.
- sbhn 8y ago
  People in the uk voted brexit, because it was just after the uk us and france were dropping bombs in lybia and syria, arab spring they called it, the destruction of infrastructure created millions of refugees, the bbc amplyied the importance and strength of isis, and flashed images of thousands of the refugees in calais tripping over each other trying to illegally cross the channel into the uk. The good souls of england thought they were under attack, bless em.
sova 8y ago
The line to know: "Remarkably, combining as few as four consensus choices outperformed the wisdom of thousands of individuals." Confer away!
patcon 8y ago
This TED video tuned me into this research earlier in 2018, and while not rigorously tested, there are some even more interesting hypotheses that are being sussed out of the experiment: https://www.ted.com/talks/mariano_sigman_and_dan_ariely_how_can_groups_make_good_decisions/transcript?language=en https://www.ted.com/talks/mariano_sigman_and_dan_ariely_how_...
ebetica0 8y ago
I wonder how much statistical independence has to play in this. e.g. the large crowd can be biased when they influence each other in a way that the small debates cannot. What happens when you have small independent debates versus large independent crowds?
PaulHoule 8y ago
We had poker planning introduced to my team and within two months or so we got to the point where most of the time we all get the same number or get something like 8-13-21.
reggieband 8y ago
Whenever I come across such ideas it makes me want to more deeply investigate the Communicative Rationality [1] of Jurgen Habermas. Despite modern trends towards a conservative stoicism (e.g. Jordan Peterson) I actually think Social Critical Theory feels to be moving in the more-correct direction. However, the current negative association with moral-relativism and postmodernism is a new McCarthyism. The bridge should be pragmatism but even that is too closely associated with socialism. Too many isms but the ones that are scaring me most now are authoritarianism, fascism and totalitarianism. Ideas like communicative rationality feel to me like the most reasonable solutions. [1] https://en.wikipedia.org/wiki/Communicative_rationality https://en.wikipedia.org/wiki/Communicative_rationality [2] https://en.wikipedia.org/wiki/Critical_theory https://en.wikipedia.org/wiki/Critical_theory

235 points by Dowwie 8y ago

The aggregation of many independent estimates can outperform the most accurate individual judgment. This centenarian finding, popularly known as the wisdom of crowds, has been applied to problems ranging from the diagnosis of cancer to financial forecasting. It is widely believed that social influence undermines collective wisdom by reducing the diversity of opinions within the crowd. Here, we show that if a large crowd is structured in small independent groups, deliberation and social influence within groups improve the crowd's collective accuracy. We asked a live crowd (N=5180) to respond to general-knowledge questions (e.g., what is the height of the Eiffel Tower?). Participants first answered individually, then deliberated and made consensus decisions in groups of five, and finally provided revised individual estimates. We found that averaging consensus decisions was substantially more accurate than aggregating the initial independent opinions. Remarkably, combining as few as four consensus choices outperformed the wisdom of thousands of individuals.

sharemywin 8y ago
wonder how that would compare to people providing a confidence to their guess and using a weighted average.

denzil_correa 8y ago

We intuitively do know this and mankind has used different variations of this concept throughout human civilization. Sortition has been used for more than 2000 years to come up with fair representation of the population in governance [0]. A more contemporary example would be the jury system in the US. But, nice to have experimental results on this concept. [0] https://en.wikipedia.org/wiki/Sortition https://en.wikipedia.org/wiki/Sortition

hobofan 8y ago

I would say that this has been known for quite some time in philosophy, but I guess it's good to have some real-life verification for it. This article on Belief Merging and Judgement Aggregation[0] is a good entry point for the field, for anybody that is interested. [0]: https://plato.stanford.edu/entries/belief-merging/ https://plato.stanford.edu/entries/belief-merging/

deleted 8y ago
[deleted]
watersb 8y ago
Off-topic, but I love the logo at the upper-left corner of the plato.stanford.edu web pages. Stanford has an amazing collection of sculptures by Auguste Rodin, including his iconic "The Thinker". If you have business or just visiting the Silicon Valley area, it's worth exploring the Stanford campus and its Rodin museum. https://en.m.wikipedia.org/wiki/Auguste_Rodin https://en.m.wikipedia.org/wiki/Auguste_Rodin

sytelus 8y ago

You can make a case that democracy is a grand application of “wisdom of large crowd”. But if that can easily and consistently outperformed then do we have better political system than democracy? What are the consequences of this?

fsloth 8y ago
Democracy is not a vehicle for correctness or efficiency. It's entire point is to distribute political power to a wide powerbase with a constant churn. This is to disable known pathologies of entrenched power bases and to discourage social stratification. Since modern states don't have an entrenched hereditary aristocracy we don't really have a concept of what democracy shields us from, since we've not had to suffer from it - except when we do, we recognize it and shun it. The social idea was that aristocracy had to be in power since they were the ones with the education. After public schooling became a things, suddenly just being able to read and write is not a distinguishing feature, and having an explicit hereditary elite became a socially untenable construct. That's not to say that a human society is a fair playing field - but it's not as unjust as in past centuries. Similarly to vacciness, when democracy works as intended you see only the "bad parts".
- agsdfgsd 8y ago
  >Since modern states don't have an entrenched hereditary aristocracy We absolutely do, but because we are bombarded with propaganda telling us they "earned it" by "working hard" and "pulling themselves up by their bootstraps" we pretend it is just a co-incidence that the people born into the top 1% almost always remain there, and people born in the bottom 99% almost always remain there. >After public schooling became a things, suddenly just being able to read and write is not a distinguishing feature It never was. The aristocracy were educated specifically on economics, justice, etc. It wasn't simply a matter of being able to read.
  - fsloth 8y ago
    Sure, I simplified a lot. The current western world order is still much more fair to the common folk than feudalism. The education aspect was not as much about how much more educated the aristocracy was rather that the unwashed masses rose to their level, and it was not anymore sufficient to say that god has decreed duke X a distinguished gentleman and commoner Y a filthy peasant. It is not easy to rise social ladders, but it's way easier with our figment of a fair society, rather than if we believed social status was a fixed quantity given by the creator of the universe. The pithy armchair psychologist in me would claim modern society is much more facilitating towards a growth mindset than a feodal one.

notahacker 8y ago

There's a big unanswered question about whether "debate" is quite so useful at improving accuracy of answers when it consists of an audience with strong priors listening to motivated reasoning rather than people deferring to the people who are most confident they have a decent grasp of the subject of a neutral trivia question. I think it's conceivable the opposite effect might occur if the experiment were be to repeated using polarising political issues, even if the questions themselves were fact-based (economic growth rates, crime rates, immigration figures, global temperature changes etc).

wisty 8y ago
The backfire effect seems to be heavily exaggerated - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2819073 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2819073 And just think about how the research is done. "Hello conservative test subject. Here's a 1 page article explaining why Climate Change is a real problem". What impact do you think it will have? How about this. Dear liberal, belief in a just world is psychologically healthy, and belief in an unjust world is linked to mental health issues, where's a wikipedia link - https://en.wikipedia.org/wiki/Just-world_hypothesis#Current_research https://en.wikipedia.org/wiki/Just-world_hypothesis#Current_... It seems likely to me that a couple of pithy arguments are unlikely to change a core belief, but is very likely to create some short-term pushback. If they do change a core belief then I think it will have stronger and longer-lasting impacts than the push-back. I also suspect there's a delay - arguments might take a while to sink in.
patcon 8y ago
In the same experiment (but still unpublished) they actually find evidence that a similar CONSENSUS effect is observed in polarized issues with no clear "best" choice. https://www.ted.com/talks/mariano_sigman_and_dan_ariely_how_can_groups_make_good_decisions/transcript?language=en https://www.ted.com/talks/mariano_sigman_and_dan_ariely_how_... The most interesting bit to me is the fact that a rare middle position stakeholder, a "high confident gray", allows groups to reach consensus more often. This potentially has huge applications in system design of the processes of democracy imho

theontheone 8y ago

I think attributing the accuracy to debates is a rather hasty conclusion. I would love to see an experiment where the individual participants rated their confidence from 1 to 10, and only the highest confidence answer in each group was taken. My hunch is this would perform just as well (if not better) than post-discussion.

ajuc 8y ago

Am I correct that they asked the same people the same questions 3 times? It's no surprise that the 3rd time the answers were the best. I'd expect this to happen even if there was no debate (if not to the same degree). > Each participant was provided with pen and an answer sheet linked to their seat number. The event’s speaker (author M.S.) conducted the crowd from the stage (Fig. 1A). In the first stage of the experiment, the speaker asked eight questions (Supplementary Table 1) and gave participants 20 seconds to respond to each of them (stage i1, left panel in Fig. 1A). Then, participants were instructed to organize in to groups of five based on a numerical code in their answer sheet (see Methods). The speaker repeated four of the eight questions and gave each group one minute to reach a consensus (stage c, middle panel in Fig. 1A ). Finally, the eight questions were presented again from stage and participants had 20 seconds to write down their individual estimate, which gave them a chance to revise their opinions and change their minds (stage i2, right panel in Fig. 1A). Participants also reported their confidence in their individual responses in a scale from 0 to 10.

thenaturalist 8y ago
I can’t follow your reasoning. How does asking you the exact same question twice improve your accuracy? If you don’t know how high the Eiffel Tower is, why would you know it the second time? Conversely, you likely created a mental anchor when giving an estimate the first time and would be hard pressed to provide an answer contradicting your first estimate - even if it might be more accurate. There were no answers provided between asking the questions as far as I understand this excerpt. In the group only 4/8 questions were asked and here a difference in accuracy can be made. Maybe you’re mixed with travelers who recently visited Paris, historians or people who due to other circumstances or pure luck provide more accurate estimates, effectively influencing the anchor of your first estimate.
- ajuc 8y ago
  Given more time I can remember something, or I can notice that the last question gives some hints or even unrelated associations that help to answer the first question. I don't think "if you don't know the answer in 20 seconds you won't ever know it" is true.

ivanmaeder 8y ago

I've only read the summary but Philip Tetlock comes to this conclusion based on his work with "The Good Judgment Project" (described in his book "Superforecasting"). The GJP is a kind of experiment he's been running for a few years in an attempt to learn how to improve predictions. https://en.wikipedia.org/wiki/The_Good_Judgment_Project https://en.wikipedia.org/wiki/The_Good_Judgment_Project From what I remember: overall, teams did better than individuals and wisdom of the crowds because they were able to feed off and combine each other's points of view and separate knowledge. However! It's important for teams to not let groupthink dominate—individuals within teams needed to challenge each other.

amelius 8y ago

Perhaps we can save democracy by replacing the voting mechanism by placing people in groups of 10, and letting them reach consensus before making a vote.

aisengard 8y ago
I mean, this is essentially how the House of Representatives works.
deytempo 8y ago
You mean like trial by jury. I like it.
- mchannon 8y ago
  Trial by jury gets one set of 10 (12, whatever), whereas the consensus model described would have n sets of 10. I seriously doubt the wisdom of a crowd goes up when the total crowd size shrinks to 10. I suspect it plummets. This does call into question the notion of a multiple-jury trial (maybe videoconferenced in, where the individual juries do not interact with each other). Based on this study's findings, it would appear that juries might get their verdicts wrong less frequently with this system. Imagine if the first OJ trial or Trayvon Martin trial had been before multiple juries; would society have been better off if the results were less dependent on the biases of juries small enough to be cherry-picked?
  - sova 8y ago
    To summarize the findings of this research article for you: Wisdom of the Crowd vs one person = good Wisdom of the crowd vs tiny groups 4+ = bad
  - deytempo 8y ago
    In this case a jury is not really about everyone in it being wise, it’s about there being a better chance that an informed decision will take place due to those in the group discussing the decision before voting rather than just voting for whatever CNN recommends. With the current system you just have people doing whatever they are going to do without having to explain it to anyone else.

baud147258 8y ago

It reminds me of that assertion that the IQ of a crowd is the lowest IQ divided by the number of people in the crowd. Seems the study validate this with that smaller groups performs better

prewett 8y ago
Except that the previous study refuted that idea: the average answer was better than individual answers, which is definitely not IQ_crowd = IQ_min / n.

hkt 8y ago

Deliberative democracy is great. Put people in enriched decisionmaking environments for better outcomes, and then get greater democratic legitimacy from doing so than even some representatives? Win win. This is the opposite of populism too: when we voted Brexit in the UK, it was an uncontrolled, information-scarce (and falsehood-rich) environment. A room where people were organised and where they asked for their own experts and talked to one another would never have delivered that result.

sbhn 8y ago
People in the uk voted brexit, because it was just after the uk us and france were dropping bombs in lybia and syria, arab spring they called it, the destruction of infrastructure created millions of refugees, the bbc amplyied the importance and strength of isis, and flashed images of thousands of the refugees in calais tripping over each other trying to illegally cross the channel into the uk. The good souls of england thought they were under attack, bless em.

sova 8y ago

The line to know: "Remarkably, combining as few as four consensus choices outperformed the wisdom of thousands of individuals." Confer away!

patcon 8y ago

This TED video tuned me into this research earlier in 2018, and while not rigorously tested, there are some even more interesting hypotheses that are being sussed out of the experiment: https://www.ted.com/talks/mariano_sigman_and_dan_ariely_how_can_groups_make_good_decisions/transcript?language=en https://www.ted.com/talks/mariano_sigman_and_dan_ariely_how_...

ebetica0 8y ago

I wonder how much statistical independence has to play in this. e.g. the large crowd can be biased when they influence each other in a way that the small debates cannot. What happens when you have small independent debates versus large independent crowds?

PaulHoule 8y ago

We had poker planning introduced to my team and within two months or so we got to the point where most of the time we all get the same number or get something like 8-13-21.

reggieband 8y ago

Whenever I come across such ideas it makes me want to more deeply investigate the Communicative Rationality [1] of Jurgen Habermas. Despite modern trends towards a conservative stoicism (e.g. Jordan Peterson) I actually think Social Critical Theory feels to be moving in the more-correct direction. However, the current negative association with moral-relativism and postmodernism is a new McCarthyism. The bridge should be pragmatism but even that is too closely associated with socialism. Too many isms but the ones that are scaring me most now are authoritarianism, fascism and totalitarianism. Ideas like communicative rationality feel to me like the most reasonable solutions. [1] https://en.wikipedia.org/wiki/Communicative_rationality https://en.wikipedia.org/wiki/Communicative_rationality [2] https://en.wikipedia.org/wiki/Critical_theory https://en.wikipedia.org/wiki/Critical_theory