12 ms·

A first lesson in meta-rationality

159 points by alboaie 5y ago

alboaie 5y ago
As people and societies we evolve towards metarationality. The reality is too complex to be handled within a single theory. We have to develop metarationality (wisdom) and pragamaticaly make jumps between contradictory theories. Any other aproach is doomed because the complexity will be always too much for any "single rationality" approach.
abss 5y ago
Another good introduction: https://drossbucket.com/2017/09/30/metarationality-a-messy-introduction/ https://drossbucket.com/2017/09/30/metarationality-a-messy-i...
- alboaie 5y ago
  Despite admitting verbally that a map is not the territory, rationalists hope that if they take one map, and keep updating it long enough, this map will asymptotically approach the territory. In other words, that in every moment, using one map is the right strategy. Meta-rationalists don’t believe in the ability to update one map sufficiently (or perhaps just sufficiently quickly), and intentionally use different maps for different contexts. (Which of course does not prevent them from updating the individual maps.) As a side effect of this strategy, the meta-rationalist is always aware that the currently used map is just a map; one of many possible maps. The rationalist, having invested too much time and energy into updating one map, may find it emotionally too difficult to admit that the map does not fit the territory, when they encounter a new part of territory where the existing map fits poorly. Which means that on the emotional level, rationalists treat their one map as the territory. Furthermore, meta-rationalists don’t really believe that if you take one map and keep updating it long enough, you will necessarily asymptotically approach the territory. First, the incoming information is already interpreted by the map in use; second, the instructions for updating are themselves contained in the map. So it is quite possible that different maps, even after updating on tons of data from the territory, would still converge towards different attractors. And even if, hypothetically, given infinite computing power, they would converge towards the same place, it is still possible that they will not come sufficiently close during one human life, or that a sufficiently advanced map would fit into a human brain. Therefore, using multiple maps may be the optimal approach for a human. (Even if you choose “the current scientific knowledge” as one of your starting maps.)
  - abss 5y ago
    Yes.. metarationality is basicaly a sort of belief system about how and what we can undestand. If we dont undestand our limits and we do not use rationalities as tools we end in scientism or very wrong belief systems built on "fakery" and self delusions..
    - beaconstudios 5y ago
      FYI, it is a synonym for the epistemology (philosophy of knowledge) called constructivism.
  - d0mine 5y ago
    I’ve heard about this idea in a form: «All models are false. Some are useful.»
  - morsch 5y ago
    The above comment should be attributed to https://www.lesswrong.com/posts/hxxN75ZQ5GY4Tjwkv/inscrutable-ideas?commentId=DKCc9CDawqcG4aqcm https://www.lesswrong.com/posts/hxxN75ZQ5GY4Tjwkv/inscrutabl...
  - Retric 5y ago
    That line of thinking assumes you can’t share maps, as such it’s of minimal practical value. To abuse the analogy, a glovebox of old road atlas doesn’t beat Google maps. On the other hand mixing Google maps with your personal knowledge that a bridge is out is a useful meta map.
  - roenxi 5y ago
    That comment is picking at a fair critique, but some details seem to be wrong. > rationalists hope that if they take one map, and keep updating it long enough, this map will asymptotically approach the territory That is, as far as can be detected, what the human brain does. It isn't just the rationalists who have a view and keep updating it, hoping it will asymptotically approach the territory. It is exceedingly difficult to have a strategy that doesn't do that and still be a semi-functional member of society. I'm struggling to see how someone could hold 'different' maps because they become one map in your head. Rationalists are perfectly comfortable with there being multiple possible scenarios leading to an outcome. My guess is that this observation is going to the fact that rationalists are very, very uncomfortable (to the point of falling apart, sometimes) in accepting "because I say so" as sufficient evidence to update a view, change behaviour stop arguing and be a good sport about the whole thing. Which is very much a social faux-pas when dealing with high status people and often a mistake when dealing with inarticulate people who are nevertheless correct in their view.
tlb 5y ago
An interesting category of problems are like Bongard problems in that you have to deduce the rule from examples, but the examples are presented one at a time at random long intervals so you have to work from memory. Most real-world learning is like this. When working from memory, it's normal for your memory to have already parsed the previous situation into features. As some of the later examples in the blog illustrate, it's easy to fall into parsing examples into the wrong set of features, which is how you'll remember them. While I could solve all the problems in the article, I doubt I could solve any but the simplest if I was shown 1 image per day over 12 days and not allowed to write anything down. Perhaps the lesson is that when you're trying to deduce a rule (say, for what conditions your software crashes in) you can increase your rule-discovering power greatly by making notes and being able to look at several examples side-by-side.
- rrmm 5y ago
  You also have to know you want to solve a problem. Once you get to the point where you have any hypothesis whatsoever, no matter how weak, a systematic approach (saving examples as test cases) helps to avoid confirmation bias and makes testing further hypothesis less costly. Another hard one is when there is a simple, probabilistic rule. You usually end up with an over-complicated rule to cover all your data instead of the true rule. (Of course that gets down to what is at the basis of the probability: are you satisfied with a probability?)
  - skybrian 5y ago
    Probabilistic rules themselves tend to require much more data, which can be expensive. In computing we try to write deterministic tests that either pass or fail, which means you can run them once after a change and know what the state is. Even if you just suspect flakiness you may have to run the test hundreds of times to be confident that the probability of failure is sufficiently low.
- lisper 5y ago
  This is exactly right, and it is exactly what makes quantum mechanics and relativity so hard to wrap your brain around because by the time you get around to learning them you have almost certainly deeply internalized a classical model of the world. It's just obvious that classical mechanics is "correct", that the world consists of objects embedded in a three-dimensional space that exist in specific places at specific times, and even talking about a world where this is not true doesn't even make sense, let alone qualify as a viable candidate for actual truth. It is equally "obvious" that the heavens are governed by different laws of physics than the earth, because things on earth fall down if unsupported and naturally come to rest and things in the heavens don't. And of course all of these things are equally wrong. One can and should apply the same lesson to social and political statements. For example, people get hung up on arguing about things like whether or not "God exists" as if they were arguing about a question of objective fact when actually what they are arguing about is the meaning of the words "God" and "exists." I wrote a longer take on all this about six years ago: http://blog.rongarret.info/2015/02/31-flavors-of-ontology.html http://blog.rongarret.info/2015/02/31-flavors-of-ontology.ht...
yewenjie 5y ago
David Chapman has multiple sites for approaching the same problem. However, all the sites (or 'books') are incomplete. Every time I come across something by him I just want to read a complete book on the topic of metarationality from cover to cover.
- Matt_Arnold 5y ago
  Yeah, as part of my free podcast audiobook version of Meaningness, I'm constantly asking him for updates, in order to provide a structure in which he'll work on one book at a time and completely finish it. So far it's working out pretty well. https://fluidity.libsyn.com/ https://fluidity.libsyn.com/
- skybrian 5y ago
  I’d settle for more progress on In the Cells of the Eggplant. Oddly, this page is on that site but not linked from the table of contents, nor does it show up as a recently changed page.
- timboy03 5y ago
  Chapman's long-term hidden-agenda project is to teach us all by extended example that the notion of a "book" is intrinsically nebulous, and that the decision of whether and where one of his books stops and another begins (necessary for, say, counting how many books he has written) is itself dependent on the purpose you have for reading his books.
mckirk 5y ago
Am I the only one that gets driven kind of crazy by these kinds of problems? I'm not completely sure so far what it is, but I'm guessing it's the frustration of having to find a needle in a haystack of essentially infinite size, as depending on how complicated you want to see the problem, there's an infinitude of potential 'solutions' and you never really know which level of complexity the author had in mind. I love logic puzzles, where the system is constrained and you have to work within it, but these find-the-rule problems really aren't my thing so far. Maybe I'd need to develop a higher frustration tolerance for them, heh.
- narag 5y ago
  I don't know, the first took me two seconds, the second five and the third five too. I stopped reading there because the article says next ones are harder and I would like to take a quiet time to read the whole thing. I use to solve chess problems at lichess, a similar concept. Maybe. Is it really an infinite haystack? It's context. And simplest solutions first, gradually think of more complicated ones. First try to find a pattern visually, then use simple concepts, then more complex concepts.
- solveit 5y ago
  You're certainly not the only one, but the point of the article is that solving these kinds of problems, with all the vagueness that implies, is an important feature of human intelligence.
  - mckirk 5y ago
    Since I generally seem to do fairly well in problem solving, that is exactly what I'm trying to figure out: Are these problems actually representative of an important skill that you need for general problem solving, or are they, through their nature of being man-made puzzles, actually in a realm of their own? When I'm looking at some pattern that I'm trying to find a rule for in real-life, I don't think I'm running into the same frustration and in fact greatly enjoy trying to figure out rules for how things work (or so I believe, at least). I think a crucial difference is that I know that the problems I encounter in real-life are only "as complex as necessary", and the data I'm looking at is a direct result of some process that serves a specific goal; presumably one I think "makes sense", as I wouldn't look for a rule otherwise. In contrast, puzzles are made to be complicated on purpose, and I suspect that annoys me subconsciously to the point where my brain complains about engaging with it. But it's only these kinds of "figure out the rules" puzzles, so there has to be another important difference compared to logic puzzles. Possibly the difference is: for the logic puzzle, the "meta-rules" for the problem are made explicit and I know the solution-space exactly. For the Bongard problems here I found myself thinking for example: "wait, is it always just two groups distinguished by single rule, or can there be dependencies on the positions of the symbols within the groups as well? What kind of solution am I even looking for?", and that also apparently frustrates me. Sorry for the wall of text, but I've actually been trying to figure out why these kinds of problems get on my nerves for quite a long time, lol.
OisinMoran 5y ago
This was a good read and I really enjoyed it (I'm another person who was turned onto Bongard problems by Hofstadter), but two parts weren't particularly strong. The first one was the dismissal of intuition in a way that seemed pretty straw man like to me: "Mostly, “intuition” just means “mental activity we don’t have a good explanation for,” or maybe “mental activity we don’t have conscious access to.” It is a useless concept, because we don’t have good explanation for much if any mental activity, nor conscious access to much of it. By these definitions, nearly everything is “intuition,” so it’s not a meaningful category." I think the author could have spent longer trying to come up with a better definition of what someone would mean by intuition with relation to these problems instead of just setting up a poor one then immediately tearing it down. Intuition here would be contrasted against the deliberate procedural thinking of "let's list out qualities of these shapes" and would be something like seeing the solution straight away, but can also be combined with the procedural thinking too with the intuition originating possible useful avenues and then the deliberate part working through them. The contrast is that you could easily write down one set of the steps to be replicated by others (the deliberate part: "I counted the sides on all shapes") but less so the other (intuition: "I thought x", "x jumped out"). The second is that the example they use for mushiness really isn't. There is a perfectly concrete solution to that that doesn't involve any mushiness and is simply that the convex hull of one set is triangular while the others are circular. The only mushiness involved is that saying "triangles vs circles" feels like enough of answer to us to not need to specify any more. We think that we can continue with just this answer and be able to correctly identify any future instances so it seems mushy but you can probably think of examples that would confound the mushy solution but be fine under the more concrete convex hull one.
- teknopaul 5y ago
  I thought that one was the most interesting too. The convex hull appears circular but it is not, in one case you have to join the dots between the center point of triangles, none of the points on the triangles are on a circle, or maybe all of them are if we discard the hull abstraction, in which case there are three circles. Imagine trying to write code that identifies that. However it's one of the most obvious to me.
mikhailfranco 5y ago
For the third I got: triangle never in circle - circle never in triangle compared to the given answer: triangle bigger than circle - circle bigger than triangle My solution is more general (worse), because it ignores size in non-containment arrangements, but also slightly more specific (better), because it constrains the single containment example in each set. Neither of the rules say anything about overlapping cases, but there are no overlapping examples in the given sets. So there is a underlying constraint of no overlaps, but it applies to both sides, so it is not a distinguishing factor.
- laszlokorte 5y ago
  The issue with your solution is that it can not decide for a single combination of circle and triangle if it would belong to the left or the right side.
- EE84M3i 5y ago
  >The contents of the six boxes on the left all have something in common. The six on right also all have something in common, which is the opposite of the ones on the left. I think you and the author might disagree on the meaning of "opposite" here. I think they mean logical negation and you are using a more colloquial interpretation.
- teknopaul 5y ago
  There is no correct answer (see nebulosity) , the point is to learn how the problem works. And what a marvelous thing your brain is by being able to come up with any solution.
- CRConrad 5y ago
  The idea is to arrive at a rule you can use for any given picture, to decide if it goes on the left or right: > In a Bongard problem, you have to figure out what the rule is. You are given twelve specific images, and the result of applying the rule to each. (The rule assigns an image to either the left or right group.) Once you have discovered the rule, applying it to new images would be trivial. Your rules don't do that. Most of the pictures have neither "triangle in circle" nor "circle in triangle", so your rules don't apply to... Ten out of the twelve pictures.
yamrzou 5y ago
Very Interesting. This reminds me of The Abstraction and Reasoning Corpus [1] by François Chollet, accompanying his paper On the Measure of Intelligence [2]... Edit: Found a recent article mentioning both and discussing a NeurIPS paper on using Bongard problems to test AI systems [3]. [1] https://github.com/fchollet/ARC https://github.com/fchollet/ARC [2] https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.01547 [3] https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/how-do-you-test-the-iq-of-ai https://spectrum.ieee.org/tech-talk/artificial-intelligence/...
brudgers 5y ago
I am reminded that the simplest regex for the words "apex, ibex, index" is 'apex|ibex|index'. A commonality of all the boxes on the right is being on the right. A commonality of all the boxes on the left is being on the left. There is no offside in golf. The rules of the game only apply if when we are playing the game. Here, Wittgenstein might have said Bongard problems are another language game and the confusion arises from using words in a peculiar way...the game is pretending there is a problem in a Bongard problem.
- Gys 5y ago
  ‘There is no spoon’
- EricBurnett 5y ago
  It depends how you define "simple" : '(ap|ib|ind)ex' is fewer symbols and a smaller character graph. There is no offside in golf...yet. But the rules are surprisingly long, and will only get longer as loopholes are found and exploited. Humans trying to codify a game, or any system, cannot do so precisely; others can and will find the gaps between the written rules and the intended meaning, and play a different game but pretend it's the same. Or I guess, every game is a language game? Which I think is a point of meta rationality: we can't avoid including questions of the rules of the problem. It's clouds all the way down.
  - brudgers 5y ago
    ‘*’ is fewer characters. If we are playing that language game.
    - handoflixue 5y ago
      > ‘*’ is fewer characters. yes, but that produces false positives, whereas both 'apex|ibex|index' and '(ap|ib|ind)ex' do not. Your original claim was that 'apex|ibex|index' was "simplest", which means it has some property you think that '(ap|ib|ind)ex' lacks, but you haven't yet articulated what that IS.
chubot 5y ago
From the Church-Turing Thesis, we know there’s nothing special going on! We know humans can’t do anything more than a computer can. I see people making such claims about human cognition all the time, and I have no idea how it follows. (note the author is paraphrasing "people" here) The Church-Turing Thesis says nothing about human cognition. It is perfectly plausible that a human can do things a computer can't. (Scott Aaronson has a paper "Why Philosophers Should Care About Computational Complexity" which sheds some light on why that might be, but it's far from the only possible reason.) The burden of proof is on people who claim that human cognition can be simulated by computer, not the other way around. To me, it seems far more likely that it can't. Human cognition can obviously be simulated by "the laws of physics", since brains are material, but it seems very likely that computers are less powerful than that. That's my refutation of the (silly IMO) "simulation argument". I'd argue it's simply not possible to simulate another universe. You can simulate something like SimCity or whatever, but not a real universe. The people who make that argument always seem to leave out the possibility that it's physically impossible. In fact I would actually take the simulation argument ("we are almost certainly living in a simulation") as proof by contradiction that simulation is impossible.
- throwamon 5y ago
  > That's my refutation of the (silly IMO) "simulation argument" I'm not one of these people, but your rebuttal wouldn't convince me if I were. Maybe we are the SimCity of a much more complex universe.
  - chubot 5y ago
    OK, but I'm saying the burden of proof is on who think that simulation is possible. It's not at all obvious that it is. As far as I can tell, the idea was basically made popular by movies [1], and there is no science behind it. All the science I know of points the other way -- simulating anything is incredibly hard and slow. It requires approximations and shortcuts to make it work for specific cases, and it doesn't work in the general case. (Maybe some alternative model like quantum computation will be different, but we're MUCH much further there. I think "adding two small numbers" is still an issue for state of the art quantum computers.) ----- Here's a nice example from a few days ago, trying to represent even a tiny part of the human brain in a computer: https://news.ycombinator.com/item?id=27362883 https://news.ycombinator.com/item?id=27362883 [1] I think this is more literally true than you might expect; IIRC the published papers in philosophy liberally reference The Matrix, maybe because it attracts readers.
hamilyon2 5y ago
Author links to Turing-Church thesis, which is widely assumed to be right to talk about things not related to any single thing that thesis is about. It is about computable functions and abstract machines.
hliyan 5y ago
I've always considered "pattern recognition for the purpose of prediction" a core function of the human brain that precedes language, rationality or any form of logic. So it feels counter-intuitive to me to label this meta-rationality or even associate it with rationality. I subscribe somewhat to the idea that much of our rational decision making is ex post facto, i.e a verbal narration that comes after the decision making to explain the decision. The actual decision making being an opaque process that takes place inside our brain's neural network. Confession: I lost the author about halfway through the article.
- diab0lic 5y ago
  This ex post facto explanation is referred to as rationalization and is ironically irrational. If you’ve already made a decision you’ve forfeited the option to do so rationally. I do agree that most of what gets labeled rational is in fact ex post facto rationalization. I do it myself all the time, haha.
  - rrmm 5y ago
    Ex post facto rationalization is a major annoyance of mine. Let's just be honest with ourselves and recognize that lots of what we want isn't rational. That's ok, use your rationality to keep yourself from making huge, dumb mistakes, but don't pretend you're doing it for "rational" reasons or through a "rational" process. But I'm kind of over-strict about it and like to reserve "rational" for processes you can do with first-order logic. Given the complexity and lack of information in real life, that happens almost never. Of course this is not a terribly rational point of view.
pontifier 5y ago
I have recently come up with a model that has been useful to me for thinking about thinking. It involves the realization that different brain regions must communicate, but also contain their own representation of reality. Partial thought precursors echo back and forth between these regions, with each region amplifying or dampening parts of the idea that it recognizes as valid. When multiple brain regions begin to agree on its validity to a high level, the aha moment occurs. This model has some characteristics of waveform collapse, and discrete task specific neural networks. When multiple tasks specific networks arrive at consensus that a model matches experience, the proto-idea forms. This proto-idea can then be evaluated, and inspected. New scenarios are reflected off this new idea, to see if it continues to make sense. Converting an idea into words makes it useful to others, and allows sharing of ideas. This process requires refinement by echoing back and forth with the proto-idea until the words match it's shape. In order for these words to be understood effectively, they need to make sense to the brains that are receiving them. That means the words chosen need to activate multiple brain regions that the listener may use to evaluate this new idea and have the aha moment themselves. This process is easier when the 2 brains have many shared experiences to draw on, or communication is bi-directional to allow message refinement.
- webmaven 5y ago
  > It involves the realization that different brain regions must communicate, but also contain their own representation of reality. Sure, but this is complicated by regions being fuzzy and overlapping (or perhaps interpenetrating).
  - pontifier 5y ago
    Each part in a healthy brain should ideally take input, process it somehow, and reflect back an echo that amplifies and dampens the parts that match it's representation of reality. The lines between regions aren't as important. I'm almost thinking about how this could be turned into a device for conveying information with actual echos.
    - webmaven 5y ago
zeroonetwothree 5y ago
I don’t see how Bongard problems are human complete if the author can’t even solve half of them. Does that mean he doesn’t have human intelligence? I think a better candidate for human complete is “knowing what other humans are thinking”. AKA “theory of mind”.
- Viliam1234 5y ago
  That would exclude many neurodivergent people. Perhaps also other cultures, and social classes. It is easier to guess the state of minds of people who are similar to you. Because then your natural algorithm "what would I feel in such situation? what would make me say these words, act this way?" is more likely to match how they feel and think. I suspect that many people overestimate their ability to "read other people's minds". First, they rarely verify their guesses. (I see another person and conclude that they are angry. I usually don't approach them and ask "hey, are you angry?". Therefore, if my guess was wrong, I am not going to learn it.) Second, if they turn out to be wrong, it's always the other person's fault. (If an autist cannot guess a neurotypical person's thoughts, it's the autist's fault. If a neurotypical person cannot guess an autist's thoughts, guess what, that's also the autist's fault.) Third, it is easier to guess thoughts of people we frequently talk to, because people usually think today the same thing they thought yesterday, and they already told us what they thought yesterday.
- skybrian 5y ago
  Being able to solve half of them by computer would be amazing progress, indicating a breakthrough in machine learning research. But whether that means artificial general intelligence is solved is another question. Most people can’t play Go very well either. It’s difficult to say whether the solution will generalize without having it, but easy to imagine that it might.
wydfre 5y ago
If anyone is interested in this, they should look into Alfred Korzybski and general semantics. He invented the term "The map is not the territory", in case you want an idea of who he is. IIRC, he at one point says in his book, "Science and Sanity: An Introduction to Non-Aristotelian Systems and General Semantics" something along the lines that the mistakes most people make are in categorization. Something along the lines of "Some things look the same but they are different, and some things look different but they are the same". It's a very interesting book and I loved how Non-Aristotelian logic was used in Null-A by A.E. van Vogt, which introduced it to me.
slx26 5y ago
We should keep one morning of school a week, no matter our age, to get together with other humans and discuss about stuff like this. Individual reading and HN comments are fine, but we are missing the best part of learning, which is actually verbalizing and battle-testing those vague ideas. It would really help in cases like this... Have no friends? Make a club!
cousin_it 5y ago
A site with lots of Bongard problems: https://www.foundalis.com/res/bps/bpidx.htm https://www.foundalis.com/res/bps/bpidx.htm
tomcatfish 5y ago
> However, by “system” I mean, roughly, a set of rules that can be printed in a book weighing less than ten kilograms, and which a person can consciously follow. Is this tongue in cheek or is it a very strange thing to say? That's definitely not a statement that should be made with no justification. I can guess what the author meant, but I don't really want to guess at basics of their argument.
- tlb 5y ago
  It's important to exclude infinite books, which could just contain a lookup table for every possible situation rather than any general rules. And also finite but exponentially large books, which could contain a lookup table for every possible image (and be something like 2^(1024x768x24) pages long.)
raghuveerdotnet 5y ago
I still don’t understand how this is any different from the notion of problem solving. And I really didn’t understand the human-AI equivalence, is it just the substrate independent nature of turing-completeness? If so, I think we still lack the epistemic toolkit to say anything conclusive about creating an equivalence, or for that matter any form of comparative relationship, especially given that AGI is still not a problem that is well-defined, let alone discussing the solution space. No?
sonkol 5y ago
I know this is a little bit out of topic but I think meta-rationality is more about organizing other people/machines intelligences to achieve your goal even when you are not highly intelligent.
timboy03 5y ago
One tricky thing about Bongard problems is that for any given problem there are likely many different rules that could distinguish the six positive examples from the six negative examples. For example, maybe a problem that is "really" about circles vs. triangles also happens to have more black pixels in the left images than in the right images. A key skill in solving these problems is not just to find a compact and discriminating description, but to find such a description that is also one that a human Bongard problem designer would be likely to think was a cool and elegant puzzle that needs an "Aha" moment to recognize. If you find such a description, then you're very likely to be right. I suspect that that last part (recognizing when you have found a solution that is pleasing enough to be the answer) is likely to be the biggest challenge for ML-based approaches to Bongard problems
lorepieri 5y ago
These problems remind me of https://github.com/fchollet/ARC https://github.com/fchollet/ARC "ARC can be seen as a general artificial intelligence benchmark, as a program synthesis benchmark, or as a psychometric intelligence test. It is targeted at both humans and artificially intelligent systems that aim at emulating a human-like form of general fluid intelligence."

159 points by alboaie 5y ago

As people and societies we evolve towards metarationality. The reality is too complex to be handled within a single theory. We have to develop metarationality (wisdom) and pragamaticaly make jumps between contradictory theories. Any other aproach is doomed because the complexity will be always too much for any "single rationality" approach.

abss 5y ago

Another good introduction: https://drossbucket.com/2017/09/30/metarationality-a-messy-introduction/ https://drossbucket.com/2017/09/30/metarationality-a-messy-i...

alboaie 5y ago
Despite admitting verbally that a map is not the territory, rationalists hope that if they take one map, and keep updating it long enough, this map will asymptotically approach the territory. In other words, that in every moment, using one map is the right strategy. Meta-rationalists don’t believe in the ability to update one map sufficiently (or perhaps just sufficiently quickly), and intentionally use different maps for different contexts. (Which of course does not prevent them from updating the individual maps.) As a side effect of this strategy, the meta-rationalist is always aware that the currently used map is just a map; one of many possible maps. The rationalist, having invested too much time and energy into updating one map, may find it emotionally too difficult to admit that the map does not fit the territory, when they encounter a new part of territory where the existing map fits poorly. Which means that on the emotional level, rationalists treat their one map as the territory. Furthermore, meta-rationalists don’t really believe that if you take one map and keep updating it long enough, you will necessarily asymptotically approach the territory. First, the incoming information is already interpreted by the map in use; second, the instructions for updating are themselves contained in the map. So it is quite possible that different maps, even after updating on tons of data from the territory, would still converge towards different attractors. And even if, hypothetically, given infinite computing power, they would converge towards the same place, it is still possible that they will not come sufficiently close during one human life, or that a sufficiently advanced map would fit into a human brain. Therefore, using multiple maps may be the optimal approach for a human. (Even if you choose “the current scientific knowledge” as one of your starting maps.)
- abss 5y ago
  Yes.. metarationality is basicaly a sort of belief system about how and what we can undestand. If we dont undestand our limits and we do not use rationalities as tools we end in scientism or very wrong belief systems built on "fakery" and self delusions..
  - beaconstudios 5y ago
    FYI, it is a synonym for the epistemology (philosophy of knowledge) called constructivism.
- d0mine 5y ago
  I’ve heard about this idea in a form: «All models are false. Some are useful.»
- morsch 5y ago
  The above comment should be attributed to https://www.lesswrong.com/posts/hxxN75ZQ5GY4Tjwkv/inscrutable-ideas?commentId=DKCc9CDawqcG4aqcm https://www.lesswrong.com/posts/hxxN75ZQ5GY4Tjwkv/inscrutabl...
- Retric 5y ago
  That line of thinking assumes you can’t share maps, as such it’s of minimal practical value. To abuse the analogy, a glovebox of old road atlas doesn’t beat Google maps. On the other hand mixing Google maps with your personal knowledge that a bridge is out is a useful meta map.
- roenxi 5y ago
  That comment is picking at a fair critique, but some details seem to be wrong. > rationalists hope that if they take one map, and keep updating it long enough, this map will asymptotically approach the territory That is, as far as can be detected, what the human brain does. It isn't just the rationalists who have a view and keep updating it, hoping it will asymptotically approach the territory. It is exceedingly difficult to have a strategy that doesn't do that and still be a semi-functional member of society. I'm struggling to see how someone could hold 'different' maps because they become one map in your head. Rationalists are perfectly comfortable with there being multiple possible scenarios leading to an outcome. My guess is that this observation is going to the fact that rationalists are very, very uncomfortable (to the point of falling apart, sometimes) in accepting "because I say so" as sufficient evidence to update a view, change behaviour stop arguing and be a good sport about the whole thing. Which is very much a social faux-pas when dealing with high status people and often a mistake when dealing with inarticulate people who are nevertheless correct in their view.

tlb 5y ago

An interesting category of problems are like Bongard problems in that you have to deduce the rule from examples, but the examples are presented one at a time at random long intervals so you have to work from memory. Most real-world learning is like this. When working from memory, it's normal for your memory to have already parsed the previous situation into features. As some of the later examples in the blog illustrate, it's easy to fall into parsing examples into the wrong set of features, which is how you'll remember them. While I could solve all the problems in the article, I doubt I could solve any but the simplest if I was shown 1 image per day over 12 days and not allowed to write anything down. Perhaps the lesson is that when you're trying to deduce a rule (say, for what conditions your software crashes in) you can increase your rule-discovering power greatly by making notes and being able to look at several examples side-by-side.

rrmm 5y ago
You also have to know you want to solve a problem. Once you get to the point where you have any hypothesis whatsoever, no matter how weak, a systematic approach (saving examples as test cases) helps to avoid confirmation bias and makes testing further hypothesis less costly. Another hard one is when there is a simple, probabilistic rule. You usually end up with an over-complicated rule to cover all your data instead of the true rule. (Of course that gets down to what is at the basis of the probability: are you satisfied with a probability?)
- skybrian 5y ago
  Probabilistic rules themselves tend to require much more data, which can be expensive. In computing we try to write deterministic tests that either pass or fail, which means you can run them once after a change and know what the state is. Even if you just suspect flakiness you may have to run the test hundreds of times to be confident that the probability of failure is sufficiently low.
lisper 5y ago
This is exactly right, and it is exactly what makes quantum mechanics and relativity so hard to wrap your brain around because by the time you get around to learning them you have almost certainly deeply internalized a classical model of the world. It's just obvious that classical mechanics is "correct", that the world consists of objects embedded in a three-dimensional space that exist in specific places at specific times, and even talking about a world where this is not true doesn't even make sense, let alone qualify as a viable candidate for actual truth. It is equally "obvious" that the heavens are governed by different laws of physics than the earth, because things on earth fall down if unsupported and naturally come to rest and things in the heavens don't. And of course all of these things are equally wrong. One can and should apply the same lesson to social and political statements. For example, people get hung up on arguing about things like whether or not "God exists" as if they were arguing about a question of objective fact when actually what they are arguing about is the meaning of the words "God" and "exists." I wrote a longer take on all this about six years ago: http://blog.rongarret.info/2015/02/31-flavors-of-ontology.html http://blog.rongarret.info/2015/02/31-flavors-of-ontology.ht...

yewenjie 5y ago

David Chapman has multiple sites for approaching the same problem. However, all the sites (or 'books') are incomplete. Every time I come across something by him I just want to read a complete book on the topic of metarationality from cover to cover.

Matt_Arnold 5y ago
Yeah, as part of my free podcast audiobook version of Meaningness, I'm constantly asking him for updates, in order to provide a structure in which he'll work on one book at a time and completely finish it. So far it's working out pretty well. https://fluidity.libsyn.com/ https://fluidity.libsyn.com/
skybrian 5y ago
I’d settle for more progress on In the Cells of the Eggplant. Oddly, this page is on that site but not linked from the table of contents, nor does it show up as a recently changed page.
timboy03 5y ago
Chapman's long-term hidden-agenda project is to teach us all by extended example that the notion of a "book" is intrinsically nebulous, and that the decision of whether and where one of his books stops and another begins (necessary for, say, counting how many books he has written) is itself dependent on the purpose you have for reading his books.

mckirk 5y ago

Am I the only one that gets driven kind of crazy by these kinds of problems? I'm not completely sure so far what it is, but I'm guessing it's the frustration of having to find a needle in a haystack of essentially infinite size, as depending on how complicated you want to see the problem, there's an infinitude of potential 'solutions' and you never really know which level of complexity the author had in mind. I love logic puzzles, where the system is constrained and you have to work within it, but these find-the-rule problems really aren't my thing so far. Maybe I'd need to develop a higher frustration tolerance for them, heh.

narag 5y ago
I don't know, the first took me two seconds, the second five and the third five too. I stopped reading there because the article says next ones are harder and I would like to take a quiet time to read the whole thing. I use to solve chess problems at lichess, a similar concept. Maybe. Is it really an infinite haystack? It's context. And simplest solutions first, gradually think of more complicated ones. First try to find a pattern visually, then use simple concepts, then more complex concepts.
solveit 5y ago
You're certainly not the only one, but the point of the article is that solving these kinds of problems, with all the vagueness that implies, is an important feature of human intelligence.
- mckirk 5y ago
  Since I generally seem to do fairly well in problem solving, that is exactly what I'm trying to figure out: Are these problems actually representative of an important skill that you need for general problem solving, or are they, through their nature of being man-made puzzles, actually in a realm of their own? When I'm looking at some pattern that I'm trying to find a rule for in real-life, I don't think I'm running into the same frustration and in fact greatly enjoy trying to figure out rules for how things work (or so I believe, at least). I think a crucial difference is that I know that the problems I encounter in real-life are only "as complex as necessary", and the data I'm looking at is a direct result of some process that serves a specific goal; presumably one I think "makes sense", as I wouldn't look for a rule otherwise. In contrast, puzzles are made to be complicated on purpose, and I suspect that annoys me subconsciously to the point where my brain complains about engaging with it. But it's only these kinds of "figure out the rules" puzzles, so there has to be another important difference compared to logic puzzles. Possibly the difference is: for the logic puzzle, the "meta-rules" for the problem are made explicit and I know the solution-space exactly. For the Bongard problems here I found myself thinking for example: "wait, is it always just two groups distinguished by single rule, or can there be dependencies on the positions of the symbols within the groups as well? What kind of solution am I even looking for?", and that also apparently frustrates me. Sorry for the wall of text, but I've actually been trying to figure out why these kinds of problems get on my nerves for quite a long time, lol.

OisinMoran 5y ago

This was a good read and I really enjoyed it (I'm another person who was turned onto Bongard problems by Hofstadter), but two parts weren't particularly strong. The first one was the dismissal of intuition in a way that seemed pretty straw man like to me: "Mostly, “intuition” just means “mental activity we don’t have a good explanation for,” or maybe “mental activity we don’t have conscious access to.” It is a useless concept, because we don’t have good explanation for much if any mental activity, nor conscious access to much of it. By these definitions, nearly everything is “intuition,” so it’s not a meaningful category." I think the author could have spent longer trying to come up with a better definition of what someone would mean by intuition with relation to these problems instead of just setting up a poor one then immediately tearing it down. Intuition here would be contrasted against the deliberate procedural thinking of "let's list out qualities of these shapes" and would be something like seeing the solution straight away, but can also be combined with the procedural thinking too with the intuition originating possible useful avenues and then the deliberate part working through them. The contrast is that you could easily write down one set of the steps to be replicated by others (the deliberate part: "I counted the sides on all shapes") but less so the other (intuition: "I thought x", "x jumped out"). The second is that the example they use for mushiness really isn't. There is a perfectly concrete solution to that that doesn't involve any mushiness and is simply that the convex hull of one set is triangular while the others are circular. The only mushiness involved is that saying "triangles vs circles" feels like enough of answer to us to not need to specify any more. We think that we can continue with just this answer and be able to correctly identify any future instances so it seems mushy but you can probably think of examples that would confound the mushy solution but be fine under the more concrete convex hull one.

teknopaul 5y ago
I thought that one was the most interesting too. The convex hull appears circular but it is not, in one case you have to join the dots between the center point of triangles, none of the points on the triangles are on a circle, or maybe all of them are if we discard the hull abstraction, in which case there are three circles. Imagine trying to write code that identifies that. However it's one of the most obvious to me.

mikhailfranco 5y ago

For the third I got: triangle never in circle - circle never in triangle compared to the given answer: triangle bigger than circle - circle bigger than triangle My solution is more general (worse), because it ignores size in non-containment arrangements, but also slightly more specific (better), because it constrains the single containment example in each set. Neither of the rules say anything about overlapping cases, but there are no overlapping examples in the given sets. So there is a underlying constraint of no overlaps, but it applies to both sides, so it is not a distinguishing factor.

laszlokorte 5y ago
The issue with your solution is that it can not decide for a single combination of circle and triangle if it would belong to the left or the right side.
EE84M3i 5y ago
>The contents of the six boxes on the left all have something in common. The six on right also all have something in common, which is the opposite of the ones on the left. I think you and the author might disagree on the meaning of "opposite" here. I think they mean logical negation and you are using a more colloquial interpretation.
teknopaul 5y ago
There is no correct answer (see nebulosity) , the point is to learn how the problem works. And what a marvelous thing your brain is by being able to come up with any solution.
CRConrad 5y ago
The idea is to arrive at a rule you can use for any given picture, to decide if it goes on the left or right: > In a Bongard problem, you have to figure out what the rule is. You are given twelve specific images, and the result of applying the rule to each. (The rule assigns an image to either the left or right group.) Once you have discovered the rule, applying it to new images would be trivial. Your rules don't do that. Most of the pictures have neither "triangle in circle" nor "circle in triangle", so your rules don't apply to... Ten out of the twelve pictures.

yamrzou 5y ago

Very Interesting. This reminds me of The Abstraction and Reasoning Corpus [1] by François Chollet, accompanying his paper On the Measure of Intelligence [2]... Edit: Found a recent article mentioning both and discussing a NeurIPS paper on using Bongard problems to test AI systems [3]. [1] https://github.com/fchollet/ARC https://github.com/fchollet/ARC [2] https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.01547 [3] https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/how-do-you-test-the-iq-of-ai https://spectrum.ieee.org/tech-talk/artificial-intelligence/...

brudgers 5y ago

I am reminded that the simplest regex for the words "apex, ibex, index" is 'apex|ibex|index'. A commonality of all the boxes on the right is being on the right. A commonality of all the boxes on the left is being on the left. There is no offside in golf. The rules of the game only apply if when we are playing the game. Here, Wittgenstein might have said Bongard problems are another language game and the confusion arises from using words in a peculiar way...the game is pretending there is a problem in a Bongard problem.

Gys 5y ago
‘There is no spoon’
EricBurnett 5y ago
It depends how you define "simple" : '(ap|ib|ind)ex' is fewer symbols and a smaller character graph. There is no offside in golf...yet. But the rules are surprisingly long, and will only get longer as loopholes are found and exploited. Humans trying to codify a game, or any system, cannot do so precisely; others can and will find the gaps between the written rules and the intended meaning, and play a different game but pretend it's the same. Or I guess, every game is a language game? Which I think is a point of meta rationality: we can't avoid including questions of the rules of the problem. It's clouds all the way down.
- brudgers 5y ago
  ‘*’ is fewer characters. If we are playing that language game.
  - handoflixue 5y ago
    > ‘*’ is fewer characters. yes, but that produces false positives, whereas both 'apex|ibex|index' and '(ap|ib|ind)ex' do not. Your original claim was that 'apex|ibex|index' was "simplest", which means it has some property you think that '(ap|ib|ind)ex' lacks, but you haven't yet articulated what that IS.

chubot 5y ago

From the Church-Turing Thesis, we know there’s nothing special going on! We know humans can’t do anything more than a computer can. I see people making such claims about human cognition all the time, and I have no idea how it follows. (note the author is paraphrasing "people" here) The Church-Turing Thesis says nothing about human cognition. It is perfectly plausible that a human can do things a computer can't. (Scott Aaronson has a paper "Why Philosophers Should Care About Computational Complexity" which sheds some light on why that might be, but it's far from the only possible reason.) The burden of proof is on people who claim that human cognition can be simulated by computer, not the other way around. To me, it seems far more likely that it can't. Human cognition can obviously be simulated by "the laws of physics", since brains are material, but it seems very likely that computers are less powerful than that. That's my refutation of the (silly IMO) "simulation argument". I'd argue it's simply not possible to simulate another universe. You can simulate something like SimCity or whatever, but not a real universe. The people who make that argument always seem to leave out the possibility that it's physically impossible. In fact I would actually take the simulation argument ("we are almost certainly living in a simulation") as proof by contradiction that simulation is impossible.

throwamon 5y ago
> That's my refutation of the (silly IMO) "simulation argument" I'm not one of these people, but your rebuttal wouldn't convince me if I were. Maybe we are the SimCity of a much more complex universe.
- chubot 5y ago
  OK, but I'm saying the burden of proof is on who think that simulation is possible. It's not at all obvious that it is. As far as I can tell, the idea was basically made popular by movies [1], and there is no science behind it. All the science I know of points the other way -- simulating anything is incredibly hard and slow. It requires approximations and shortcuts to make it work for specific cases, and it doesn't work in the general case. (Maybe some alternative model like quantum computation will be different, but we're MUCH much further there. I think "adding two small numbers" is still an issue for state of the art quantum computers.) ----- Here's a nice example from a few days ago, trying to represent even a tiny part of the human brain in a computer: https://news.ycombinator.com/item?id=27362883 https://news.ycombinator.com/item?id=27362883 [1] I think this is more literally true than you might expect; IIRC the published papers in philosophy liberally reference The Matrix, maybe because it attracts readers.

hamilyon2 5y ago

Author links to Turing-Church thesis, which is widely assumed to be right to talk about things not related to any single thing that thesis is about. It is about computable functions and abstract machines.

hliyan 5y ago

I've always considered "pattern recognition for the purpose of prediction" a core function of the human brain that precedes language, rationality or any form of logic. So it feels counter-intuitive to me to label this meta-rationality or even associate it with rationality. I subscribe somewhat to the idea that much of our rational decision making is ex post facto, i.e a verbal narration that comes after the decision making to explain the decision. The actual decision making being an opaque process that takes place inside our brain's neural network. Confession: I lost the author about halfway through the article.

diab0lic 5y ago
This ex post facto explanation is referred to as rationalization and is ironically irrational. If you’ve already made a decision you’ve forfeited the option to do so rationally. I do agree that most of what gets labeled rational is in fact ex post facto rationalization. I do it myself all the time, haha.
- rrmm 5y ago
  Ex post facto rationalization is a major annoyance of mine. Let's just be honest with ourselves and recognize that lots of what we want isn't rational. That's ok, use your rationality to keep yourself from making huge, dumb mistakes, but don't pretend you're doing it for "rational" reasons or through a "rational" process. But I'm kind of over-strict about it and like to reserve "rational" for processes you can do with first-order logic. Given the complexity and lack of information in real life, that happens almost never. Of course this is not a terribly rational point of view.

pontifier 5y ago

I have recently come up with a model that has been useful to me for thinking about thinking. It involves the realization that different brain regions must communicate, but also contain their own representation of reality. Partial thought precursors echo back and forth between these regions, with each region amplifying or dampening parts of the idea that it recognizes as valid. When multiple brain regions begin to agree on its validity to a high level, the aha moment occurs. This model has some characteristics of waveform collapse, and discrete task specific neural networks. When multiple tasks specific networks arrive at consensus that a model matches experience, the proto-idea forms. This proto-idea can then be evaluated, and inspected. New scenarios are reflected off this new idea, to see if it continues to make sense. Converting an idea into words makes it useful to others, and allows sharing of ideas. This process requires refinement by echoing back and forth with the proto-idea until the words match it's shape. In order for these words to be understood effectively, they need to make sense to the brains that are receiving them. That means the words chosen need to activate multiple brain regions that the listener may use to evaluate this new idea and have the aha moment themselves. This process is easier when the 2 brains have many shared experiences to draw on, or communication is bi-directional to allow message refinement.

webmaven 5y ago
> It involves the realization that different brain regions must communicate, but also contain their own representation of reality. Sure, but this is complicated by regions being fuzzy and overlapping (or perhaps interpenetrating).
- pontifier 5y ago
  Each part in a healthy brain should ideally take input, process it somehow, and reflect back an echo that amplifies and dampens the parts that match it's representation of reality. The lines between regions aren't as important. I'm almost thinking about how this could be turned into a device for conveying information with actual echos.
  - webmaven 5y ago

zeroonetwothree 5y ago

I don’t see how Bongard problems are human complete if the author can’t even solve half of them. Does that mean he doesn’t have human intelligence? I think a better candidate for human complete is “knowing what other humans are thinking”. AKA “theory of mind”.

Viliam1234 5y ago
That would exclude many neurodivergent people. Perhaps also other cultures, and social classes. It is easier to guess the state of minds of people who are similar to you. Because then your natural algorithm "what would I feel in such situation? what would make me say these words, act this way?" is more likely to match how they feel and think. I suspect that many people overestimate their ability to "read other people's minds". First, they rarely verify their guesses. (I see another person and conclude that they are angry. I usually don't approach them and ask "hey, are you angry?". Therefore, if my guess was wrong, I am not going to learn it.) Second, if they turn out to be wrong, it's always the other person's fault. (If an autist cannot guess a neurotypical person's thoughts, it's the autist's fault. If a neurotypical person cannot guess an autist's thoughts, guess what, that's also the autist's fault.) Third, it is easier to guess thoughts of people we frequently talk to, because people usually think today the same thing they thought yesterday, and they already told us what they thought yesterday.
skybrian 5y ago
Being able to solve half of them by computer would be amazing progress, indicating a breakthrough in machine learning research. But whether that means artificial general intelligence is solved is another question. Most people can’t play Go very well either. It’s difficult to say whether the solution will generalize without having it, but easy to imagine that it might.

wydfre 5y ago

If anyone is interested in this, they should look into Alfred Korzybski and general semantics. He invented the term "The map is not the territory", in case you want an idea of who he is. IIRC, he at one point says in his book, "Science and Sanity: An Introduction to Non-Aristotelian Systems and General Semantics" something along the lines that the mistakes most people make are in categorization. Something along the lines of "Some things look the same but they are different, and some things look different but they are the same". It's a very interesting book and I loved how Non-Aristotelian logic was used in Null-A by A.E. van Vogt, which introduced it to me.

slx26 5y ago

We should keep one morning of school a week, no matter our age, to get together with other humans and discuss about stuff like this. Individual reading and HN comments are fine, but we are missing the best part of learning, which is actually verbalizing and battle-testing those vague ideas. It would really help in cases like this... Have no friends? Make a club!

cousin_it 5y ago

A site with lots of Bongard problems: https://www.foundalis.com/res/bps/bpidx.htm https://www.foundalis.com/res/bps/bpidx.htm

tomcatfish 5y ago

> However, by “system” I mean, roughly, a set of rules that can be printed in a book weighing less than ten kilograms, and which a person can consciously follow. Is this tongue in cheek or is it a very strange thing to say? That's definitely not a statement that should be made with no justification. I can guess what the author meant, but I don't really want to guess at basics of their argument.

tlb 5y ago
It's important to exclude infinite books, which could just contain a lookup table for every possible situation rather than any general rules. And also finite but exponentially large books, which could contain a lookup table for every possible image (and be something like 2^(1024x768x24) pages long.)

raghuveerdotnet 5y ago

I still don’t understand how this is any different from the notion of problem solving. And I really didn’t understand the human-AI equivalence, is it just the substrate independent nature of turing-completeness? If so, I think we still lack the epistemic toolkit to say anything conclusive about creating an equivalence, or for that matter any form of comparative relationship, especially given that AGI is still not a problem that is well-defined, let alone discussing the solution space. No?

sonkol 5y ago

I know this is a little bit out of topic but I think meta-rationality is more about organizing other people/machines intelligences to achieve your goal even when you are not highly intelligent.

timboy03 5y ago

One tricky thing about Bongard problems is that for any given problem there are likely many different rules that could distinguish the six positive examples from the six negative examples. For example, maybe a problem that is "really" about circles vs. triangles also happens to have more black pixels in the left images than in the right images. A key skill in solving these problems is not just to find a compact and discriminating description, but to find such a description that is also one that a human Bongard problem designer would be likely to think was a cool and elegant puzzle that needs an "Aha" moment to recognize. If you find such a description, then you're very likely to be right. I suspect that that last part (recognizing when you have found a solution that is pleasing enough to be the answer) is likely to be the biggest challenge for ML-based approaches to Bongard problems

lorepieri 5y ago

These problems remind me of https://github.com/fchollet/ARC https://github.com/fchollet/ARC "ARC can be seen as a general artificial intelligence benchmark, as a program synthesis benchmark, or as a psychometric intelligence test. It is targeted at both humans and artificially intelligent systems that aim at emulating a human-like form of general fluid intelligence."