8 ms·

All AI models might be the same

311 points by jxmorris12 1y ago

tyronehed 1y ago
Especially if they are all me-too copies of a Transformer. When we arrive at AGI, you can be certain it will not contain a Transformer.
- jxmorris12 1y ago
  I don't think architecture matters. It seems to be more a function of the data somehow. I once saw a LessWrong post claiming that the Platonic Representation Hypothesis doesn't hold when you only embed random noise, as opposed to natural images: http://lesswrong.com/posts/Su2pg7iwBM55yjQdt/exploring-the-platonic-representation-hypothesis-beyond-in http://lesswrong.com/posts/Su2pg7iwBM55yjQdt/exploring-the-p...
  - blibble 1y ago
    > I don't think architecture matters. It seems to be more a function of the data somehow. of course it matters if I supply the ants in my garden with instructions on how to build tanks and stealth bombers they're still not going to be able to conquer my front room
  - SJC_Hacker 1y ago
    Its a bit like saying algorithms don't matter for solving computational problems. Two different algorithms might produce equivalent results but if you have to wait years for an answer, when seconds matter, the slow algorithm isn't helpful. I believe the current approach of using mostly a feed-forward in the inference stage, with well-filtered training data and backpropagation for discrete "training cycles" has limitations. I know this has been tried in the past, but something modelling how animal brains actually function, with continuous feedback, no explicit "training" (we're always being trained), might be the key. Unfortunately our knowledge of "whats really going on" in the brains is still limited, investigative methods are crude as the brain is difficult to image at the resolution we need, and in real time. Last I checked no one's quite figured out how memory works, for example. Whether its "stored in the network" somehow through feedback (like a SR-latch or flip-flop in electronics) or whether there's some underlying chemical process within the neuron itself (we know that chemicals definitely regulate brain function, don't know how much it goes the other way and it can be used to encode state)
TheSaifurRahman 1y ago
This only works when different sources share similar feature distributions and semantic relationships. The M or B game breaks down when you play with someone who knows obscure people you've never heard of. Either you can't recognize their references, or your sense of "semantic distance" differs from theirs. The solution is to match knowledge levels: experts play with experts, generalists with generalists. The same applies to decoding ancient texts, if ancient civilizations focused on completely different concepts than we do today, our modern semantic models won't help us understand their writing.
- npinsker 1y ago
  I've played this game with friends occasionally and -- when it's a person -- don't think I've ever completed a game.
TheSaifurRahman 1y ago
Has there been research on using this to make models smaller? If models converge on similar representations, we should be able to build more efficient architectures around those core features.
- yorwba 1y ago
  It's more likely that such an architecture would be bigger rather than smaller. https://arxiv.org/abs/2412.20292 https://arxiv.org/abs/2412.20292 demonstrated that score-matching diffusion models approximate a process that combines patches from different training images. To build a model that makes use of this fact, all you need to do is look up the right patch in the training data. Of course a model the size of its training data would typically be rather unwieldy to use. If you want something smaller, we're back to approximations created by training the old-fashioned way.
  - samsartor 1y ago
    I have mixed feelings about this interpretation: that diffusion models approximately produce moseics from patches of training data. It does a good job helping people understand why diffusion models are able to work. I used it myself in talk almost 3 years ago! And it isn't a lie exactly, the linked paper is totally sound. It's just that it only works if you assume your model is an absolute optimal minimization of the loss (under some inductive biases). It isn't. No machine learning more complicated than OLS holds up to that standard. _And that's the actual reason they work._ Undefit models don't just approximate, they interpolate, extrapolate, generalize a bit, and ideally smooth out the occasional total garbage mixed in with your data. In fact, diffusion models work so well because they can correct their own garbage! If extra fingers start to show up in step 5, then steps 6 and 7 still have a chance to reinterpret that as noise and correct back into distribution. And then there's all the stuff you can do with diffusion models. In my research I hack into the model and use it to decompose images into the surface material properties and lighting! That doesn't make much sense as averaging of memorized patches. Given all that, it is a very useful interpretation. But I wouldn't take it too literally.
    - 1y ago
deleted 1y ago
[deleted]
empath75 1y ago
This is kind of fascinating because I just tried to play mussolini or bread with chatgpt and it is absolutely _awful_ at it, even with reasoning models. It just assumes that your answers are going to be reasonably bread-like or reasonably mussolini-like, and doesn't think laterally at all. It just kept asking me about varieties of baked goods. edit: It did much better after I added some extra explanation -- that it could be anything that it may be very unlike either choice, and not to try and narrow down too quickly
- fsmv 1y ago
  I think an LLM is a bit too high level for this game or maybe it just would need a lengthy prompt to explain the game. If you used word2vec directly it's the exact right thing to play this game with. Those embeddings exist in an LLM but it's trained to respond like text found online not play this game.
- Quarrelsome 1y ago
  what surprised me when I asked if it knew what Mussolini or Bread was, it parroted this article. This article was posted in the last 24 hours and querying a search engine for "Mussolini or Bread" didn't yield it anywhere near a top result.
  - hatthew 1y ago
    searching for '"Mussolini or Bread"' (with quotes), or searching for 'Mussolini or Bread game' (without quotes) both return this article at the top.
    - Quarrelsome 1y ago
      Idiot me forgot to try the quotes. Thank you.
Xcelerate 1y ago
Edit: I wrote my comment a bit too early before finishing the whole article. I'll leave my comment below, but it's actually not very closely related to the topic at hand or the author's paper. I agree with the gist of the article (which IMO is basically that universal computation is universal regardless of how you perform it), but there are two big issues that prevent this observation from helping us in a practical sense: 1. Not all models are equally efficient. We already have many methods to perform universal search (e.g., Levin's, Hutter's, and Schmidhuber's versions), but they are painfully slow despite being optimal in a narrow sense that doesn't extrapolate well to real world performance. 2. Solomonoff induction is only optimal for infinite data (i.e., it can be used to create a predictor that asymptotically dominates any other algorithmic predictor). As far as I can tell, the problem remains totally unsolved for finite data, due to the additive constant that results from the question: which universal model of computation should be applied to finite data? You can easily construct a Turing machine that is universal and perfectly reproduces the training data, yet nevertheless dramatically fails to generalize. No one has made a strong case for any specific natural prior over universal Turing machines (and if you try to define some measure to quantify the "size" of a Turing machine you realize this method starts to fail once the number of transition tables becomes large enough to start exhibiting redundancy).
- im3w1l 1y ago
  Regarding your second point I think there are two cases here that should be kept separate. The first is that you are teleported into a parallel dimension where literally everything works differently from here. In that case I do agree that there are several reasonable choices of models of computation. You simply have to pick one and hope it wasn't too bad. But the second case is that you encounter some phenomenon here in our ordinary world. And in that case I think you can do way better by reasoning about the phenomenon and trying to guess at plausible mechanics based on your preexisting knowledge of how the world works. In particular, I think guessing that "there is some short natural language description of how the phenomenon works, based on a language grounded in the corpus of human writing" is a very reasonable prior.
- deleted 1y ago
  [deleted]
dr_dshiv 1y ago
What about the platonic bits? Any other articles that give more details there?
- vintagedave 1y ago
  It refers to Plato's Theory of Forms: https://en.wikipedia.org/wiki/Theory_of_forms https://en.wikipedia.org/wiki/Theory_of_forms Consider how every chair you've seen is different. Yet they are all chairs. What is 'chair-ness'? The Forms theory is that there is a single ideal chair, and this truly exists - not materially, but non-materially as a concept or way of understanding chairs, but real. That is, the 'essence' of a chair exists. All chairs are imperfect representations of the essence of a chair, being perfect chairs or representing the essence of a chair to a greater or lesser degree. You might find Neal Stephenson's bool Anathem a wonderful read. It's one of my favorites. I want to tell you why without spoilers but I can't (it won't be what you expect) -- but if you read it perhaps you'll find it interesting re this topic.
somethingsome 1y ago
Mmmh I'm deeply skeptical of some parts. > One explanation for why this game works is that there is only one way in which things are related There is not, this is a completely non transitive relationship. On another point, suppose you keep the same vocabulary, but permute the signification of the words, the neural network will still learn relationships, completely different ones, but it's representation may converge toward a better compression for that set of words, but I'm dubious that this new compression scheme will ressemble the previous one (?) I would say that given an optimal encoding of the relationships, we can achieve an extreme compression, but not all encodings lead to the same compression at the end. If I add 'bla' between every words in a text, that is easy to compress, but now, if I add an increasing sequence of words between each words, the meaning is still there, but the compression will not be the same, as the network will try to generate the words in-between. (thinking out loud)
coffeecoders 1y ago
I think “we might decode whale speech or ancient languages” is a huge stretch. Context is the most important part of what makes language useful. There is billions of human-written texts, grounded in shared experience that makes our AI good at language. We don't have that for a whale.
- klank 1y ago
  If a lion could speak, would we understand it?
  - eddythompson80 1y ago
    There is nothing really special about speech as a form of communication. All animals communicate with each other and with other animals. Informational density and, uhhhhh, cyclomatic complexity might be different between speech and a dance or a grunt or whatever.
    - klank 1y ago
      I was referencing Wittgenstein's "If a lion could speak, we would not understand it." Wittgenstein believed (and I am strongly inclined to agree with him) that our ability to convey meaning through communication was intrinsically tied to (or, rather, sprang forth from) our physical, lived experiences. Thus, to your point, assuming communication, because "there's nothing really special about speech", does that mean we would be able to understand a lion, if the lion could speak? Wittgenstein would say probably not. At least not initially and not until we had built shared lived experiences.
      - cdrini 1y ago
        Hmm I'm not convinced we don't have a lot of shared experience. We live on the same planet. We both hunger, eat, and drink. We see the sun, the grass, the sky. We both have muscles that stretch and compress. We both sleep and yawn. I mean who knows, maybe their perception of these shared experiences would be different enough to make communication difficult, but still, I think it's undeniably shared experience.
streptomycin 1y ago
Is it closer to Mussolini or bread? Mussolini. Is it closer to Mussolini or David Beckham? Uhh, I guess Mussolini. (Ok, they’re definitely thinking of a person.) That reasoning doesn't follow. Many things besides people would have the same answers, for instance any animal that seems more like Mussolini than Beckham.
- jxmorris12 1y ago
  Whoops. I hope you can overlook this minor logical error.
  - streptomycin 1y ago
    Oh yeah it's absolutely an interesting article!
- pjio 1y ago
  I believe the joke is about David Beckham not really being (perceived as) human, even when compared to personified evil
- Fomite 1y ago
  Oswald Mosley
- never_inline 1y ago
  My thought exactly. My word would be "Gun" or "Artillery". easily defeats his logic. But that should also explain us why pure embeddings search is not sufficient for RAG.
- canjobear 1y ago
  I think the point is that, despite the game being massively logically underspecified, people are still able to home in on an answer in practice. Which shows that they have a shared fuzzy semantic space.
gerdesj 1y ago
The devil is in the details. I recently gave the "Veeam Intelligence" a spin. Veeam is a backup system spanning quite a lot of IT systems with a lot of options - it is quite complicated but it is also a bounded domain - the app does as the app does. It is very mature and has extremely good technical documentation and a massive amount of technical information docs (TIDs) and a vibrant and very well informed set of web forums, staffed by ... staff and even the likes of Anton Gostev - https://www.veeam.com/company/management-team.html https://www.veeam.com/company/management-team.html Surely they have close to the perfect data set to train on? I asked a question about moving existing VMware replicas from one datastore to another and how to keep my replication jobs working correctly. In this field, you may not be familiar with my particular requirements but this is not a niche issue. The "VI" came up with a reasonable sounding answer involving a wizard. I hunted around the GUI looking for it (I had actually used that wizard a while back). So I asked where it was and was given directions. It wasn't there. The wizard was genuine but its usage here was a hallucination. A human might have done the same thing with some half remembered knowledge but would soon fix that with the docs or the app itself. I will stick to reading the docs. They are really well written and I am reasonably proficient in this field so actually - a decent index is all I need to get a job done. I might get some of my staff to play with this thing when given a few tasks that they are unfamiliar with and see what it comes up with. I am sure that domain specific LLMs are where it is at but we need some sort of efficient "fact checker" system.
ieie3366 1y ago
LLMs are bruteforce reverse engineered human brains. Think about it. Any written text out there is written by human brains. The ”function” to output this is whatever happens inside the brain, insanely complex. LLM ”training” is just brute forcing the same function into existence. ”Human brain output X, llm output Y, mutate it times billion until X and Y start matching”
tgsovlerkhgsel 1y ago
I've noticed that many of the large, separately developed AIs often answer with remarkably similar wording to the same question.
- lloeki 1y ago
  I'm not exactly surprised that "separately developed" AI that hinge on essentially the same core techniques and are fed largely identical corpus often answer with remarkably similar wording to the same question.
  - imtringued 1y ago
    I find it grating that the article is sweeping this under the rug. If you train your AI on Shakespeare it won't suddenly know how to control a robot. Meanwhile all Shakespeare bots will be remarkably similar. Coincidence?
  - krackers 1y ago
    At least for LLMs, the platonic representation hypothesis doesn't seem that surprising. The training data for models that are large enough to faithfully capture reality will necessarily have a lot of overlap. So in this weak form the platonic representation hypothesis just says that if you provide the roughly same data to models with roughly the same architecture, they learn roughly the same embeddings. This doesn't seem that surprising. I'd even expect a slightly stronger version where you feed the same data to different models to still hold. After all the modes capture the same reality, so there's probably some way to transform from the "latent space" of one model to the other.
kindkang2024 1y ago
The Dao can be spoken of, yet what is spoken is not the eternal Dao. So, what is the Dao? Personally, I see it as will — something we humans could express through words. For any given will, even though we use different words in different languages — Chinese, Japanese, English — these are simply different representations of the same will. Large language models learn from word tokens and begin to grasp these wills — and in doing so, they become the Dao. In that sense, I agree: “All AI models might be the same.”
foxes 1y ago
So in the limit the models representation space has one dimension per "concept" or something, but making it couple things together is what actually makes it useful? An infinite dimensional model with just one dim per concept would be sorta useless, but you need things tied together?
IAmNotACellist 1y ago
I agree LLMs are converging on a current representation of reality based on the collective works of humanity. What we need to do is provide AIs with realtime sensory input, simulated hormones each with their own half-lifes based on metabolic conditions and energy usage, a constant thinking loop, and discover a synthetic psilocybin that's capable of causing creative, cross-neural connections similar to human brains. We have the stoned ape theory, we need the stoned AI theory.
- buffet_overflow 1y ago
  Or perhaps we make them attractions at a theme park, but let Anthony Hopkins have admin access to the source code. What could go wrong?
- chamomeal 1y ago
  I’m so incredibly tired of reading about AI but if I saw “we gave an AI shrooms” I’d be all over it
  - josh-sematic 1y ago
    I imagine the temperature parameter is the closest thing we have to that right now. Turn it up a little to make things a bit more creative. Turn it up more and eventually it will wildly swing around in its dialogue. Even more and it eventually produces utter gibberish.
lsy 1y ago
The example given for inverting an embedding back to text doesn't help the idea that this effect is reflecting some "shared statistical model of reality": What would be the plausible whalesong mapping of "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby"? There isn't anything core to reality about Kentucky, its Derby, the Gregorian calendar, America, horse breeds, etc. These are all cultural inventions that happen to have particular importance in global human culture because of accidents of history, and are well-attested in training sets. At best we are seeing some statistical convergence on training sets because everyone is training on the same pile and scraping the barrel for any differences.
- benlivengood 1y ago
  You also can't translate "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby" into Hellenistic Greek or some modern indigenous languages because there isn't enough shared context; you'd need to give humans speaking those languages a glossary for any of the translation to make sense, or allow them to interrogate an LLM to act as the glossary. I'd say our current largest LLMs probably contain sufficient detail to explain a concept like a named race horse starting from QCD+gravity and ending up at cultural human events, given a foothold of some common ground to translate into a new unknown language. In a sense, that's what a model of reality is. I think it's possible because LLMs figure out translation between human languages by default with enough pretraining.
  - photonthug 1y ago
    > You also can't translate .. into Hellenistic Greek or some modern indigenous languages because there isn't enough shared context; you'd need to give humans speaking those languages a glossary for any of the translation to make sense What? By substitution, this means you can translate it. As long as we're assuming a large enough basis of concept vectors of course it works. > I'd say our current largest LLMs probably contain sufficient detail to explain a concept like a named race horse starting from QCD+gravity and ending up at cultural human events What? I'm curious how you'd propose to move from gravity to culture. This is like TFAs assertion that the M+B game might be as expressive as 20 questions / universal. M or B is just (bad,sentient) or (good,object). Sure, entangling a few concepts is slightly more expressive than playing with a completely flattened world of 1/0 due to some increase in dimensionality. But trying to pinpoint anything like (neutral,concept) fails because the basis set isn't fundamentally large enough. Any explanation of how this could work will have to cheat, like when TFA smuggles new information in by communicating details about distance-from-basis. For example to really get to the word or concept of "neutral" from inferred good/bad dichotomy of bread/mussolini, you would have to answer "Hmmmmmm, closer to bread I guess" in one iteration and then "Umm.. closer to Mussolini I guess" when asked again, and then have the interrogator notice the uncertainty/hesitation/contradiction and then infer neutrality. This is just the simple case.. physics to culture seems much harder
stillpointlab 1y ago
I have to be careful of confirmation bias when I read stuff like this because I have the intuition that we are uncovering a single intelligence with each of the different LLMs. I even feel, when switching between the big three (OpenAI, Google, Anthropic) that there is a lot of similarity in how they speak and think - but I am aware of my bias so I try not to let it cloud my judgement. On the topic of compression, I am reminded of an anecdote about Heidegger. Apparently he had a bias towards German and Greek, claiming that these languages were the only suitable forms for philosophy. His claim was based on the "puns" in language, or homonyms. He had some intuition that deep truths about reality were hidden in these over-loaded words, and that the particular puns in German and Greek were essential to understand the most fundamental philosophical ideas. This feels similar to the idea of shared embeddings being a critical aspect of LLM emergent intelligence. This "superposition" of meaning in representation space again aligns with my intuitions. I'm glad there are people seriously studying this.
- b112 1y ago
  LLMs don't think, nor are they intelligent or exhibiting intelligence. Language does have constraints, yet it evolves via its users to encompass new meanings. Thus those constraints are artificial, unless you artificially enforce static language use. And of course, for an LLM to use those new concepts, it needs to be retokenized by being trained on new data. For example, if we trained LLMs only on books, encyclopedias, newpapers, and personal letters from 1850, it would have zero capacity to speak comprehensibly or even seem cogent on much of the modern world. And it would forever remain in that disconnected positon. LLMs do not think, understand anything, nor learn. Should you wish to call tokenization, learning, then you'd better call a clock "learning" from the gears and cogs that enable its function. LLMs do not think, learn, or exhibit intelligence. (I feel this is not said enough). We will never, ever get AGI from an LLM. Ever. I am sympathetic to the wonder of LLMs. To seeing them as such. But I see some art as wonderous too. Some machinery is beautiful in execution and to use. But that doesn't change truths.
  - kevin42 1y ago
    You made a lot of bold assertions there. It's as if you have a complete and definitive theory of human intelligence to compare it against. Which if true, would be incredible, because there isn't a scientifically accepted theory, nor is there consensus from a philosophical standpoint. I can't say that you are wrong, you might be right, especially about AGI. And I think it's unlikely that LLMs are the direct path to AGI. But, just looking at how human brains work, it seems unlikely that we would be intelligent either if we used your same reductionist logic. An individual neuron doesn't "think" or "understand" anything. It's a biological cell that simply fires an electrochemical signal when its input threshold is met. It has no understanding of language or context. By your logic, since the fundamental components are just simple signal processors, the brain cannot possibly learn or be intelligent. Yet, from the complex interaction of ~86 billion of these simple biochemical machines, the emergent properties of thought, understanding, and consciousness arise. Dismissing an LLM's capabilities because its underlying operations are basically just math operating on tokenized data is like dismissing human consciousness because it's "just electrochemistry" in a network of cells. Both arguments mistake the low-level mechanism for the high-level emergent phenomenon.
megaloblasto 1y ago
Really cool idea that I hadn't considered yet. If true, seems like a big plus for open source and not having a few companies controll all the models. If they all converge to the same intelligence, one open source model would make all proprietary models obsolete.
seeknotfind 1y ago
Platonism - not even once. Green is the smell of my grandmother's lawn on a hot summer day. Just because things are similar to a lot of people doesn't mean their fundamentally the same.
darepublic 1y ago
Why is claude shannon closer to mussolini than to david beckham?
robertclaus 1y ago
The board game DaDaDa is a great example of this.
stevage 1y ago
I tried playing Mussolini or Bread with ChatGPT, but it didn't go very well. It seemed to have trouble grasping the rules, and kept getting overly specific when we were miles from the right concept.
deleted 1y ago
[deleted]
amoss 1y ago
When I read that "Ilya gave a famously incomprehensible talk about the connections between intelligence and compression" it makes me wonder if Marcus Hutter has now been forgotten? If so more people should take a look at http://prize.hutter1.net/ http://prize.hutter1.net/
zoeey 1y ago
[dead]
tempfile 1y ago
I know it's only one tiny part, but I can't resist: > Is it closer to Mussolini or David Beckham? Uhh, I guess Mussolini. (Ok, they’re definitely thinking of a person.) This deduction is absurd! The only information you have at all is that it's more like a person than it is like bread (which could be almost anything).
derbOac 1y ago
Seemed to me they are just saying that typical LLMs are consistent: https://en.m.wikipedia.org/wiki/Consistent_estimator https://en.m.wikipedia.org/wiki/Consistent_estimator
pona-a 1y ago
I think some of this effect is explained by different models being trained on almost identical corpa. Even if you try to erase certain works, contamination will still represent some of it, just in less obvious ways.
throwpoaster 1y ago
I asked Grok, o3-pro, and Claude a question about piezoelectric effects. They all got it "right", but Claude called out a second order effect that arose from the use case that the other two missed. I get it, they might all be exploring the same space, but Claude went an extra, important, hop.
- Rover222 1y ago
  I'm just curious if that was Grok 4 or 3
  - throwpoaster 1y ago
    3! So it’s not a fair comparison. The other two were both paid. I am very curious to run real world engineering problems through SuperHeavy.
    - Rover222 1y ago
      Have you tried SuperHeavy yet? It seems to be crushing on almost any measurable benchmark. I've only used Grok 4 (not heavy) when stuck on a problem that claude/cursor cannot get past, and Grok 4 is often able to come up with an elegant solution.
antonvs 1y ago
> One explanation for why this game works is that there is only one way in which things are related, and this comes from the underlying world we live in. I think what this might be trying to say is something more like: ...there are many ways in which things can be related, but those relationships come from the underlying world we live in. I.e. there are obviously many ways in which things can be related, but if we assume the quote is not entirely counterfactual, then it must be getting at something else. I suppose "way" here is being used in a different sense, but it isn't clear.
cultofmetatron 1y ago
someone just discovered the AI equivalent of "everything turns to crabs"
d_burfoot 1y ago
For those who don't know about Plato's Theory of Forms, the remarkable empirical fact is that all humans learn roughly the same concepts representing words like "dog", "house", "person", "boat", and so on, even if those people grew up in different places and never had any overlap between their observational experience. You and I may never have observed the same dog, but we still agree on what a "dog" is. This phenomenon appears to occur for LLM learning as well, though it is less remarkable due to the fact that LLMs likely have significant overlap in their training data. I believe this is good news for Alignment, because, as Plato pointed out, one of the most important forms is the Form of the Good - a (theoretical) universal human ideal containing notions of justice, virtue, compassion, excellence, etc. If this Form truly exists, and LLMs can learn it, it may be possible to train them to pursue it (or refuse requests that are opposed to it).
- wagwang 1y ago
  Basically... Jungian archetypes
- hobbyjogger 1y ago
  This vastly oversimplifies the notion of what is "good" or "fair" in a given situation. Just because we can generally agree on classifying simple objects like a boat or a house doesn't entail that we share a universal notion about the virtuous thing to do in a given situation. Humans (even in the same society/environment) have wildly differing ideas about moral issues like abortion, assisted suicide, medical testing on animals/stem cells, etc. Consider this chart from a 2010 Gallup Poll: https://content.gallup.com/origin/gallupinc/GallupSpaces/Production/Cms/POLL/ztybao6qoegc_-aam95srw.gif https://content.gallup.com/origin/gallupinc/GallupSpaces/Pro...
  - reactordev 1y ago
    Ahh, semantic search problem rears its head once more. While we can agree on what a "dog" is. We disagree on it's value.
- pphysch 1y ago
  > all humans learn roughly the same concepts representing words like "dog", "house", "person", "boat", and so on, even if those people grew up in different places and never had any overlap between their observational experience. This "roughly" is doing a lot of heavy lifting to support the "Plato was right" thesis. The fact is we inhabit a shared reality with shared laws of physics and evolutionary pressures and so on; there are only so many ways to float a boat. That doesn't necessarily mean Platonic Forms exist and that everyone arrives independently at the same Form. Indeed you will find that people have fundamentally different ideas of what words like "freedom", "economy", "government" mean, despite using the same syntax.

311 points by jxmorris12 1y ago

Especially if they are all me-too copies of a Transformer. When we arrive at AGI, you can be certain it will not contain a Transformer.

jxmorris12 1y ago
I don't think architecture matters. It seems to be more a function of the data somehow. I once saw a LessWrong post claiming that the Platonic Representation Hypothesis doesn't hold when you only embed random noise, as opposed to natural images: http://lesswrong.com/posts/Su2pg7iwBM55yjQdt/exploring-the-platonic-representation-hypothesis-beyond-in http://lesswrong.com/posts/Su2pg7iwBM55yjQdt/exploring-the-p...
- blibble 1y ago
  > I don't think architecture matters. It seems to be more a function of the data somehow. of course it matters if I supply the ants in my garden with instructions on how to build tanks and stealth bombers they're still not going to be able to conquer my front room
- SJC_Hacker 1y ago
  Its a bit like saying algorithms don't matter for solving computational problems. Two different algorithms might produce equivalent results but if you have to wait years for an answer, when seconds matter, the slow algorithm isn't helpful. I believe the current approach of using mostly a feed-forward in the inference stage, with well-filtered training data and backpropagation for discrete "training cycles" has limitations. I know this has been tried in the past, but something modelling how animal brains actually function, with continuous feedback, no explicit "training" (we're always being trained), might be the key. Unfortunately our knowledge of "whats really going on" in the brains is still limited, investigative methods are crude as the brain is difficult to image at the resolution we need, and in real time. Last I checked no one's quite figured out how memory works, for example. Whether its "stored in the network" somehow through feedback (like a SR-latch or flip-flop in electronics) or whether there's some underlying chemical process within the neuron itself (we know that chemicals definitely regulate brain function, don't know how much it goes the other way and it can be used to encode state)

TheSaifurRahman 1y ago

This only works when different sources share similar feature distributions and semantic relationships. The M or B game breaks down when you play with someone who knows obscure people you've never heard of. Either you can't recognize their references, or your sense of "semantic distance" differs from theirs. The solution is to match knowledge levels: experts play with experts, generalists with generalists. The same applies to decoding ancient texts, if ancient civilizations focused on completely different concepts than we do today, our modern semantic models won't help us understand their writing.

npinsker 1y ago
I've played this game with friends occasionally and -- when it's a person -- don't think I've ever completed a game.

TheSaifurRahman 1y ago

Has there been research on using this to make models smaller? If models converge on similar representations, we should be able to build more efficient architectures around those core features.

yorwba 1y ago
It's more likely that such an architecture would be bigger rather than smaller. https://arxiv.org/abs/2412.20292 https://arxiv.org/abs/2412.20292 demonstrated that score-matching diffusion models approximate a process that combines patches from different training images. To build a model that makes use of this fact, all you need to do is look up the right patch in the training data. Of course a model the size of its training data would typically be rather unwieldy to use. If you want something smaller, we're back to approximations created by training the old-fashioned way.
- samsartor 1y ago
  I have mixed feelings about this interpretation: that diffusion models approximately produce moseics from patches of training data. It does a good job helping people understand why diffusion models are able to work. I used it myself in talk almost 3 years ago! And it isn't a lie exactly, the linked paper is totally sound. It's just that it only works if you assume your model is an absolute optimal minimization of the loss (under some inductive biases). It isn't. No machine learning more complicated than OLS holds up to that standard. _And that's the actual reason they work._ Undefit models don't just approximate, they interpolate, extrapolate, generalize a bit, and ideally smooth out the occasional total garbage mixed in with your data. In fact, diffusion models work so well because they can correct their own garbage! If extra fingers start to show up in step 5, then steps 6 and 7 still have a chance to reinterpret that as noise and correct back into distribution. And then there's all the stuff you can do with diffusion models. In my research I hack into the model and use it to decompose images into the surface material properties and lighting! That doesn't make much sense as averaging of memorized patches. Given all that, it is a very useful interpretation. But I wouldn't take it too literally.
  - 1y ago

deleted 1y ago

[deleted]

empath75 1y ago

This is kind of fascinating because I just tried to play mussolini or bread with chatgpt and it is absolutely _awful_ at it, even with reasoning models. It just assumes that your answers are going to be reasonably bread-like or reasonably mussolini-like, and doesn't think laterally at all. It just kept asking me about varieties of baked goods. edit: It did much better after I added some extra explanation -- that it could be anything that it may be very unlike either choice, and not to try and narrow down too quickly

fsmv 1y ago
I think an LLM is a bit too high level for this game or maybe it just would need a lengthy prompt to explain the game. If you used word2vec directly it's the exact right thing to play this game with. Those embeddings exist in an LLM but it's trained to respond like text found online not play this game.
Quarrelsome 1y ago
what surprised me when I asked if it knew what Mussolini or Bread was, it parroted this article. This article was posted in the last 24 hours and querying a search engine for "Mussolini or Bread" didn't yield it anywhere near a top result.
- hatthew 1y ago
  searching for '"Mussolini or Bread"' (with quotes), or searching for 'Mussolini or Bread game' (without quotes) both return this article at the top.
  - Quarrelsome 1y ago
    Idiot me forgot to try the quotes. Thank you.

Xcelerate 1y ago

Edit: I wrote my comment a bit too early before finishing the whole article. I'll leave my comment below, but it's actually not very closely related to the topic at hand or the author's paper. I agree with the gist of the article (which IMO is basically that universal computation is universal regardless of how you perform it), but there are two big issues that prevent this observation from helping us in a practical sense: 1. Not all models are equally efficient. We already have many methods to perform universal search (e.g., Levin's, Hutter's, and Schmidhuber's versions), but they are painfully slow despite being optimal in a narrow sense that doesn't extrapolate well to real world performance. 2. Solomonoff induction is only optimal for infinite data (i.e., it can be used to create a predictor that asymptotically dominates any other algorithmic predictor). As far as I can tell, the problem remains totally unsolved for finite data, due to the additive constant that results from the question: which universal model of computation should be applied to finite data? You can easily construct a Turing machine that is universal and perfectly reproduces the training data, yet nevertheless dramatically fails to generalize. No one has made a strong case for any specific natural prior over universal Turing machines (and if you try to define some measure to quantify the "size" of a Turing machine you realize this method starts to fail once the number of transition tables becomes large enough to start exhibiting redundancy).

im3w1l 1y ago
Regarding your second point I think there are two cases here that should be kept separate. The first is that you are teleported into a parallel dimension where literally everything works differently from here. In that case I do agree that there are several reasonable choices of models of computation. You simply have to pick one and hope it wasn't too bad. But the second case is that you encounter some phenomenon here in our ordinary world. And in that case I think you can do way better by reasoning about the phenomenon and trying to guess at plausible mechanics based on your preexisting knowledge of how the world works. In particular, I think guessing that "there is some short natural language description of how the phenomenon works, based on a language grounded in the corpus of human writing" is a very reasonable prior.
deleted 1y ago
[deleted]

dr_dshiv 1y ago

What about the platonic bits? Any other articles that give more details there?

vintagedave 1y ago
It refers to Plato's Theory of Forms: https://en.wikipedia.org/wiki/Theory_of_forms https://en.wikipedia.org/wiki/Theory_of_forms Consider how every chair you've seen is different. Yet they are all chairs. What is 'chair-ness'? The Forms theory is that there is a single ideal chair, and this truly exists - not materially, but non-materially as a concept or way of understanding chairs, but real. That is, the 'essence' of a chair exists. All chairs are imperfect representations of the essence of a chair, being perfect chairs or representing the essence of a chair to a greater or lesser degree. You might find Neal Stephenson's bool Anathem a wonderful read. It's one of my favorites. I want to tell you why without spoilers but I can't (it won't be what you expect) -- but if you read it perhaps you'll find it interesting re this topic.

somethingsome 1y ago

Mmmh I'm deeply skeptical of some parts. > One explanation for why this game works is that there is only one way in which things are related There is not, this is a completely non transitive relationship. On another point, suppose you keep the same vocabulary, but permute the signification of the words, the neural network will still learn relationships, completely different ones, but it's representation may converge toward a better compression for that set of words, but I'm dubious that this new compression scheme will ressemble the previous one (?) I would say that given an optimal encoding of the relationships, we can achieve an extreme compression, but not all encodings lead to the same compression at the end. If I add 'bla' between every words in a text, that is easy to compress, but now, if I add an increasing sequence of words between each words, the meaning is still there, but the compression will not be the same, as the network will try to generate the words in-between. (thinking out loud)

coffeecoders 1y ago

I think “we might decode whale speech or ancient languages” is a huge stretch. Context is the most important part of what makes language useful. There is billions of human-written texts, grounded in shared experience that makes our AI good at language. We don't have that for a whale.

klank 1y ago
If a lion could speak, would we understand it?
- eddythompson80 1y ago
  There is nothing really special about speech as a form of communication. All animals communicate with each other and with other animals. Informational density and, uhhhhh, cyclomatic complexity might be different between speech and a dance or a grunt or whatever.
  - klank 1y ago
    I was referencing Wittgenstein's "If a lion could speak, we would not understand it." Wittgenstein believed (and I am strongly inclined to agree with him) that our ability to convey meaning through communication was intrinsically tied to (or, rather, sprang forth from) our physical, lived experiences. Thus, to your point, assuming communication, because "there's nothing really special about speech", does that mean we would be able to understand a lion, if the lion could speak? Wittgenstein would say probably not. At least not initially and not until we had built shared lived experiences.
    - cdrini 1y ago
      Hmm I'm not convinced we don't have a lot of shared experience. We live on the same planet. We both hunger, eat, and drink. We see the sun, the grass, the sky. We both have muscles that stretch and compress. We both sleep and yawn. I mean who knows, maybe their perception of these shared experiences would be different enough to make communication difficult, but still, I think it's undeniably shared experience.

streptomycin 1y ago

Is it closer to Mussolini or bread? Mussolini. Is it closer to Mussolini or David Beckham? Uhh, I guess Mussolini. (Ok, they’re definitely thinking of a person.) That reasoning doesn't follow. Many things besides people would have the same answers, for instance any animal that seems more like Mussolini than Beckham.

jxmorris12 1y ago
Whoops. I hope you can overlook this minor logical error.
- streptomycin 1y ago
  Oh yeah it's absolutely an interesting article!
pjio 1y ago
I believe the joke is about David Beckham not really being (perceived as) human, even when compared to personified evil
Fomite 1y ago
Oswald Mosley
never_inline 1y ago
My thought exactly. My word would be "Gun" or "Artillery". easily defeats his logic. But that should also explain us why pure embeddings search is not sufficient for RAG.
canjobear 1y ago
I think the point is that, despite the game being massively logically underspecified, people are still able to home in on an answer in practice. Which shows that they have a shared fuzzy semantic space.

gerdesj 1y ago

The devil is in the details. I recently gave the "Veeam Intelligence" a spin. Veeam is a backup system spanning quite a lot of IT systems with a lot of options - it is quite complicated but it is also a bounded domain - the app does as the app does. It is very mature and has extremely good technical documentation and a massive amount of technical information docs (TIDs) and a vibrant and very well informed set of web forums, staffed by ... staff and even the likes of Anton Gostev - https://www.veeam.com/company/management-team.html https://www.veeam.com/company/management-team.html Surely they have close to the perfect data set to train on? I asked a question about moving existing VMware replicas from one datastore to another and how to keep my replication jobs working correctly. In this field, you may not be familiar with my particular requirements but this is not a niche issue. The "VI" came up with a reasonable sounding answer involving a wizard. I hunted around the GUI looking for it (I had actually used that wizard a while back). So I asked where it was and was given directions. It wasn't there. The wizard was genuine but its usage here was a hallucination. A human might have done the same thing with some half remembered knowledge but would soon fix that with the docs or the app itself. I will stick to reading the docs. They are really well written and I am reasonably proficient in this field so actually - a decent index is all I need to get a job done. I might get some of my staff to play with this thing when given a few tasks that they are unfamiliar with and see what it comes up with. I am sure that domain specific LLMs are where it is at but we need some sort of efficient "fact checker" system.

ieie3366 1y ago

LLMs are bruteforce reverse engineered human brains. Think about it. Any written text out there is written by human brains. The ”function” to output this is whatever happens inside the brain, insanely complex. LLM ”training” is just brute forcing the same function into existence. ”Human brain output X, llm output Y, mutate it times billion until X and Y start matching”

tgsovlerkhgsel 1y ago

I've noticed that many of the large, separately developed AIs often answer with remarkably similar wording to the same question.

lloeki 1y ago
I'm not exactly surprised that "separately developed" AI that hinge on essentially the same core techniques and are fed largely identical corpus often answer with remarkably similar wording to the same question.
- imtringued 1y ago
  I find it grating that the article is sweeping this under the rug. If you train your AI on Shakespeare it won't suddenly know how to control a robot. Meanwhile all Shakespeare bots will be remarkably similar. Coincidence?
- krackers 1y ago
  At least for LLMs, the platonic representation hypothesis doesn't seem that surprising. The training data for models that are large enough to faithfully capture reality will necessarily have a lot of overlap. So in this weak form the platonic representation hypothesis just says that if you provide the roughly same data to models with roughly the same architecture, they learn roughly the same embeddings. This doesn't seem that surprising. I'd even expect a slightly stronger version where you feed the same data to different models to still hold. After all the modes capture the same reality, so there's probably some way to transform from the "latent space" of one model to the other.

kindkang2024 1y ago

The Dao can be spoken of, yet what is spoken is not the eternal Dao. So, what is the Dao? Personally, I see it as will — something we humans could express through words. For any given will, even though we use different words in different languages — Chinese, Japanese, English — these are simply different representations of the same will. Large language models learn from word tokens and begin to grasp these wills — and in doing so, they become the Dao. In that sense, I agree: “All AI models might be the same.”

foxes 1y ago

So in the limit the models representation space has one dimension per "concept" or something, but making it couple things together is what actually makes it useful? An infinite dimensional model with just one dim per concept would be sorta useless, but you need things tied together?

IAmNotACellist 1y ago

I agree LLMs are converging on a current representation of reality based on the collective works of humanity. What we need to do is provide AIs with realtime sensory input, simulated hormones each with their own half-lifes based on metabolic conditions and energy usage, a constant thinking loop, and discover a synthetic psilocybin that's capable of causing creative, cross-neural connections similar to human brains. We have the stoned ape theory, we need the stoned AI theory.

buffet_overflow 1y ago
Or perhaps we make them attractions at a theme park, but let Anthony Hopkins have admin access to the source code. What could go wrong?
chamomeal 1y ago
I’m so incredibly tired of reading about AI but if I saw “we gave an AI shrooms” I’d be all over it
- josh-sematic 1y ago
  I imagine the temperature parameter is the closest thing we have to that right now. Turn it up a little to make things a bit more creative. Turn it up more and eventually it will wildly swing around in its dialogue. Even more and it eventually produces utter gibberish.

lsy 1y ago

The example given for inverting an embedding back to text doesn't help the idea that this effect is reflecting some "shared statistical model of reality": What would be the plausible whalesong mapping of "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby"? There isn't anything core to reality about Kentucky, its Derby, the Gregorian calendar, America, horse breeds, etc. These are all cultural inventions that happen to have particular importance in global human culture because of accidents of history, and are well-attested in training sets. At best we are seeing some statistical convergence on training sets because everyone is training on the same pile and scraping the barrel for any differences.

benlivengood 1y ago
You also can't translate "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby" into Hellenistic Greek or some modern indigenous languages because there isn't enough shared context; you'd need to give humans speaking those languages a glossary for any of the translation to make sense, or allow them to interrogate an LLM to act as the glossary. I'd say our current largest LLMs probably contain sufficient detail to explain a concept like a named race horse starting from QCD+gravity and ending up at cultural human events, given a foothold of some common ground to translate into a new unknown language. In a sense, that's what a model of reality is. I think it's possible because LLMs figure out translation between human languages by default with enough pretraining.
- photonthug 1y ago
  > You also can't translate .. into Hellenistic Greek or some modern indigenous languages because there isn't enough shared context; you'd need to give humans speaking those languages a glossary for any of the translation to make sense What? By substitution, this means you can translate it. As long as we're assuming a large enough basis of concept vectors of course it works. > I'd say our current largest LLMs probably contain sufficient detail to explain a concept like a named race horse starting from QCD+gravity and ending up at cultural human events What? I'm curious how you'd propose to move from gravity to culture. This is like TFAs assertion that the M+B game might be as expressive as 20 questions / universal. M or B is just (bad,sentient) or (good,object). Sure, entangling a few concepts is slightly more expressive than playing with a completely flattened world of 1/0 due to some increase in dimensionality. But trying to pinpoint anything like (neutral,concept) fails because the basis set isn't fundamentally large enough. Any explanation of how this could work will have to cheat, like when TFA smuggles new information in by communicating details about distance-from-basis. For example to really get to the word or concept of "neutral" from inferred good/bad dichotomy of bread/mussolini, you would have to answer "Hmmmmmm, closer to bread I guess" in one iteration and then "Umm.. closer to Mussolini I guess" when asked again, and then have the interrogator notice the uncertainty/hesitation/contradiction and then infer neutrality. This is just the simple case.. physics to culture seems much harder

stillpointlab 1y ago

I have to be careful of confirmation bias when I read stuff like this because I have the intuition that we are uncovering a single intelligence with each of the different LLMs. I even feel, when switching between the big three (OpenAI, Google, Anthropic) that there is a lot of similarity in how they speak and think - but I am aware of my bias so I try not to let it cloud my judgement. On the topic of compression, I am reminded of an anecdote about Heidegger. Apparently he had a bias towards German and Greek, claiming that these languages were the only suitable forms for philosophy. His claim was based on the "puns" in language, or homonyms. He had some intuition that deep truths about reality were hidden in these over-loaded words, and that the particular puns in German and Greek were essential to understand the most fundamental philosophical ideas. This feels similar to the idea of shared embeddings being a critical aspect of LLM emergent intelligence. This "superposition" of meaning in representation space again aligns with my intuitions. I'm glad there are people seriously studying this.

b112 1y ago
LLMs don't think, nor are they intelligent or exhibiting intelligence. Language does have constraints, yet it evolves via its users to encompass new meanings. Thus those constraints are artificial, unless you artificially enforce static language use. And of course, for an LLM to use those new concepts, it needs to be retokenized by being trained on new data. For example, if we trained LLMs only on books, encyclopedias, newpapers, and personal letters from 1850, it would have zero capacity to speak comprehensibly or even seem cogent on much of the modern world. And it would forever remain in that disconnected positon. LLMs do not think, understand anything, nor learn. Should you wish to call tokenization, learning, then you'd better call a clock "learning" from the gears and cogs that enable its function. LLMs do not think, learn, or exhibit intelligence. (I feel this is not said enough). We will never, ever get AGI from an LLM. Ever. I am sympathetic to the wonder of LLMs. To seeing them as such. But I see some art as wonderous too. Some machinery is beautiful in execution and to use. But that doesn't change truths.
- kevin42 1y ago
  You made a lot of bold assertions there. It's as if you have a complete and definitive theory of human intelligence to compare it against. Which if true, would be incredible, because there isn't a scientifically accepted theory, nor is there consensus from a philosophical standpoint. I can't say that you are wrong, you might be right, especially about AGI. And I think it's unlikely that LLMs are the direct path to AGI. But, just looking at how human brains work, it seems unlikely that we would be intelligent either if we used your same reductionist logic. An individual neuron doesn't "think" or "understand" anything. It's a biological cell that simply fires an electrochemical signal when its input threshold is met. It has no understanding of language or context. By your logic, since the fundamental components are just simple signal processors, the brain cannot possibly learn or be intelligent. Yet, from the complex interaction of ~86 billion of these simple biochemical machines, the emergent properties of thought, understanding, and consciousness arise. Dismissing an LLM's capabilities because its underlying operations are basically just math operating on tokenized data is like dismissing human consciousness because it's "just electrochemistry" in a network of cells. Both arguments mistake the low-level mechanism for the high-level emergent phenomenon.

megaloblasto 1y ago

Really cool idea that I hadn't considered yet. If true, seems like a big plus for open source and not having a few companies controll all the models. If they all converge to the same intelligence, one open source model would make all proprietary models obsolete.

seeknotfind 1y ago

Platonism - not even once. Green is the smell of my grandmother's lawn on a hot summer day. Just because things are similar to a lot of people doesn't mean their fundamentally the same.

darepublic 1y ago

Why is claude shannon closer to mussolini than to david beckham?

robertclaus 1y ago

The board game DaDaDa is a great example of this.

stevage 1y ago

I tried playing Mussolini or Bread with ChatGPT, but it didn't go very well. It seemed to have trouble grasping the rules, and kept getting overly specific when we were miles from the right concept.

deleted 1y ago

[deleted]

amoss 1y ago

When I read that "Ilya gave a famously incomprehensible talk about the connections between intelligence and compression" it makes me wonder if Marcus Hutter has now been forgotten? If so more people should take a look at http://prize.hutter1.net/ http://prize.hutter1.net/

zoeey 1y ago

[dead]

tempfile 1y ago

I know it's only one tiny part, but I can't resist: > Is it closer to Mussolini or David Beckham? Uhh, I guess Mussolini. (Ok, they’re definitely thinking of a person.) This deduction is absurd! The only information you have at all is that it's more like a person than it is like bread (which could be almost anything).

derbOac 1y ago

Seemed to me they are just saying that typical LLMs are consistent: https://en.m.wikipedia.org/wiki/Consistent_estimator https://en.m.wikipedia.org/wiki/Consistent_estimator

pona-a 1y ago

I think some of this effect is explained by different models being trained on almost identical corpa. Even if you try to erase certain works, contamination will still represent some of it, just in less obvious ways.

throwpoaster 1y ago

I asked Grok, o3-pro, and Claude a question about piezoelectric effects. They all got it "right", but Claude called out a second order effect that arose from the use case that the other two missed. I get it, they might all be exploring the same space, but Claude went an extra, important, hop.

Rover222 1y ago
I'm just curious if that was Grok 4 or 3
- throwpoaster 1y ago
  3! So it’s not a fair comparison. The other two were both paid. I am very curious to run real world engineering problems through SuperHeavy.
  - Rover222 1y ago
    Have you tried SuperHeavy yet? It seems to be crushing on almost any measurable benchmark. I've only used Grok 4 (not heavy) when stuck on a problem that claude/cursor cannot get past, and Grok 4 is often able to come up with an elegant solution.

antonvs 1y ago

> One explanation for why this game works is that there is only one way in which things are related, and this comes from the underlying world we live in. I think what this might be trying to say is something more like: ...there are many ways in which things can be related, but those relationships come from the underlying world we live in. I.e. there are obviously many ways in which things can be related, but if we assume the quote is not entirely counterfactual, then it must be getting at something else. I suppose "way" here is being used in a different sense, but it isn't clear.

cultofmetatron 1y ago

someone just discovered the AI equivalent of "everything turns to crabs"

d_burfoot 1y ago

For those who don't know about Plato's Theory of Forms, the remarkable empirical fact is that all humans learn roughly the same concepts representing words like "dog", "house", "person", "boat", and so on, even if those people grew up in different places and never had any overlap between their observational experience. You and I may never have observed the same dog, but we still agree on what a "dog" is. This phenomenon appears to occur for LLM learning as well, though it is less remarkable due to the fact that LLMs likely have significant overlap in their training data. I believe this is good news for Alignment, because, as Plato pointed out, one of the most important forms is the Form of the Good - a (theoretical) universal human ideal containing notions of justice, virtue, compassion, excellence, etc. If this Form truly exists, and LLMs can learn it, it may be possible to train them to pursue it (or refuse requests that are opposed to it).

wagwang 1y ago
Basically... Jungian archetypes
hobbyjogger 1y ago
This vastly oversimplifies the notion of what is "good" or "fair" in a given situation. Just because we can generally agree on classifying simple objects like a boat or a house doesn't entail that we share a universal notion about the virtuous thing to do in a given situation. Humans (even in the same society/environment) have wildly differing ideas about moral issues like abortion, assisted suicide, medical testing on animals/stem cells, etc. Consider this chart from a 2010 Gallup Poll: https://content.gallup.com/origin/gallupinc/GallupSpaces/Production/Cms/POLL/ztybao6qoegc_-aam95srw.gif https://content.gallup.com/origin/gallupinc/GallupSpaces/Pro...
- reactordev 1y ago
  Ahh, semantic search problem rears its head once more. While we can agree on what a "dog" is. We disagree on it's value.
pphysch 1y ago
> all humans learn roughly the same concepts representing words like "dog", "house", "person", "boat", and so on, even if those people grew up in different places and never had any overlap between their observational experience. This "roughly" is doing a lot of heavy lifting to support the "Plato was right" thesis. The fact is we inhabit a shared reality with shared laws of physics and evolutionary pressures and so on; there are only so many ways to float a boat. That doesn't necessarily mean Platonic Forms exist and that everyone arrives independently at the same Form. Indeed you will find that people have fundamentally different ideas of what words like "freedom", "economy", "government" mean, despite using the same syntax.