8 ms·

If you're a scientist who works in protein folding (or one of those other areas) and strongly believe that science's goal is to produce falsifiable hypotheses,

by dekhn 3y ago

If you're a scientist who works in protein folding (or one of those other areas) and strongly believe that science's goal is to produce falsifiable hypotheses, these new approaches will be extremely depressing, especially if you aren't proficient enough with ML to reproduce this work in your own hands. If you're a scientist who accepts that probabilist models beat interpretable ones (articulated well here: https://norvig.com/chomsky.html https://norvig.com/chomsky.html), then you'll be quite happy because this is yet another validation of the value of statistical approaches in moving our ability to predict the universe forward. If you're the sort of person who believes that human brains are capable of understanding the "why" of how things work in all its true detail, you'll find this an interesting challenge- can we actually interpret these models, or are human brains too feeble to understand complex systems without sophisticated models? If you're the sort of person who likes simple models with as few parameters as possible, you're probably excited because developing more comprehensible or interpretable models that have equivalent predictive ability is a very attractive research subject. (FWIW, I'm in the camp of "we should simultaneously seek simpler, more interpretable models, while also seeking to improve native human intelligence using computational augmentation")

narrator 3y ago
What if our understanding of the laws of the natural sciences are subtly flawed and AI just corrects perfectly for our flawed understanding without telling us what the error in our theory was? Forget trying to understand dark matter. Just use this model to correct for how the universe works. What is actually wrong with our current model and if dark matter exists or not or something else is causing things doesn't matter. "Shut up and calculate" becomes "Shut up and do inference."
- dekhn 3y ago
  All models are wrong, but some models are useful.
  - narrator 3y ago
    The black box AI models could calculate epicycles perfectly so the middle ages Catholic Church could say just use those instead of being a geocentrrism denier.
- RandomLensman 3y ago
  High accuracy could result from pretty incorrect models. When and where that woukd then go completely off the rails is difficult to say.
- visarga 3y ago
  ML is accustomed with the idea that all models are bad, and there are ways to test how good or bad they are. It's all approximations and imperfect representations, but they can be good enough for some applications. If you think carefully humans operate in the same regime. Our concepts are all like that - imperfect, approximative, glossing over some details. Our fundamental grounding and test is survival, an unforgiving filter, but lax enough to allow for anti-vaxxer movements during the pandemic - survival test is not testing for truth directly, only for ideas that fail to support life.
  - mistermann 3y ago
    Also lax enough for the hilarious mismanagement of the situation by "the experts". At least anti-vaxxers have an excuse.
- coffeebeqn 3y ago
  Wouldn’t learning new data and results give us more hints to the true meaning of the thing? I fail to see how this is a bad thing in anyone’s eye.
croniev 3y ago
I'm in the following camp: It is wrong to think about the world or the models as "complex systems" that may or may not be understood by human intelligence. There is no meaning beyond that which is created by humans. There is no 'truth' that we can grasp in parts but not entirely. Being unable to understand these complex systems means that we have framed them in such a way (f.e. millions of matrix operations) that does not allow for our symbol-based, causal reasoning mode. That is on us, not our capabilities or the universe. All our theories are built on observation, so these empirical models yielding such useful results is a great thing - it satisfies the need for observing and acting. Missing explainability of the models merely means we have less ability to act more precisely - but it does not devalue our ability to act coarsely.
- visarga 3y ago
  But the human brain has limited working memory and experience. Even in software development we are often teetering at the edge of the mental power to grasp and relate ideas. We have tried so much to manage complexity, but real world complexity doesn't care about human capabilities. So there might be high dimensional problems where we simply can't use our brains directly.
  - jvanderbot 3y ago
    A human mind is perfectly capable of following the same instructions as the computer did. Computers are stupidly simple and completely deterministic. The concern is about "holding it all in your head", and depending on your preferred level of abstraction, "all" can perfectly reasonably be held in your head. For example: "This program generates the most likely outputs" makes perfect sense to me, even if I don't understand some of the code. I understand the system. Programmers went through this decades ago. Physicists had to do it too. Now, chemists I suppose.
    - GenerocUsername 3y ago
      This is just wrong. While computer operations in solutions are computable by humans, the billions of rapid computations are unachievable by humans. In just a few seconds, a computer can perform more basic arithmetic operations than a human could in a lifetime.
deleted 3y ago
[deleted]
divbzero 3y ago
There have been times in the past when usable technology surpassed our scientific understanding, and instead of being depressing it provided a map for scientific exploration. For example, the steam engine was developed by engineers in the 1600s/1700s (Savery, Newcomen, and others) but thermodynamics wasn’t developed by scientists until the 1800s (Carnot, Rankine, and others).
- jprete 3y ago
  I think the various contributors to the invention of the steam engine had a good idea of what they were trying to do and how their idea would physically work. Wikipedia lists the prerequisites as the concepts of a vacuum and pressure, methods for creating a vacuum and generating steam, and the piston and cylinder.
  - exe34 3y ago
    That's not too different from the alpha fold people knowing that there's a sequence to sequence translation, that an enormous number of cross-talk happens between the parts of the molecule, that if you get the potential fields just right, it'll fold in the way nature intended. They're not just blindly fiddling with a bunch of levers. What they don't know is the individual detailed interactions going on and how to approximate them with analytical equations.
interroboink 3y ago
> ... and strongly believe that science's goal is to produce falsifiable hypotheses, these new approaches will be extremely depressing I don't quite understand this point — could you elaborate? My understanding is that the ML model produces a hypothesis, which can then be tested via normal scientific method (perform experiment, observe results). If we have a magic oracle that says "try this, it will work", and then we try it, and it works, we still got something falsifiable out of it. Or is your point that we won't necessarily have a coherent/elegant explanation for why it works?
- dekhn 3y ago
  People will be depressed because they spent decades getting into professorship positions and publishing papers with ostensible comprehensible interpretations of the generative processes that produced their observations, only to be "beat" in the game by a system that processed a lot of observations and can make predicts in a way that no individual human could comprehend. And those professors will have a harder time publishing, and therefore getting promoted, in the future. Whether ML models produce hypotheses is something of an epistemiological argument that I think muddies the waters without bringing any light. I would only use the term "ML models generate predictions". In a sense, the model itself is the hypothesis, not any individual prediction.
- variadix 3y ago
  There is an issue scientifically. I think this point was expressed by Feynman: the goal of scientific theories isn’t just to make better predictions, it’s to inform us about how and why the world works. Many ancient civilizations could accurately predict the position of celestial bodies with calendars derived from observations of their period, but it wasn’t until Copernicus proposed the heliocentric model and Galileo provided supporting observations that we understood the why and how, and that really matters for future progress and understanding.
  - interroboink 3y ago
    I agree the how/why is the main driving goal. That's kinda why I feel like this is not depressiong news — there's a new frontier to discover and attempt to explain. Scientists love that stuff (: Knowing how to predict the motion of planets but without having an underlying explanation encourages scientists to develop their theories. Now, once more, we know how to predict something (protein folding) but without an underlying explanation. Hurray, something to investigate! (Aside: I realize that there are also more human factors at play, and upsetting the status quo will always cause some grief. I just wanted to provide a counterpoint that there is some exciting progress represented here, too).
coffeemug 3y ago
> If you're the sort of person who believes that human brains are capable of understanding the "why" of how things work in all its true detail This seems to me an empirical question about the world. It’s clear our minds are limited, and we understand complex phenomena through abstraction. So either we discover we can continue converting advanced models to simpler abstractions we can understand, or that’s impossible. Either way, it’s something we’ll find out and will have to live with in the coming decades. If it turns out further abstractions aren’t possible, well, enlightenment thought had lasted long enough. It’s exciting to live at a time in humanity’s history when we enter a totally uncharted new paradigm.
jprete 3y ago
The goal of science has always been to discover underlying principles and not merely to predict the outcome of experiments. I don't see any way to classify an opaque ML model as a scientific artifact since by definition it can't reveal the underlying principles. Maybe one could claim the ML model itself is the scientist and everyone else is just feeding it data. I doubt human scientists would be comfortable with that, but if they aren't trying to explain anything, what are they even doing?
- fire_lake 3y ago
  What if the underlying principles of the universe are too complex for human understanding but we can train a model that very closely follows them?
  - dekhn 3y ago
    Then we should dedicate large fractions of human engineering towards finding ethical ways to improve human intelligence so that we can appreciate the underlying principles better.
    - refulgentis 3y ago
      I spend about 30 minutes reading this thread and links from it: I don't really follow your line of argument. I find it fascinating and well-communicated, the lack of understanding is on me: my attention flits around like a butterfly, in a way that makes it hard for me to follow people writing original content. High level, I see a distinction between theory and practice, between an oracle predicting without explanation, and a well-thought out theory built on a partnership between theory and experiment over centuries, ex. gravity. I have this feeling I can't shake that the knife you're using is too sharp, both in the specific example we're discussing, and in general. In the specific example, folding, my understanding is we know how proteins fold & the mechanisms at work. It just takes an ungodly amount of time to compute and you'd still confirm with reality anyway. I might be completely wrong on that. Given that, the proposal to "dedicate...engineer[s] towards finding ethical ways to improve...intelligence so that we can appreciate the underlying principles better" begs the question of if we're not appreciating the underlying principles. It feels like a close cousin of physics theory/experimentalist debate pre-LHC, circa 2006: the experimentalists wanted more focus on building colliders or new experimental methods, and at the extremes, thought string theory was a complete was of time. Which was working towards appreciating the underlying principles? I don't really know. I'm not sure there's a strong divide between the work of recording reality and explaining it. I'll peer into a microscope in the afternoon, and take a shower in the evening, and all of a sudden, free associating gives me a more high-minded explanation for what I saw. I'm not sure a distinction exists for protein folding, yes, I'm virtually certain this distinction does not exist in reality, only in extremely stilted examples (i.e. a very successful oracle at Delphi)
pishpash 3y ago
So the work to simplify ML models, reduce dimensions, etc. becomes the numeric way to seek simple actual scientific models. Scientific computing and science become one.
ThomPete 3y ago
The goal of science should always be to seek good explanations hard to vary.
RajT88 3y ago
> can we actually interpret these models, or are human brains too feeble to understand complex systems without sophisticated models? I think we will have to develop a methodology and supporting toolset to be able to derive the underlying patterns driving such ML models. It's just too much for a human to comb through by themselves and make sense of.
deleted 3y ago
[deleted]
ggm 3y ago
For some, this conversation started when the machine derived four colour map proof was announced which is almost 5 decades ago in 1976
cynicalkane 3y ago
What always struck me about Chomskyists is that they chose a notion of interpretable model that required unrealistic amounts of working interpretation. So Chomsky grammars have significant polynomial memory and computational costs for grammars as they approach something resembling human grammar. And you say, ok, the human brain can handle much more computation than that, and that's fine. But (for example) context-free grammars aren't just O(n^3) in computational cost; for a realistic description of human language they're O(n^3) in human-interpretable rules. Other Chomsky-like models of human grammars have different asymptotic behavior and different choices of n, but the same fundamental problem; the big-O constant factor isn't neurons firing but rather human connections between the n inputs. How can you conceive of human minds being able to track O(n^3) (or whatever) cost where that n is everything being communicated -- words, concepts, symbols, representations, all that jazz and the polynomial relationships between them? But I feel an apology is in order: I've had quite a few beers before coming home, and it's probably a mistake to try to express academically charged and difficult views on the Internet while in an inebriated state. Probably the alcohol has substantially decreased my mental computational power. However, it has only mildly impaired my ability to string together words and sentences in a grammatically complex fashion. In fact, I often feel that the more sober and clear-minded I am, the simpler my language is. Maybe human grammar is actually sub-polynomial. I have observed the same in ChatGPT; the more flowery and wordy it has become over time, the dumber its output.
- dekhn 3y ago
  There is a ballmer peak for pontificating. As an aside but relevant to your point, my entire introduction to DNA and protein analysis was based on Chomsky grammars. My undergrad thesis advisor David Haussler handed me a copy of an article by David Searls "The Linguistics of DNA" (https://www.scribd.com/document/461974005/The-Linguistics-of-DNA https://www.scribd.com/document/461974005/The-Linguistics-of...) . At the time, Haussler was in the middle of applying HMMs and other probabilistic graphical models to sequence analysis, and I knew all about DNA as a molecule, but not how to analyze it. Searls paper basically walks through Chomsky's hierarchy, and how to apply it, using linguistic techniques to "parse" DNA. It was mind-bending and mind-expanding for me (it takes me a long time to read papers, for example I think I read this paper over several months, learning to deal with parsing along the way). To this day I am astounded at how much those approaches (linguistics, parsing, and grammars) have evolved- and yet not much has changed! People were talking about generative models in the 90s (and earlier) in much the same way we treat LLMs today. While much of Chomsky's thinking on how to make real-world language models isn't particuarly relevant, we still are very deeply dependent on his ideas for grammar... Anyway, back to your point. While CFGs may be O(n*3) I would say that there is a implicit, latent O(n) parseable grammar underlying human linguistics, and our brains can map that latent space to its own internal representation in O(1) time, where the n roughly correlates to the complexity of the idea being transferred. It does not seem even remotely surprising that we can make multi-language models that develop their own compact internal representation that is presumably equidistant from each source language.
stouset 3y ago
> If you're the sort of person who believes that human brains are capable of understanding the "why" of how things work in all its true detail, you'll find this an interesting challenge- can we actually interpret these models, or are human brains too feeble to understand complex systems without sophisticated models? I think chess engines, weirdly enough, have disabused me of this notion. There are lots of factors a human considers when looking at a board. Piece activity. Bishop and knight imbalances. King safety. Open and semi-open file control. Tempo. And on and on. But all of them are just convenient shortcuts that allow us to substitute reasonable guesses for what really matters: exhaustively calculating a winning line through to the end. “Positional play” is a model that only matters when you can’t calculate trillions of lines thirty moves deep, and it’s infinitely more important that a move survives your opponent’s best possible responses than it is to satisfy some cohesive higher level principle.
- searealist 3y ago
  Chess engines actually do both now. They have ML models to evaluate positions, essentially a much more advanced version of your positional description, and deep calculations.
  - stouset 3y ago
    That might be the best we can practically achieve with technology, but the point stands. If positional evaluation says one thing but an exhaustive analysis of lines finds a solution 60 moves deep, that one is going to win.
    - searealist 3y ago
      Humans also do search. Also, engines arent doing an exhaustive search when they are 20 moves deep. They heavily prune.
      - stouset 3y ago
        Yes, I understand how chess engines work. Ignore the existence of engines for a moment. The reason a particular line works, at the end of the day, is simply because it does. Just because we have heuristics that help us skip a lot of evaluation doesn’t mean the heuristics have intrinsic meaning within the game. They don’t. They’re shortcuts that let us skip having to do the impossible. The heuristics will always lose to concrete analysis to a deep enough depth. And that’s my point. We come up with models that give us an intuition for “why” things are a certain way. Those models are inarguably helpful toward having a gut feeling. But models aren’t the thing itself, and every model we’ve found breaks down at some deeper point. And maybe at some level things simply “are” some way with no convenient shorthand explanation.
bamboozled 3y ago
Do you think a model will also be able to truly comprehend everything too ?

by dekhn 3y ago

narrator 3y ago

What if our understanding of the laws of the natural sciences are subtly flawed and AI just corrects perfectly for our flawed understanding without telling us what the error in our theory was? Forget trying to understand dark matter. Just use this model to correct for how the universe works. What is actually wrong with our current model and if dark matter exists or not or something else is causing things doesn't matter. "Shut up and calculate" becomes "Shut up and do inference."

dekhn 3y ago
All models are wrong, but some models are useful.
- narrator 3y ago
  The black box AI models could calculate epicycles perfectly so the middle ages Catholic Church could say just use those instead of being a geocentrrism denier.
RandomLensman 3y ago
High accuracy could result from pretty incorrect models. When and where that woukd then go completely off the rails is difficult to say.
visarga 3y ago
ML is accustomed with the idea that all models are bad, and there are ways to test how good or bad they are. It's all approximations and imperfect representations, but they can be good enough for some applications. If you think carefully humans operate in the same regime. Our concepts are all like that - imperfect, approximative, glossing over some details. Our fundamental grounding and test is survival, an unforgiving filter, but lax enough to allow for anti-vaxxer movements during the pandemic - survival test is not testing for truth directly, only for ideas that fail to support life.
- mistermann 3y ago
  Also lax enough for the hilarious mismanagement of the situation by "the experts". At least anti-vaxxers have an excuse.
coffeebeqn 3y ago
Wouldn’t learning new data and results give us more hints to the true meaning of the thing? I fail to see how this is a bad thing in anyone’s eye.

croniev 3y ago

I'm in the following camp: It is wrong to think about the world or the models as "complex systems" that may or may not be understood by human intelligence. There is no meaning beyond that which is created by humans. There is no 'truth' that we can grasp in parts but not entirely. Being unable to understand these complex systems means that we have framed them in such a way (f.e. millions of matrix operations) that does not allow for our symbol-based, causal reasoning mode. That is on us, not our capabilities or the universe. All our theories are built on observation, so these empirical models yielding such useful results is a great thing - it satisfies the need for observing and acting. Missing explainability of the models merely means we have less ability to act more precisely - but it does not devalue our ability to act coarsely.

visarga 3y ago
But the human brain has limited working memory and experience. Even in software development we are often teetering at the edge of the mental power to grasp and relate ideas. We have tried so much to manage complexity, but real world complexity doesn't care about human capabilities. So there might be high dimensional problems where we simply can't use our brains directly.
- jvanderbot 3y ago
  A human mind is perfectly capable of following the same instructions as the computer did. Computers are stupidly simple and completely deterministic. The concern is about "holding it all in your head", and depending on your preferred level of abstraction, "all" can perfectly reasonably be held in your head. For example: "This program generates the most likely outputs" makes perfect sense to me, even if I don't understand some of the code. I understand the system. Programmers went through this decades ago. Physicists had to do it too. Now, chemists I suppose.
  - GenerocUsername 3y ago
    This is just wrong. While computer operations in solutions are computable by humans, the billions of rapid computations are unachievable by humans. In just a few seconds, a computer can perform more basic arithmetic operations than a human could in a lifetime.

deleted 3y ago

[deleted]

divbzero 3y ago

There have been times in the past when usable technology surpassed our scientific understanding, and instead of being depressing it provided a map for scientific exploration. For example, the steam engine was developed by engineers in the 1600s/1700s (Savery, Newcomen, and others) but thermodynamics wasn’t developed by scientists until the 1800s (Carnot, Rankine, and others).

jprete 3y ago
I think the various contributors to the invention of the steam engine had a good idea of what they were trying to do and how their idea would physically work. Wikipedia lists the prerequisites as the concepts of a vacuum and pressure, methods for creating a vacuum and generating steam, and the piston and cylinder.
- exe34 3y ago
  That's not too different from the alpha fold people knowing that there's a sequence to sequence translation, that an enormous number of cross-talk happens between the parts of the molecule, that if you get the potential fields just right, it'll fold in the way nature intended. They're not just blindly fiddling with a bunch of levers. What they don't know is the individual detailed interactions going on and how to approximate them with analytical equations.

interroboink 3y ago

> ... and strongly believe that science's goal is to produce falsifiable hypotheses, these new approaches will be extremely depressing I don't quite understand this point — could you elaborate? My understanding is that the ML model produces a hypothesis, which can then be tested via normal scientific method (perform experiment, observe results). If we have a magic oracle that says "try this, it will work", and then we try it, and it works, we still got something falsifiable out of it. Or is your point that we won't necessarily have a coherent/elegant explanation for why it works?

dekhn 3y ago
People will be depressed because they spent decades getting into professorship positions and publishing papers with ostensible comprehensible interpretations of the generative processes that produced their observations, only to be "beat" in the game by a system that processed a lot of observations and can make predicts in a way that no individual human could comprehend. And those professors will have a harder time publishing, and therefore getting promoted, in the future. Whether ML models produce hypotheses is something of an epistemiological argument that I think muddies the waters without bringing any light. I would only use the term "ML models generate predictions". In a sense, the model itself is the hypothesis, not any individual prediction.
variadix 3y ago
There is an issue scientifically. I think this point was expressed by Feynman: the goal of scientific theories isn’t just to make better predictions, it’s to inform us about how and why the world works. Many ancient civilizations could accurately predict the position of celestial bodies with calendars derived from observations of their period, but it wasn’t until Copernicus proposed the heliocentric model and Galileo provided supporting observations that we understood the why and how, and that really matters for future progress and understanding.
- interroboink 3y ago
  I agree the how/why is the main driving goal. That's kinda why I feel like this is not depressiong news — there's a new frontier to discover and attempt to explain. Scientists love that stuff (: Knowing how to predict the motion of planets but without having an underlying explanation encourages scientists to develop their theories. Now, once more, we know how to predict something (protein folding) but without an underlying explanation. Hurray, something to investigate! (Aside: I realize that there are also more human factors at play, and upsetting the status quo will always cause some grief. I just wanted to provide a counterpoint that there is some exciting progress represented here, too).

coffeemug 3y ago

> If you're the sort of person who believes that human brains are capable of understanding the "why" of how things work in all its true detail This seems to me an empirical question about the world. It’s clear our minds are limited, and we understand complex phenomena through abstraction. So either we discover we can continue converting advanced models to simpler abstractions we can understand, or that’s impossible. Either way, it’s something we’ll find out and will have to live with in the coming decades. If it turns out further abstractions aren’t possible, well, enlightenment thought had lasted long enough. It’s exciting to live at a time in humanity’s history when we enter a totally uncharted new paradigm.

jprete 3y ago

The goal of science has always been to discover underlying principles and not merely to predict the outcome of experiments. I don't see any way to classify an opaque ML model as a scientific artifact since by definition it can't reveal the underlying principles. Maybe one could claim the ML model itself is the scientist and everyone else is just feeding it data. I doubt human scientists would be comfortable with that, but if they aren't trying to explain anything, what are they even doing?

fire_lake 3y ago
What if the underlying principles of the universe are too complex for human understanding but we can train a model that very closely follows them?
- dekhn 3y ago
  Then we should dedicate large fractions of human engineering towards finding ethical ways to improve human intelligence so that we can appreciate the underlying principles better.
  - refulgentis 3y ago
    I spend about 30 minutes reading this thread and links from it: I don't really follow your line of argument. I find it fascinating and well-communicated, the lack of understanding is on me: my attention flits around like a butterfly, in a way that makes it hard for me to follow people writing original content. High level, I see a distinction between theory and practice, between an oracle predicting without explanation, and a well-thought out theory built on a partnership between theory and experiment over centuries, ex. gravity. I have this feeling I can't shake that the knife you're using is too sharp, both in the specific example we're discussing, and in general. In the specific example, folding, my understanding is we know how proteins fold & the mechanisms at work. It just takes an ungodly amount of time to compute and you'd still confirm with reality anyway. I might be completely wrong on that. Given that, the proposal to "dedicate...engineer[s] towards finding ethical ways to improve...intelligence so that we can appreciate the underlying principles better" begs the question of if we're not appreciating the underlying principles. It feels like a close cousin of physics theory/experimentalist debate pre-LHC, circa 2006: the experimentalists wanted more focus on building colliders or new experimental methods, and at the extremes, thought string theory was a complete was of time. Which was working towards appreciating the underlying principles? I don't really know. I'm not sure there's a strong divide between the work of recording reality and explaining it. I'll peer into a microscope in the afternoon, and take a shower in the evening, and all of a sudden, free associating gives me a more high-minded explanation for what I saw. I'm not sure a distinction exists for protein folding, yes, I'm virtually certain this distinction does not exist in reality, only in extremely stilted examples (i.e. a very successful oracle at Delphi)

pishpash 3y ago

So the work to simplify ML models, reduce dimensions, etc. becomes the numeric way to seek simple actual scientific models. Scientific computing and science become one.

ThomPete 3y ago

The goal of science should always be to seek good explanations hard to vary.

RajT88 3y ago

> can we actually interpret these models, or are human brains too feeble to understand complex systems without sophisticated models? I think we will have to develop a methodology and supporting toolset to be able to derive the underlying patterns driving such ML models. It's just too much for a human to comb through by themselves and make sense of.

deleted 3y ago

[deleted]

ggm 3y ago

For some, this conversation started when the machine derived four colour map proof was announced which is almost 5 decades ago in 1976

cynicalkane 3y ago

What always struck me about Chomskyists is that they chose a notion of interpretable model that required unrealistic amounts of working interpretation. So Chomsky grammars have significant polynomial memory and computational costs for grammars as they approach something resembling human grammar. And you say, ok, the human brain can handle much more computation than that, and that's fine. But (for example) context-free grammars aren't just O(n^3) in computational cost; for a realistic description of human language they're O(n^3) in human-interpretable rules. Other Chomsky-like models of human grammars have different asymptotic behavior and different choices of n, but the same fundamental problem; the big-O constant factor isn't neurons firing but rather human connections between the n inputs. How can you conceive of human minds being able to track O(n^3) (or whatever) cost where that n is everything being communicated -- words, concepts, symbols, representations, all that jazz and the polynomial relationships between them? But I feel an apology is in order: I've had quite a few beers before coming home, and it's probably a mistake to try to express academically charged and difficult views on the Internet while in an inebriated state. Probably the alcohol has substantially decreased my mental computational power. However, it has only mildly impaired my ability to string together words and sentences in a grammatically complex fashion. In fact, I often feel that the more sober and clear-minded I am, the simpler my language is. Maybe human grammar is actually sub-polynomial. I have observed the same in ChatGPT; the more flowery and wordy it has become over time, the dumber its output.

dekhn 3y ago
There is a ballmer peak for pontificating. As an aside but relevant to your point, my entire introduction to DNA and protein analysis was based on Chomsky grammars. My undergrad thesis advisor David Haussler handed me a copy of an article by David Searls "The Linguistics of DNA" (https://www.scribd.com/document/461974005/The-Linguistics-of-DNA https://www.scribd.com/document/461974005/The-Linguistics-of...) . At the time, Haussler was in the middle of applying HMMs and other probabilistic graphical models to sequence analysis, and I knew all about DNA as a molecule, but not how to analyze it. Searls paper basically walks through Chomsky's hierarchy, and how to apply it, using linguistic techniques to "parse" DNA. It was mind-bending and mind-expanding for me (it takes me a long time to read papers, for example I think I read this paper over several months, learning to deal with parsing along the way). To this day I am astounded at how much those approaches (linguistics, parsing, and grammars) have evolved- and yet not much has changed! People were talking about generative models in the 90s (and earlier) in much the same way we treat LLMs today. While much of Chomsky's thinking on how to make real-world language models isn't particuarly relevant, we still are very deeply dependent on his ideas for grammar... Anyway, back to your point. While CFGs may be O(n*3) I would say that there is a implicit, latent O(n) parseable grammar underlying human linguistics, and our brains can map that latent space to its own internal representation in O(1) time, where the n roughly correlates to the complexity of the idea being transferred. It does not seem even remotely surprising that we can make multi-language models that develop their own compact internal representation that is presumably equidistant from each source language.

stouset 3y ago

> If you're the sort of person who believes that human brains are capable of understanding the "why" of how things work in all its true detail, you'll find this an interesting challenge- can we actually interpret these models, or are human brains too feeble to understand complex systems without sophisticated models? I think chess engines, weirdly enough, have disabused me of this notion. There are lots of factors a human considers when looking at a board. Piece activity. Bishop and knight imbalances. King safety. Open and semi-open file control. Tempo. And on and on. But all of them are just convenient shortcuts that allow us to substitute reasonable guesses for what really matters: exhaustively calculating a winning line through to the end. “Positional play” is a model that only matters when you can’t calculate trillions of lines thirty moves deep, and it’s infinitely more important that a move survives your opponent’s best possible responses than it is to satisfy some cohesive higher level principle.

searealist 3y ago
Chess engines actually do both now. They have ML models to evaluate positions, essentially a much more advanced version of your positional description, and deep calculations.
- stouset 3y ago
  That might be the best we can practically achieve with technology, but the point stands. If positional evaluation says one thing but an exhaustive analysis of lines finds a solution 60 moves deep, that one is going to win.
  - searealist 3y ago
    Humans also do search. Also, engines arent doing an exhaustive search when they are 20 moves deep. They heavily prune.
    - stouset 3y ago
      Yes, I understand how chess engines work. Ignore the existence of engines for a moment. The reason a particular line works, at the end of the day, is simply because it does. Just because we have heuristics that help us skip a lot of evaluation doesn’t mean the heuristics have intrinsic meaning within the game. They don’t. They’re shortcuts that let us skip having to do the impossible. The heuristics will always lose to concrete analysis to a deep enough depth. And that’s my point. We come up with models that give us an intuition for “why” things are a certain way. Those models are inarguably helpful toward having a gut feeling. But models aren’t the thing itself, and every model we’ve found breaks down at some deeper point. And maybe at some level things simply “are” some way with no convenient shorthand explanation.

bamboozled 3y ago

Do you think a model will also be able to truly comprehend everything too ?