3 ms·

I'm more or less a layperson when it comes to LLMs and this nascent concept of AI, but there's one argument that I keep seeing that I feel like I understand, ev

I'm more or less a layperson when it comes to LLMs and this nascent concept of AI, but there's one argument that I keep seeing that I feel like I understand, even without a thorough fluency with the underlying technology. I know that neural nets, and the mechanisms LLMs employ to train and form relational connections, can plausibly be compared to how synapses form signal paths between neurons. I can see how that makes intuitive sense. I'm struggling to articulate my cognitive dissonance here, but is there any empirical evidence that LLMs, or their underlying machine learning technology, share anything at all with biological consciousness beyond a convenient metaphor for describing "neural networks" using terms borrowed from neuroscience? I don't know that it necessarily follows that just because something was inspired by, or is somehow mimicking, the structure of the brain and its basic elements, that it should necessarily relate to its modeled reality in any literal way, let alone provide a sufficient basis for instantiating a phenomena we frankly know very little about. Not for nothing, but our models naturally cannot replicate any biological functions we do not fully understand. We haven't managed to reproduce biological tissues that are exponentially less complex than the brain, are we really claiming that we're just jumping straight past lab-grown t-bones to intelligent minds? I'm sure most of the people reading this will have seen Matt Parker's videos where they "teach" matchbooks to win a game against humans. Is anyone suggesting those matchbooks, given infinite time and repetition, would eventually spark emergent consciousness? > The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons. Sorry if I have misinterpreted anyone. I honestly thought all the "neuron" and "synapse" references were handy metaphors to explain otherwise complex computations that resemble this conceptual idea of how our brains work. But it reads a lot like some of the folks in this thread believe it's much more than metaphors, but rather a literal analog.

obirunda 2y ago
I don't think anyone in research actually believes this. Note that the whole idea behind claiming "scaling laws" will infinitely improve these models is a funding strategy rather than a research one. None of these folks think human-like consciousness will "rise" from this effort, even though they veil it to continue the hype-cycle. I guarantee all these firms are desperately looking for architectural breakthroughs, even while they wax poetic about scaling laws, they know there is a bottleneck ahead. Notice how LeCun is the only researcher being honest about this in a public fashion. Meta is committed to AI already and will at least match the spend of competitors anyway, so he doesn't have as much pressure to try and convince investors that this rabbit whole is deeper. Don't get me wrong, LLMs are a tremendous improvement on knowledge compression and distillation, but it's still unreliable enough that old school search is likely a superior method nonetheless.
- xpe 2y ago
  I don’t hold LeCun’s opinions in high regard because of his often hyperbolic statements.
- xpe 2y ago
  Put aside consciousness or hype or investment. Look at the results; LLMs are well beyond old-school search in many ways. Sure, they are flawed in someways. Previous paradigms for search, were also flawed in their own ways. Look at the arc of NLP. Large language models fit the pattern. One could even say that their development (next token prediction with a powerful function approximator) is obvious in hindsight.
  - obirunda 2y ago
    Honestly I don't disagree, I just think that humans tend to anthropomorphize to such a high extent that there is a fair bit of hyperbole promoting LLMs as more than they are. It's my opinion that the big flaws LLMs currently present aren't going to be overcome by scaling alone.
    - xpe 2y ago
      Scaling existing architectures (inference I mean) will probably help a lot. Combine that with better training and hybrid architectures, and I personally expect to see continued improvement. However, given the hype cycle, combined with broad levels of ignorance of how LLMs work, it is an open question if even amazing progress will impress people anymore.
      - obirunda 2y ago
        I'm less concerned with people's perception and strictly concerned with value. If we were to define value as the number of things that can be automated or severely improved by the technology or its future versions, there is a misalignment between value and perceived value. The value is lower than perceived because there is an assumption that what's preventing higher value delivery from the investment is for the models to get better at generating responses from prompts. But there are two issues with this position. 1. LLMs still require plenty of assistance where they are writing production ready code and making function-calls, especially if the original API wasn't designed with LLMs in mind. Unless there is a leap in architecture that's going to make all the tools we have, including non-API ones easily accessible by the models to interact with, the amount of glue code required to make it all work increases the potential for features that users can use, but not necessarily deliver higher value to developers in that a lay person still probably can't develop software with an LLM copilot in toe. So yes, no one is going to be impressed even if we see models improve further on benchmarking. 2. Long-Horizon goals. Long-term research or even project management requires interdisciplinary understanding of how all the goals around success relate to each other and most importantly how to assess if an outcome is leading to a goal accomplishment or not. There isn't an architectural foundation for the models to be grounded in a reality that presupposes these abilities. What I fail to see, is how improving next token prediction will materially move the needle on these other aspects of intelligence that aren't necessarily related to generating an output or a series of outputs orchestrated over an evolving set of requirements. Honestly lI think that the LLM portion of the human brain has likely be surpassed by existing models
throwthrowuknow 2y ago
There isn’t really any reason biological neurons should relate to their modelled reality, what does a single cell care about poetry or even simple things like a chair?
- bamboozled 2y ago
  A chair isn't only a chair, it can be a table, a bookshelf, and many others things. The real world is hard.
xpe 2y ago
I find discussions of consciousness even more taxing than religion, free will, or politics. With very careful discussion, there are some really interesting concepts in play. This paper however does not strike me as worth most people’s time. Especially not regarding consciousness.
grantcas 2y ago
[dead]

obirunda 2y ago

I don't think anyone in research actually believes this. Note that the whole idea behind claiming "scaling laws" will infinitely improve these models is a funding strategy rather than a research one. None of these folks think human-like consciousness will "rise" from this effort, even though they veil it to continue the hype-cycle. I guarantee all these firms are desperately looking for architectural breakthroughs, even while they wax poetic about scaling laws, they know there is a bottleneck ahead. Notice how LeCun is the only researcher being honest about this in a public fashion. Meta is committed to AI already and will at least match the spend of competitors anyway, so he doesn't have as much pressure to try and convince investors that this rabbit whole is deeper. Don't get me wrong, LLMs are a tremendous improvement on knowledge compression and distillation, but it's still unreliable enough that old school search is likely a superior method nonetheless.

xpe 2y ago
I don’t hold LeCun’s opinions in high regard because of his often hyperbolic statements.
xpe 2y ago
Put aside consciousness or hype or investment. Look at the results; LLMs are well beyond old-school search in many ways. Sure, they are flawed in someways. Previous paradigms for search, were also flawed in their own ways. Look at the arc of NLP. Large language models fit the pattern. One could even say that their development (next token prediction with a powerful function approximator) is obvious in hindsight.
- obirunda 2y ago
  Honestly I don't disagree, I just think that humans tend to anthropomorphize to such a high extent that there is a fair bit of hyperbole promoting LLMs as more than they are. It's my opinion that the big flaws LLMs currently present aren't going to be overcome by scaling alone.
  - xpe 2y ago
    Scaling existing architectures (inference I mean) will probably help a lot. Combine that with better training and hybrid architectures, and I personally expect to see continued improvement. However, given the hype cycle, combined with broad levels of ignorance of how LLMs work, it is an open question if even amazing progress will impress people anymore.
    - obirunda 2y ago
      I'm less concerned with people's perception and strictly concerned with value. If we were to define value as the number of things that can be automated or severely improved by the technology or its future versions, there is a misalignment between value and perceived value. The value is lower than perceived because there is an assumption that what's preventing higher value delivery from the investment is for the models to get better at generating responses from prompts. But there are two issues with this position. 1. LLMs still require plenty of assistance where they are writing production ready code and making function-calls, especially if the original API wasn't designed with LLMs in mind. Unless there is a leap in architecture that's going to make all the tools we have, including non-API ones easily accessible by the models to interact with, the amount of glue code required to make it all work increases the potential for features that users can use, but not necessarily deliver higher value to developers in that a lay person still probably can't develop software with an LLM copilot in toe. So yes, no one is going to be impressed even if we see models improve further on benchmarking. 2. Long-Horizon goals. Long-term research or even project management requires interdisciplinary understanding of how all the goals around success relate to each other and most importantly how to assess if an outcome is leading to a goal accomplishment or not. There isn't an architectural foundation for the models to be grounded in a reality that presupposes these abilities. What I fail to see, is how improving next token prediction will materially move the needle on these other aspects of intelligence that aren't necessarily related to generating an output or a series of outputs orchestrated over an evolving set of requirements. Honestly lI think that the LLM portion of the human brain has likely be surpassed by existing models

throwthrowuknow 2y ago

There isn’t really any reason biological neurons should relate to their modelled reality, what does a single cell care about poetry or even simple things like a chair?

bamboozled 2y ago
A chair isn't only a chair, it can be a table, a bookshelf, and many others things. The real world is hard.

xpe 2y ago

I find discussions of consciousness even more taxing than religion, free will, or politics. With very careful discussion, there are some really interesting concepts in play. This paper however does not strike me as worth most people’s time. Especially not regarding consciousness.

grantcas 2y ago

[dead]