3 ms·

> We know how LLMs work fundamentally We know how they work only at the lowest level (the arithmetic operations) and the highest level (the optimization criter

by wyager 2y ago

> We know how LLMs work fundamentally We know how they work only at the lowest level (the arithmetic operations) and the highest level (the optimization criterion and the representation of various layers, like the input/output layer and for things we can easily probe like embedding matrices). We do not know "what they are doing" on the inner layers. This is an area of active research. > They do not have the ability to reason nor engage in high level thought. You are speculating (and probably incorrectly), or you're really holding back some valuable research from the field of AI interpretability.

samtho 2y ago
You talk about about as if a human-created neural network is at the same level as quantum physics where there are limits as to our understanding. We know very well how large language models work even if the capabilities of this technology are actively being explored. You along with others here are far overstating the unknowns we have within the context of AI, whether this is the result of a misinformation campaign targeted at trying to boost the value of this tech or if the pop-sci takes have really gotten too prevalent, it is unclear to me.
- danielmarkbruce 2y ago
  For the definition of "understand" that most people use, humans don't understand things which are highly complex. We don't really understand the weather, we can't predict it, it's too complex. But, you can break it down into matter and forces and energy and simulate it and get pretty darn good predictions. We can now throw it in a deep learning model and get good predictions. But, to suggest we "understand it" doesn't gel with most people's definition of "understand".
- mike_hearn 2y ago
  wyager is correct. There are very serious limits to our understanding of what's going on inside these networks. Even how an LLM answers simple factual questions like "The capital of France is ..." is only now just coming into view. And the moment it gets more complex than that, interpretability is lost again.

by wyager 2y ago

samtho 2y ago

You talk about about as if a human-created neural network is at the same level as quantum physics where there are limits as to our understanding. We know very well how large language models work even if the capabilities of this technology are actively being explored. You along with others here are far overstating the unknowns we have within the context of AI, whether this is the result of a misinformation campaign targeted at trying to boost the value of this tech or if the pop-sci takes have really gotten too prevalent, it is unclear to me.

danielmarkbruce 2y ago
For the definition of "understand" that most people use, humans don't understand things which are highly complex. We don't really understand the weather, we can't predict it, it's too complex. But, you can break it down into matter and forces and energy and simulate it and get pretty darn good predictions. We can now throw it in a deep learning model and get good predictions. But, to suggest we "understand it" doesn't gel with most people's definition of "understand".
mike_hearn 2y ago
wyager is correct. There are very serious limits to our understanding of what's going on inside these networks. Even how an LLM answers simple factual questions like "The capital of France is ..." is only now just coming into view. And the moment it gets more complex than that, interpretability is lost again.