2 ms·

Machines can't `learn' anything at all. They just recognize patterns.

tstrimple 4y ago
LLM have no mechanism for determining truth. They have a dataset which determines what is truth to them. If you include "all sources" they are just going to include a lot of junk next to useful information. An ML algo consuming content generated from other ML algos (which will be the reality very soon) is basically just recycling garbage over and over. It's certainly no mechanism for elevating "truth".
- jkea 4y ago
  What are humans mechanisms for determining truth?
  - tstrimple 4y ago
    As far as I'm aware, the scientific method is the best we have at the moment and it has delivered pretty good results.
Nevermark 4y ago
They don’t just recognize patterns. They learn to recognize patters. Any algorithm that captured anything useful from exposure to data is learning. Machines most definitely learn. Watch a plot of generalization accuracy over training iterations (by testing with non-training data). You will see periods of slow improvement, alternating with periods of very fast improvement, as important relationships are discovered. Machines earn.
- Eddy_Viscosity2 4y ago
  This may just be a semantic difference. You could also say that my memory foam insoles 'learned' to shape themselves to the contours of my foot. But's not really what I was going for when meant 'learn' which is to have a deeper understanding of a concept.
  - Nevermark 4y ago
    Any model that is able to generalize or extrapolate from a few examples to new examples has "understood" a relationship not explicit in the raw training data. That "understanding" is what allows the model to capture the same information, plus implications not explicit in the data, in far fewer parameters than the raw data. It is compression by understanding. The test of how much "understanding" a model has learned, is the quality of its responses to new examples and new combinations of patterns, beyond its training experiences. -- Deep learning models are in a mathematical class called "universal approximators". They don't just learn simple associations, correlations, or conditional probabilities. Deep learning models can learn arbitrarily complex functional relationships. They provably can learn to do anything you can do. Anything any information processing device can do. To be a "universal approximator", a neural network type model needs at least two layers of learning units, with enough units and parameters in the first layer, and enough training data and computation. The problem is that simple two layer models may require an impractical quantity of units, parameters, data and computation. Thus the reason the use of many (more than 2) "deep" layers, convolutions layers, and recurrent connections, all of which can radically reduce the model sizes, data and computation needed. -- Successful language models understand most things at a level similar to how a blind person understands color - not directly, but by lots of indirect exposure to other people's experiences. But multimodal models, give multiple data modalities, can understand color, images, spacial relationship, dynamical movement, etc., directly. That is where things are quickly heading.

LLM have no mechanism for determining truth. They have a dataset which determines what is truth to them. If you include "all sources" they are just going to include a lot of junk next to useful information. An ML algo consuming content generated from other ML algos (which will be the reality very soon) is basically just recycling garbage over and over. It's certainly no mechanism for elevating "truth".

jkea 4y ago
What are humans mechanisms for determining truth?
- tstrimple 4y ago
  As far as I'm aware, the scientific method is the best we have at the moment and it has delivered pretty good results.

Nevermark 4y ago

They don’t just recognize patterns. They learn to recognize patters. Any algorithm that captured anything useful from exposure to data is learning. Machines most definitely learn. Watch a plot of generalization accuracy over training iterations (by testing with non-training data). You will see periods of slow improvement, alternating with periods of very fast improvement, as important relationships are discovered. Machines earn.

Eddy_Viscosity2 4y ago
This may just be a semantic difference. You could also say that my memory foam insoles 'learned' to shape themselves to the contours of my foot. But's not really what I was going for when meant 'learn' which is to have a deeper understanding of a concept.
- Nevermark 4y ago
  Any model that is able to generalize or extrapolate from a few examples to new examples has "understood" a relationship not explicit in the raw training data. That "understanding" is what allows the model to capture the same information, plus implications not explicit in the data, in far fewer parameters than the raw data. It is compression by understanding. The test of how much "understanding" a model has learned, is the quality of its responses to new examples and new combinations of patterns, beyond its training experiences. -- Deep learning models are in a mathematical class called "universal approximators". They don't just learn simple associations, correlations, or conditional probabilities. Deep learning models can learn arbitrarily complex functional relationships. They provably can learn to do anything you can do. Anything any information processing device can do. To be a "universal approximator", a neural network type model needs at least two layers of learning units, with enough units and parameters in the first layer, and enough training data and computation. The problem is that simple two layer models may require an impractical quantity of units, parameters, data and computation. Thus the reason the use of many (more than 2) "deep" layers, convolutions layers, and recurrent connections, all of which can radically reduce the model sizes, data and computation needed. -- Successful language models understand most things at a level similar to how a blind person understands color - not directly, but by lots of indirect exposure to other people's experiences. But multimodal models, give multiple data modalities, can understand color, images, spacial relationship, dynamical movement, etc., directly. That is where things are quickly heading.