3 ms·

i dont know much about machine learning but what i think i know is that its getting an outcome based on averages of witnessed data/events. so how's it going to

i dont know much about machine learning but what i think i know is that its getting an outcome based on averages of witnessed data/events. so how's it going to come up with anything novel? or outside of normal?

randcraw 2y ago
It can't. Without the ability to propose a hypothesis and then experimentally test it in the physical real world, no ML technique or app can add new information to the world. The only form of creativity possible using AI (as it exists today) is to recombine existing information in a new way -- as a musical composer or jazz artist creates a variation on an theme that already exists. But that can't be compared to devising something new that we would call truly creative and original, especially novel work that advances the frontier of our understanding of the world, like scientific discovery.
raincole 2y ago
> i dont know much about machine learning but what i think i know is that its getting an outcome based on averages of witnessed data/events "Extrapolation" is the word you're looking for, for a superficial understanding of machine learning. "Average" isn't even remotely close.
ben_w 2y ago
That's not what ML is. You have some input, which may include labels or not. If it does have labels, ML is the automatic creation of a function that maps the examples to the labels, and the function may be arbitrarily complex. When the function is complex enough to model the world that created the examples, it's also capable of modelling any other input; this is why LLMs can translate between languages without needing examples of the specific to-from language pair in the training set. The data may also be synthetic based on the rules of the world; this is how AlphaZero beats the best Go and chess players without any examples from human play.
richrichie 2y ago
ML is (smooth) surface fitting. That's mostly it. But that is not undermining it in any way. Approximation theory and statistical learning have long history, full of beautiful results. The kitchen sinks that we can throw at ML are incredibly powerful these days. So, we can quickly iterate through different neural net architecture configurations, check accuracy on a few standard datasets and report whatever configuration that does better on Archiv. Some might call it 'p-hacking'.
michaelt 2y ago
Hypothetically? Training data gives the model an idea what "blue" means, and what "cat" means. It can now generate sensible output about blue cats, despite blue cats not being normal.
IshKebab 2y ago
It's predicting a function. You train it on known inputs (and potentially corresponding known outputs). You get a novel output by feeding it a novel input. For example asking an LLM a question that nobody has even asked it before. The degree to which it does a good job on those questions is called "generalisation".

randcraw 2y ago

It can't. Without the ability to propose a hypothesis and then experimentally test it in the physical real world, no ML technique or app can add new information to the world. The only form of creativity possible using AI (as it exists today) is to recombine existing information in a new way -- as a musical composer or jazz artist creates a variation on an theme that already exists. But that can't be compared to devising something new that we would call truly creative and original, especially novel work that advances the frontier of our understanding of the world, like scientific discovery.

raincole 2y ago

> i dont know much about machine learning but what i think i know is that its getting an outcome based on averages of witnessed data/events "Extrapolation" is the word you're looking for, for a superficial understanding of machine learning. "Average" isn't even remotely close.

ben_w 2y ago

That's not what ML is. You have some input, which may include labels or not. If it does have labels, ML is the automatic creation of a function that maps the examples to the labels, and the function may be arbitrarily complex. When the function is complex enough to model the world that created the examples, it's also capable of modelling any other input; this is why LLMs can translate between languages without needing examples of the specific to-from language pair in the training set. The data may also be synthetic based on the rules of the world; this is how AlphaZero beats the best Go and chess players without any examples from human play.

richrichie 2y ago

ML is (smooth) surface fitting. That's mostly it. But that is not undermining it in any way. Approximation theory and statistical learning have long history, full of beautiful results. The kitchen sinks that we can throw at ML are incredibly powerful these days. So, we can quickly iterate through different neural net architecture configurations, check accuracy on a few standard datasets and report whatever configuration that does better on Archiv. Some might call it 'p-hacking'.

michaelt 2y ago

Hypothetically? Training data gives the model an idea what "blue" means, and what "cat" means. It can now generate sensible output about blue cats, despite blue cats not being normal.

IshKebab 2y ago

It's predicting a function. You train it on known inputs (and potentially corresponding known outputs). You get a novel output by feeding it a novel input. For example asking an LLM a question that nobody has even asked it before. The degree to which it does a good job on those questions is called "generalisation".