3 ms·

People often talk about AI apart from its optimization objective -- as-if it "could" find anything in the data... as-if it could find what we could find it in.

People often talk about AI apart from its optimization objective -- as-if it "could" find anything in the data... as-if it could find what we could find it in. No! The AI is radically imprisoned by its optimization objective. What it must necessarily find is whatever "pattern of associations" maximises "correlations between the items in the data". Necessarily, an AI looking at the sky can never find newton's law of gravity. Newton (et al.) had to disregard what they were looking at to place Earth in the centre and F equal to GMm/r2 etc. etc. The AI isnt "finding patterns in data", it's taking data and maximising a single objective. A very very very strange one, which necessarily treats its data as profoundly irrationally superstitious person would. Are there useful such patterns? Sure. But an AI isnt "finding patterns" as we might -- it's finding one specific kind of pattern which we can characterise quite easily. We know how useful those kinds of patterns are.

qlm 4y ago
"According to Deleuze, philosophy begins with a faire l’idiot – with ‘making oneself an idiot’. Thinking is characterized not by intelligence but by idiocy. Every philosopher who creates a new idiom, a new thinking, a new language, is an idiot. The philosopher bids farewell to all that went before. A philosopher inhabits a virgin, hitherto undescribed immanent level of thinking. By adopting the principle of faire l’idiot, thinking risks the leap into the altogether other, ventures on untrodden paths. The history of philosophy is a history of idiocy, of idiotic leaps: ‘The old idiot wanted indubitable truths at which he could arrive by himself: in the meantime he would doubt everything … The new idiot has no wish for indubitable truths … and wills the absurd – this is not the same image of thought.’ Artificial intelligence cannot think because it is incapable of faire l’idiot. It is too intelligent for becoming an idiot."
SanderNL 4y ago
The pattern finding is in the prompt, the one- or fewshotting. You can give it a novel problem and it will find something. It might not be good, it might not be what you want to see, but you can't maintain it hasn't found something. You vastly, vastly overestimate human intelligence. We took thousands, upon tens of thousands of years to get to Newton and even that is not garantueed. He was a fluke of astronomical proportions. We need mountains upon mountains of (intellectual) infrastructure to get anywhere. Newton didn't just wake up with a good idea. He took what was already there and creatively combined it into a seemingly new thing, but none of it was truly novel. How could it be? Divine inspiration? > it's finding one specific kind of pattern which we can characterise quite easily I'm curious, how would you characterise it?
- jiggawatts 4y ago
  Einstein built his towering thoughts on top of scaffolding laid by Lorenz, Riemann, and other contemporary thinkers. They in turn built their theories on top of foundations laid down by Newton, Gauss, Euler, Cauchy, and Spinoza. Nobody just "thinks up" entire physical theories in a vacuum, it's always incremental progress. I regularly see people moving the goalposts by asking ChatGPT to solve "world problems", as if asking a god. People can't do this, so why would you expect an AI trained on the output of people to just snap its digital fingers and do it?
  - Aeolos 4y ago
    > I regularly see people moving the goalposts by asking ChatGPT to solve "world problems", as if asking a god. People can't do this, so why would you expect an AI trained on the output of people to just snap its digital fingers and do it? Because (some) people are desperately looking for a _god_ that will precisely snap its digital fingers and... do it. Seen under this prism, some of the reactions here start to make sense: "I gave it this particular problem that I thought up, and it made a mistake / couldn't answer it, hence I cannot trust it / it's just a parrot / not creative / not _really_ intelligent." It does not matter how many things it _can_ do, all it matters is that there is this one thing that it can _not_ do. It may be more learnt or creative than any single human, but it is not yet _omniscient_.
dangond 4y ago
> Necessarily, an AI looking at the sky can never find newton's law of gravity. Newton (et al.) had to disregard what they were looking at to place Earth in the centre and F equal to GMm/r2 etc. etc. This seems strange to say to me, since this can easily be done by regression models given the right training data. If a sufficiently powerful system is given that training data and trained to minimize loss, it will necessarily converge on a model approximating the underlying laws that created the training data.
- mjburgess 4y ago
  If you define the optimization objective to be y = m . x1 . x2 . x3^-2, or perhaps the family y = m . x1^a . x2^b . x3^c -- you might be able to select the right function based on an "experimentally prepared" dataset. That dataset would not be photographs of the night sky, and those models you're fitting would have to be pre-specified. We can show, easily, no automated statistical modelling of the night sky will produce F=GMm/r^2 The optimization objective for automated modelling here makes the resulting model nothing more than an interpolation through data points. Those points, the position of the planets in the sky say, are not caused by F=GMm/r^2 -- but by a near-infinite nuumber of causes, including our relative positions to those planets *which is not in the night sky!*. If you already know the form of the answer, and have already done the right experiments, yes: the solution is function optimization

qlm 4y ago

"According to Deleuze, philosophy begins with a faire l’idiot – with ‘making oneself an idiot’. Thinking is characterized not by intelligence but by idiocy. Every philosopher who creates a new idiom, a new thinking, a new language, is an idiot. The philosopher bids farewell to all that went before. A philosopher inhabits a virgin, hitherto undescribed immanent level of thinking. By adopting the principle of faire l’idiot, thinking risks the leap into the altogether other, ventures on untrodden paths. The history of philosophy is a history of idiocy, of idiotic leaps: ‘The old idiot wanted indubitable truths at which he could arrive by himself: in the meantime he would doubt everything … The new idiot has no wish for indubitable truths … and wills the absurd – this is not the same image of thought.’ Artificial intelligence cannot think because it is incapable of faire l’idiot. It is too intelligent for becoming an idiot."

SanderNL 4y ago

The pattern finding is in the prompt, the one- or fewshotting. You can give it a novel problem and it will find something. It might not be good, it might not be what you want to see, but you can't maintain it hasn't found something. You vastly, vastly overestimate human intelligence. We took thousands, upon tens of thousands of years to get to Newton and even that is not garantueed. He was a fluke of astronomical proportions. We need mountains upon mountains of (intellectual) infrastructure to get anywhere. Newton didn't just wake up with a good idea. He took what was already there and creatively combined it into a seemingly new thing, but none of it was truly novel. How could it be? Divine inspiration? > it's finding one specific kind of pattern which we can characterise quite easily I'm curious, how would you characterise it?

jiggawatts 4y ago
Einstein built his towering thoughts on top of scaffolding laid by Lorenz, Riemann, and other contemporary thinkers. They in turn built their theories on top of foundations laid down by Newton, Gauss, Euler, Cauchy, and Spinoza. Nobody just "thinks up" entire physical theories in a vacuum, it's always incremental progress. I regularly see people moving the goalposts by asking ChatGPT to solve "world problems", as if asking a god. People can't do this, so why would you expect an AI trained on the output of people to just snap its digital fingers and do it?
- Aeolos 4y ago
  > I regularly see people moving the goalposts by asking ChatGPT to solve "world problems", as if asking a god. People can't do this, so why would you expect an AI trained on the output of people to just snap its digital fingers and do it? Because (some) people are desperately looking for a _god_ that will precisely snap its digital fingers and... do it. Seen under this prism, some of the reactions here start to make sense: "I gave it this particular problem that I thought up, and it made a mistake / couldn't answer it, hence I cannot trust it / it's just a parrot / not creative / not _really_ intelligent." It does not matter how many things it _can_ do, all it matters is that there is this one thing that it can _not_ do. It may be more learnt or creative than any single human, but it is not yet _omniscient_.

dangond 4y ago

> Necessarily, an AI looking at the sky can never find newton's law of gravity. Newton (et al.) had to disregard what they were looking at to place Earth in the centre and F equal to GMm/r2 etc. etc. This seems strange to say to me, since this can easily be done by regression models given the right training data. If a sufficiently powerful system is given that training data and trained to minimize loss, it will necessarily converge on a model approximating the underlying laws that created the training data.

mjburgess 4y ago
If you define the optimization objective to be y = m . x1 . x2 . x3^-2, or perhaps the family y = m . x1^a . x2^b . x3^c -- you might be able to select the right function based on an "experimentally prepared" dataset. That dataset would not be photographs of the night sky, and those models you're fitting would have to be pre-specified. We can show, easily, no automated statistical modelling of the night sky will produce F=GMm/r^2 The optimization objective for automated modelling here makes the resulting model nothing more than an interpolation through data points. Those points, the position of the planets in the sky say, are not caused by F=GMm/r^2 -- but by a near-infinite nuumber of causes, including our relative positions to those planets *which is not in the night sky!*. If you already know the form of the answer, and have already done the right experiments, yes: the solution is function optimization