5 ms·
Before the recent AI boom, I was mystified by the possibility of AI and emulating humans (in no small part thanks to works of fiction showing AI powered android
by SkyBelow 2y ago
Before the recent AI boom, I was mystified by the possibility of AI and emulating humans (in no small part thanks to works of fiction showing AI powered androids). Then I created and trained some neural networks. Smaller ones, doing much of nothing special. That was enough to break the mysticism. To realize it was just multiplying matrices. Training them was a bit more advanced, but still applied mathematics.
Only recently have I begun to appreciate that the simplicity of the operation, applied to a large enough matrices, may still capture enough of the nature of intelligence and sentience. In the end we can be broken down into (relatively) simple chemical reactions, and it is the massive scale of these reactions that create real intelligence and sentience.
- naasking 2y agoExactly, the people who are derisive of those who consider ML models to exhibit glimmers of true intelligence because it's only matrix multiplications always amuse me. It's like they don't even realize the contradiction in holding the position that seemingly complex and intelligent outward behaviour should not be used as an indication of actual complexity and intelligence.
- citizen_friend 2y agoIf you study this historically you will see that every generation thinks they have the mechanism to explain brains (gear systems, analog control/cybernetics, perceptrons). My conclusion is we tend to overestimate our understanding and the power of our inventions.
- naasking 2y agoThe difference is that we now actually have a proof of computational power and computational universality.
- citizen_friend 2y agoAnalog circuits have the same computational power. Piecewise linear functions have the same computational universality.
- naasking 2y agoExcept we didn't know any of that, nor did know how to construct physical analogs in order to achieve universal computation. At best, we had limited task-specific computation, like clocks and planetary motion.
- citizen_friend 2y agoWe knew about universal function approximators like. Polynomials and trig functions since the 1700s. Turing and godel were around 1910 and 1920. The cybernetics movement is big in the 30s and 40s. Perceptrons 50s and 60s
- naasking 2y agoTaylor expansions for all functions do not exist. Furthermore, our characterization of infinity was still poor, so we didn't even have a solid notion of what it would mean for a formalism to be able to compute all computable functions. The notion of a universal computer arguably didn't exist until Babbage. I stand by my position that having a mathematical proof of computational universality is a significant difference that separates today from all prior eras that sought to understand the brain through contemporaneous technology.
- citizen_friend 2y ago> Taylor expansions That’s not what I’m talking about. This is a basic analysis topic: https://en.m.wikipedia.org/wiki/Stone%E2%80%93Weierstrass_theorem https://en.m.wikipedia.org/wiki/Stone%E2%80%93Weierstrass_th... At least mid 1800s for a proof. 1700s also explored Fourier series. > stand by my position And you’re still ignoring the cybernetics, and perceptrons movement I keep referring to which was more than 100 years ago, and informed by Turing.
- mistermann 2y agoNext step, in case you get bored: why (in fact, not a minor distinction) does such a simple approach work so well?