4 ms·
Except LLM's are built on neural networks. That are based on how neurons work. The first tech that actually copies aspects of us.
by ImHereToVote 3y ago
Except LLM's are built on neural networks. That are based on how neurons work. The first tech that actually copies aspects of us.
- TaylorAlexander 3y agosigh Neural networks are not based on how neurons work. They do not copy aspects of us. They call them neural networks because they are sort of conceptually like networks of neurons in the brain but they’re so different as to make false the statement that they are based on neurons.
- Terr_ 3y ago*brandishes crutches* "Behold! The Mechanical Leg! The first technology that actually copies aspects of our very selves! Think of what wonders of self-discovery it shall reveal!" :p P.S.: "My god, it is stronger in compression rather than shear-stresses, how eerily similar to real legs! We're on to something here!"
- finefrenchbrit 3y ago[dead]
- robwwilliams 3y agoIf you study retinal synaptic circuitry you will not sigh so heavily and you will in fact see striking homologies with hardware neural networks, including feedback between layers and discretized (action potential) outputs via the optic nerve. I recommend reading Synaptic Organization of the Brain or getting into if you are brave, the primary literature on retinal processing of visual input.
- smokel 3y agoThe book "The Synaptic Organization of the Brain" appears to be from 2003. Is it still relevant, or is there perhaps a more recent book worth checking out?
- robwwilliams 3y agoIt is great even though older. The chapter by Sterling on retina is amazing. Yes there is an updated version by Gordon Shepherd and colleagues: Handbook of Brain Microcircuits, but I actually prefer the 2003 edition.
- TaylorAlexander 3y agoActually it’s funny my best friend is a neuroscientist and studies the retina and in particular the way different types of retinal cells respond to stimulus. I have watched her give presentations on her work and I do see that there are some similarities. But it is nonetheless the case that “neural networks” are not called that because they are based on the way neurons work.
- ImHereToVote 3y agoThey literally are: https://www.ibm.com/topics/neural-networks#:~:text=Their%20name%20and%20structure%20are,layers%2C%20and%20an%20output%20layer https://www.ibm.com/topics/neural-networks#:~:text=Their%20n...
- __loam 3y agoI will continue to sigh. The visual cortex is relatively simple and linear. You're not saying something that's as impressive as you think it is.
- l33t7332273 3y agoI think the point of the example is that that is an important part of our brains that is relatively simple and linear and we’ve been able to mimic it.
- robwwilliams 3y agoAnything but simple and anything but linear.
- martindbp 3y agoSigh... Everyone knows artificial neurons are not like biological neurons. The network is the important part, which really is analogous to the brain, while what came before (SVMs and random forests) are nothing like it.
- mecsred 3y agoSigh... Every man knows the mechanisms of the mind are yet unlike the cogs and pinions of clockwork. It remains the machinery, the relation of spring and escapement, that is most relevant. Hitherto in human history, I think, such structure has not been described.
- TeMPOraL 3y agoIf you build a neural network out of cogs and pinions, sure. Comparing the brain to most complex machines in history wasn't a mistake, any more than refining laws of physics were. Successive approximations. And we're no longer at the point where we're just comparing brain to most complex machines. We have information theory now. We figured out computation, in form independent of physical medium used. So we're really trying to determine the computational model behind the brain, and one of the ways to do it is to implement some computational models in whatever is most convenient (usually software running on silicon), and see if it's similar. Slowly but surely, we're mapping and matching computational aspects of the brain. LLMs are just one recent case where we got a spectacularly good match.
- TaylorAlexander 3y ago> Everyone knows artificial neurons are not like biological neurons. Not, apparently, the person I was replying to!
- ImHereToVote 3y agoI'm him, and I didn't say that. ANNs didn't arise in a vacuum and they aren't called neural networks for the fun of it. https://www.ibm.com/topics/neural-networks#:~:text=Their%20name%20and%20structure%20are,layers%2C%20and%20an%20output%20layer https://www.ibm.com/topics/neural-networks#:~:text=Their%20n....
- renewiltord 3y agoDoesn't really matter to modern CS, but Rosenblatt's original perceptron paper is a good read on this. ANNs were specifically inspired by Natural NNs and there were many attempts to build ANNs using models of how the human brain works, specifically down to the neuron.
- dekhn 3y agoI;m sure you know but one of the best ways to get neuro folks worked up is to say anything about neural networks being anything like neurons in brains. (IMHO, Rosenblatt is an underappreciated genius; he had a working shallow computer vision hardware computer long before people even appreciated what an accomplishment that was. The hardware was fascinating- literally self-turning potentiometer knobs to update weights.
- renewiltord 3y agoIf I'm being honest, I do know they get annoyed by that stuff but I've never really understood why. It's a somewhat common pattern in Mathematics as an avenue for hypotheses to take an existing phenomenon, model some subset of its capabilities, use that to define a new class of behaviour, follow that through to conclusions, then use that to go back to seeing if those conclusions apply to the original phenomenon. A theoretical such thing might be for us to look at, say, human arms and say "Well, this gripping thing is a cool piece of functionality. Let's build an artificial device that does this. But we don't have muscle contraction tech, so we'll put actuators in the gripping portion. All right, we've built an arm. It seems like if we place it in this position it minimizes mechanical wear when not in action and makes it unlikely for initial movement to create undesired results. I wonder if human arms+hands have the same behaviour. Ah, looks like not, but that would have been interesting if it were the case" Essentially that's just the process of extracting substructure and then seeing if there is a homomorphism (smooshy type abuse here) between two structures as a way to detect yet hidden structure. Category theory is almost all this. I suppose the reason they find it annoying is that there are many mappings that are non-homomorphic and so these are the false cognates of concepts. Still, I think the whole "An ANN is not a brain" thing is overdone. Of course not. A mechanical arm is not an arm, but they both have response curves, and one can consider a SLAM approach for the former and compare with the proprioceptive view of the latter. It just needs some squinting. Anyway, considering your familiarity with R and his work, I think I'm not speaking to the uninitiated, but I thought it worth writing anyway.
- famouswaffles 3y agoThey are though. They quite literally are. Saying otherwise is like saying planes weren't based on how birds work when Wright brothers spent a lot of time in the 1800s studying birds. Both Humans and GPT are neural networks. Who cares that GPT doesn't have feathers or flap its wings? That's not the question to care bout. We are interested in whether GPT flies. You can sigh to Kingdom come and nothing will change that. We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse, so we stuck with backpropagation. We've made more complicated neurons that better resemble biological neurons, but it is faster and works better if you just add extra simple neurons, so we do that instead. Spiking neural networks have connection patterns more similar to what you see in the brain, but they learn slower and are tougher to work with than regular layered neural networks, so we use layered neural networks instead. The Wright brothers probably experimented with gluing feathers onto their gliders, but eventually decided it wasn’t worth the effort. Because that's not what is important.
- ImHereToVote 3y agoThere are drones with feathers now however. The spring in feather flaps help conserve energy, but only in flapping wings obviously.
- ImHereToVote 3y agoScience history should be mandatory for undergrads. I didn't think what I said is controversial. This is established history. Sorry if it scares you.
- deleted 3y ago[deleted]
- tsukurimashou 3y agoI wish this myth would die
- ImHereToVote 3y agoThis is basic scientific history. You are simply uneducated, or scared of the implications. https://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks/History/history1.html#:~:text=Neural%20Networks%20%2D%20History&text=In%201943%2C%20neurophysiologist%20Warren%20McCulloch,neural%20network%20using%20electrical%20circuits https://cs.stanford.edu/people/eroberts/courses/soco/project.... Key word: "neurophysiologist"
- dragonwriter 3y agoNeural networks aren't based on how biological neurons work, though they are, I think, based on an outdated and even when less outdated simplified model of how they might work.