3 ms·
> Even if many types of architectures converge to the same loss over time, finding the one that converges the fastest is quite valuable given the cost of runnin
by teruakohatu 2y ago
> Even if many types of architectures converge to the same loss over time, finding the one that converges the fastest is quite valuable given the cost of running GPU's at scale.
This! Not just fastest but with the lowest resources in total.
Fully connected neural networks are universal functions. Technically we don’t need anything but a FNN, but memory requirements and speed would be abysmal far beyond the realm of practicality.
- actionfromafar 2y agoUnless we could build chips in 3D?
- foota 2y agoNot even then, a truly fully connected network would have super exponential runtime (it would take N^N time to evaluate)
- ivan_gammel 2y agoWe need quantum computing there. I remember seeing a recent article about quantum processes in the brain. If that’s true, QC may be the missing part.
- tsimionescu 2y agoThis is just word salad. There is no known quantum algorithm that can compute the result of a fully-connected neural network exponentially faster than classical computers can. QCs have a known exponential advantage over classical computers only for a very limited class of problems, mostly related to the Quantum Fourier Transform. Animal brains have little to nothing in common to artifical neural networks. There is no reason whatsoever to think that there is any relation between the complexity class of brain functions and ANN inference. And the hypothesized (and still wildly speculative) quantum behaviors happening in the animal brain are at the level of the behavior of individual neurons, not of the network connections between neurons. So even if there is some kind of quantum computation happening, it's happening in individual neurons, not at the network level, and that would only go to show even more that animal brains are profoundly different from ANNs.
- eru 2y agoCompare and contrast https://www.smbc-comics.com/comic/the-talk-3 https://www.smbc-comics.com/comic/the-talk-3 (Summary: quantum computing is unlikely to help.)
- mvkel 2y agoWetware is the future.
- fennecfoxy 2y agoCan't wait to see this defiantly spray painted across a torn up brick wall while computronium brained super intelligences slowly disassemble our planet to make paperclips.
- mvkel 2y agohttps://imgur.com/zeBkh2P https://imgur.com/zeBkh2P
- ComputerGuru 2y agoHeat extraction.
- bob1029 2y agoWe are already doing this.