5 ms·

brain floating point ... cool name i guess brain's synaptic precision could go way lower, as low as 26 distinct synapse weights: https://elifesciences.org/art

brain floating point ... cool name i guess brain's synaptic precision could go way lower, as low as 26 distinct synapse weights: https://elifesciences.org/articles/10778 https://elifesciences.org/articles/10778 > A particularly interesting research direction puts these three trends together, with a system running on large-scale ML accelerator hardware, with a goal of being able to train a model that can perform thousands or millions of tasks in a single model. Such a model might be made up of many different components of different structures, yup, he is building a brain

quotemstr 7y ago
> i guess brain's synaptic precision could go way lower, as low as 26 distinct synapse weights: https://elifesciences.org/articles/10778 https://elifesciences.org/articles/10778 Thanks for the link. Artificial neural networks all the way down to binary weights [1] although this approach doesn't seem like the most efficient one. It's interesting how we're still seeing a ton of variability in ML architectures: it suggests we haven't stumbled on the right area yet. It reminds me how early aviation had a huge diversity of aircraft plans, but now, after a lot of optimization, we've settled on that one standard airliner shape everyone uses everywhere. [1] https://arxiv.org/abs/1602.02830 https://arxiv.org/abs/1602.02830
- buboard 7y ago
  A formal theory of deep learning is proving to be much more elusive that avionics. Interesting times though
the8472 7y ago
> yup, he is building a brain Plus applying ML to improving the underlying hardware and software. On the other hand it's not recursive yet and moore's law ending throws a wrench into exponential self-improvement, but it's still a little concerning.
gnode 7y ago
> many different components of different structures > yup, he is building a brain The limbic system of the brain is made of many different structures. It handles much of the ancient fixed function instinctive operation of the brain -- heavily involved in sleep, reflexes, appetite, and motivation, for example. However, our general intelligence and learning capability is mostly due to our neocortex, which has a highly regular structure. The neocortex also subsumes the roles of much of the limbic system as development progresses, overriding the specific structures. This suggests to me that intelligence / learning doesn't benefit from specific structures, but from general structures, capable of encoding behaviour as data (by long-term potentiation of synaptic weights in the brain's case).

quotemstr 7y ago

> i guess brain's synaptic precision could go way lower, as low as 26 distinct synapse weights: https://elifesciences.org/articles/10778 https://elifesciences.org/articles/10778 Thanks for the link. Artificial neural networks all the way down to binary weights [1] although this approach doesn't seem like the most efficient one. It's interesting how we're still seeing a ton of variability in ML architectures: it suggests we haven't stumbled on the right area yet. It reminds me how early aviation had a huge diversity of aircraft plans, but now, after a lot of optimization, we've settled on that one standard airliner shape everyone uses everywhere. [1] https://arxiv.org/abs/1602.02830 https://arxiv.org/abs/1602.02830

buboard 7y ago
A formal theory of deep learning is proving to be much more elusive that avionics. Interesting times though

the8472 7y ago

> yup, he is building a brain Plus applying ML to improving the underlying hardware and software. On the other hand it's not recursive yet and moore's law ending throws a wrench into exponential self-improvement, but it's still a little concerning.

gnode 7y ago

> many different components of different structures > yup, he is building a brain The limbic system of the brain is made of many different structures. It handles much of the ancient fixed function instinctive operation of the brain -- heavily involved in sleep, reflexes, appetite, and motivation, for example. However, our general intelligence and learning capability is mostly due to our neocortex, which has a highly regular structure. The neocortex also subsumes the roles of much of the limbic system as development progresses, overriding the specific structures. This suggests to me that intelligence / learning doesn't benefit from specific structures, but from general structures, capable of encoding behaviour as data (by long-term potentiation of synaptic weights in the brain's case).