The models described in this article run on the powerful GPU of a modern computer or a smartphone. Yet, let’s speculate about what a “more physical” implementat

The models described in this article run on the powerful GPU of a modern computer or a smartphone. Yet, let’s speculate about what a “more physical” implementation of such a system could look like. We can imagine it as a grid of tiny independent computers, simulating individual cells. Each of those computers would require approximately 10Kb of ROM to store the “cell genome”: neural network weights and the control code, and about 256 bytes of RAM for the cell state and intermediate activations. The cells must be able to communicate their 16-value state vectors to neighbors. Each cell would also require an RGB-diode to display the color of the pixel it represents. A single cell update would require about 10k multiply-add operations and does not have to be synchronised across the grid. We propose that cells might wait for random time intervals between updates. The system described above is uniform and decentralised. Yet, our method provides a way to program it to reach the predefined global state, and recover this state in case of multi-element failures and restarts. We therefore conjecture this kind of modeling may be used for designing reliable, self-organising agents. On the more theoretical machine learning front, we show an instance of a decentralized model able to accomplish remarkably complex tasks. We believe this direction to be opposite to the more traditional global modeling used in the majority of contemporary work in the deep learning field, and we hope this work to be an inspiration to explore more decentralized learning modeling. I've been hoping that someone would build the above computer for over 20 years. I used to just get laughed at, or people would mock me for not knowing what I'm talking about (even though I have a computer engineering degree). That transitioned to at least speculating on what I was saying, and will soon transition to being old hat. Oh well. But I look at computers today with a billion transistors, running at the same 3 GHz as they did 20 years ago, and it breaks my heart. I imagine what might have been, if we had arrays of hundreds/thousands of z80 or x86 or 68000 or DEC Alpha or MIPS chips to do whatever we wanted with. Imagine the emergent behavior that might have happened. Instead we're locked into this proprietary/narrow way of working with shaders and specialized neural computers doing SIMD and I just find it all so uninspiring. But I'm hopeful that some of the recent work with getting these older processors running on FPGAs will bear fruit. https://news.ycombinator.com/item?id=17833829 https://news.ycombinator.com/item?id=17833829 https://news.ycombinator.com/item?id=19329083 https://news.ycombinator.com/item?id=19329083 https://news.ycombinator.com/item?id=10089807 https://news.ycombinator.com/item?id=10089807 https://news.ycombinator.com/item?id=3267428 https://news.ycombinator.com/item?id=3267428 https://news.ycombinator.com/item?id=19885901 https://news.ycombinator.com/item?id=19885901 https://www.google.com/search?q=processor+on+FPGA+site:news.ycombinator.com https://www.google.com/search?q=processor+on+FPGA+site:news....