3 ms·

I believe the future is 4*4 bit look up tables with output latches, with a bit to/from each Cartesian neighbor. Clock them like the colors of a chessboard, in 2

I believe the future is 4*4 bit look up tables with output latches, with a bit to/from each Cartesian neighbor. Clock them like the colors of a chessboard, in 2 phases, and you don't have to worry about timing dependencies. All of your code gets converted to a directed acyclic graph(DAG), executing at Ghz rates if you want. Imagine a machine that can output a million parallel GPT-4 streams at 1000 tokens per second each. If the architecture changes it's just a different DAG. Unlike with FPGAs and their custom blocks that have to be optimally used, you can compile a DAG almost instantly.

smusamashah 3y ago
Is this something from a research finding or is it your idea?
- mikewarot 3y ago
  It's my idea... I've been writing about it for over a decade. I hope to get a small version of it made via tiny tapeout later this year once I'm out of the precariat. The idea is that you can decompose any non-cyclic code, for example, a matrix multiply, FFT, etc.. into a directed graph of bitwise operations, then map those operations out into the grid of LUTs. Pipe inputs in one side of the grid, and get the outputs out the other side, and all of the bitwise operations happen in lock step parallelism. (Unlike an FPGA which is asyncronous) If you can decompose one stage of an LLM down into a bitwise graph of computation, you can easily map it into the grid. If there's a bad cell in the grid, you can map around it. Because all of the high speed logic lines are short (to the neighbors) you don't have the long lines / high capacitance issues driving signals all the way across an FPGA or ASIC, thus you can really crank up the clock rate (and/or it's very power efficient). It should be trivial, design wise, to scale from a 16x16 grid up through chips that could crank out Petaflops. Here's a simulator for the thing, written in Pascal. https://github.com/mikewarot/Bitgrid https://github.com/mikewarot/Bitgrid
  - sitkack 3y ago
    You can simulate this on a GPU now. This is like running a CA on a GPU with slightly different rules. I think you would also be delighted to know that many of the non-GPU ai accelerators are in a mesh topology. Cerebras, Tenstorrent, Esperanto https://www.esperanto.ai/wp-content/uploads/2021/08/HC2021.Esperanto.Ditzel.Final_.pdf https://www.esperanto.ai/wp-content/uploads/2021/08/HC2021.E...
    - mikewarot 3y ago
      I've been intrigued by this approach. On a more highly optimized (but harder to program) take is the GA144[1] from Chuck Moore, the inventor of Forth. It's a grid of 144 F18 Forth based processors in a cartesian grid. These processors are far more limited, but then again they take far less power as well. [1] https://www.greenarraychips.com/ https://www.greenarraychips.com/
      - sitkack 3y ago
        That is a pretty cool chip. Chuck Moore is a fascinating character. You would probably get a kick out of David Ackley's T2 Tile Project. https://t2tile.com/ https://t2tile.com/ https://www.youtube.com/@T2TileProject/videos https://www.youtube.com/@T2TileProject/videos https://www.youtube.com/@DaveAckley https://www.youtube.com/@DaveAckley https://en.wikipedia.org/wiki/Ackley_function https://en.wikipedia.org/wiki/Ackley_function https://www.cs.unm.edu/~ackley/ https://www.cs.unm.edu/~ackley/ Fun fact, Tenstorrent wanted to add instructions to enqueue data between processors connected in a mesh and Arm said no (they don't do architectural licenses anymore), so Tenstorrent used RISCV. Implementing something like a GA144 but using small RISC-V processors would be an interesting hack. https://www.youtube.com/watch?v=6QRKpd28NEE https://www.youtube.com/watch?v=6QRKpd28NEE
Dylan16807 3y ago
1. If you write FPGA code as a grid of lookup tables then I would expect it to be easy to compile instantly. 2. In what way is this "acyclic"? 3. Won't putting your code into this form be the hard part? Even if you start with a DAG, 99.99% of them won't fit this form without intense restructuring. So you just moved the hard step over by one.

smusamashah 3y ago

Is this something from a research finding or is it your idea?

mikewarot 3y ago
It's my idea... I've been writing about it for over a decade. I hope to get a small version of it made via tiny tapeout later this year once I'm out of the precariat. The idea is that you can decompose any non-cyclic code, for example, a matrix multiply, FFT, etc.. into a directed graph of bitwise operations, then map those operations out into the grid of LUTs. Pipe inputs in one side of the grid, and get the outputs out the other side, and all of the bitwise operations happen in lock step parallelism. (Unlike an FPGA which is asyncronous) If you can decompose one stage of an LLM down into a bitwise graph of computation, you can easily map it into the grid. If there's a bad cell in the grid, you can map around it. Because all of the high speed logic lines are short (to the neighbors) you don't have the long lines / high capacitance issues driving signals all the way across an FPGA or ASIC, thus you can really crank up the clock rate (and/or it's very power efficient). It should be trivial, design wise, to scale from a 16x16 grid up through chips that could crank out Petaflops. Here's a simulator for the thing, written in Pascal. https://github.com/mikewarot/Bitgrid https://github.com/mikewarot/Bitgrid
- sitkack 3y ago
  You can simulate this on a GPU now. This is like running a CA on a GPU with slightly different rules. I think you would also be delighted to know that many of the non-GPU ai accelerators are in a mesh topology. Cerebras, Tenstorrent, Esperanto https://www.esperanto.ai/wp-content/uploads/2021/08/HC2021.Esperanto.Ditzel.Final_.pdf https://www.esperanto.ai/wp-content/uploads/2021/08/HC2021.E...
  - mikewarot 3y ago
    I've been intrigued by this approach. On a more highly optimized (but harder to program) take is the GA144[1] from Chuck Moore, the inventor of Forth. It's a grid of 144 F18 Forth based processors in a cartesian grid. These processors are far more limited, but then again they take far less power as well. [1] https://www.greenarraychips.com/ https://www.greenarraychips.com/
    - sitkack 3y ago
      That is a pretty cool chip. Chuck Moore is a fascinating character. You would probably get a kick out of David Ackley's T2 Tile Project. https://t2tile.com/ https://t2tile.com/ https://www.youtube.com/@T2TileProject/videos https://www.youtube.com/@T2TileProject/videos https://www.youtube.com/@DaveAckley https://www.youtube.com/@DaveAckley https://en.wikipedia.org/wiki/Ackley_function https://en.wikipedia.org/wiki/Ackley_function https://www.cs.unm.edu/~ackley/ https://www.cs.unm.edu/~ackley/ Fun fact, Tenstorrent wanted to add instructions to enqueue data between processors connected in a mesh and Arm said no (they don't do architectural licenses anymore), so Tenstorrent used RISCV. Implementing something like a GA144 but using small RISC-V processors would be an interesting hack. https://www.youtube.com/watch?v=6QRKpd28NEE https://www.youtube.com/watch?v=6QRKpd28NEE

Dylan16807 3y ago

1. If you write FPGA code as a grid of lookup tables then I would expect it to be easy to compile instantly. 2. In what way is this "acyclic"? 3. Won't putting your code into this form be the hard part? Even if you start with a DAG, 99.99% of them won't fit this form without intense restructuring. So you just moved the hard step over by one.