4 ms·

If freight trains are so good, why do we use trucks? Freight trains carry a lot, but are really bad at branching. Same with GPUs.

deleted 2y ago
[deleted]
mikewarot 2y ago
Freight trains are good, but pipelines are better for bulk liquids. GPUs are massively parallel limited purpose CPUs. You're always waiting for the program counter to get to the end, and getting data to/from Ram. My project (since the 1980s) is to put compute and memory together, so you can either store 16 bits, or do a 4 bit look up with the same data in every cell (which has 4x16 bits total). Having no memory-compute barrier should allow for pushing data through a program, in a physical sense. Like an FPGA, but easier to program. Imagine such a chip at a few Ghz, delivering one output per clock cycle for a given task.
- jeffreygoesto 2y ago
  In my experience the step of leaving von Neumann architecture and programming that data flows across the compute engines is the more difficult step than the crappy FPGA tools. I tell people that with an FPGA you teach the computer what to be, not what to do. A bit like functional vs. imperative programming. Therefore I can't yet imagine how to easier onboard people who learned on a "classical CPU". Can you go a bit deeper in what makes your system easier to program?
  - mikewarot 2y ago
    FPGAs are heterogeneous, and the tools are all about getting the maximum utilization of a given chip's features while also routing signals to get the fastest reliable cycle times. Compile times sometimes stretch out into overnight sessions. By making each cell identical you remove most of the floor planning concerns. By latching the data from each cell, you remove timing concerns, and deliver deterministic performance. This has other benefits like being possible to route around a bad cell, or to build a "wall" around a given section of code. The emulator will always give correct output baring hardware failure. I hope to get through tiny Tapeout and have a chip before the end of the year. As for programming, the code has to compile to a directed acyclic graph of boolean logical operations. I suspect that it would be fairly easy to use as a backend for tinygrad or perhaps LLVM as a stretch goal.
    - jeffreygoesto 2y ago
      Ah thanks! Reminds me of systolic arrays somehow. Good luck and hopefully there are no bugs in your design!
- TheMode 2y ago
  Are you speaking about a sort of (multi-state) cellular automata?
  - mikewarot 2y ago
    No, each cell has it's own 4x4 bit look up table (64 bits of "program" or "data" depending on the mode of that cell), so it's more like an FPGA with the routing fabric stripped out.
    - nynx 2y ago
      This sounds like just an fpga. What do you mean by “routing fabric stripped out?” How does data get from one cell to another?
      - mikewarot 2y ago
        Each cell only hooks to it's neighbors, up, down, left, right... so no "fast" wires across the chip, etc.
        jldugger 2y ago
        This sounds like the Systolic Array (https://en.wikipedia.org/wiki/Systolic_array https://en.wikipedia.org/wiki/Systolic_array) design google based its TensorFlow Processing Units on.
        squantor 2y ago
        This really sounds like the Xilinx XC6200 series FPGA. It has a few routing hierarchies but does strip out most of the routing a classical (for that time) FPGA has. I heard numbers that 70% of a FPGA is routing. The XC6200 became news once as they[1] used an evolutionary algorithm to create a configuration that can detect a single frequency tone. The resulting configuration was determined that it was impossible that it could work, but it did. Placing the configuration in a different part of the FPGA broke it. It seemed that algorithm used the analog properties of the FPGA in that specific location to get to a smaller result. [1] https://www.idi.ntnu.no/emner/tdt22/2011/Thompsonieeeehw.pdf https://www.idi.ntnu.no/emner/tdt22/2011/Thompsonieeeehw.pdf
TomK32 2y ago
Trains in general a great, why do people insist on sitting without any company in an expensive car which they even have to drive themselves?
- alwa 2y ago
  The gp comment speaks more of freight than of passengers. But the same analogy applies… trains are great when you need to transport a whole bunch of people exactly the same way, but they’re less useful if every person needs to start and end at a totally different place. In the same sense, parallelization (and computing on a GPU) are great when you need to perform the same computation on a huge number of data. When each computation is more independent, it might make more sense to perform it through the CPU.
- deleted 2y ago
  [deleted]
amanaplanacanal 2y ago
Trucks are better at branching, but also heavily subsidized. In the 19th century when trains were being run everywhere, they were heavily subsidized too.

deleted 2y ago

[deleted]

mikewarot 2y ago

Freight trains are good, but pipelines are better for bulk liquids. GPUs are massively parallel limited purpose CPUs. You're always waiting for the program counter to get to the end, and getting data to/from Ram. My project (since the 1980s) is to put compute and memory together, so you can either store 16 bits, or do a 4 bit look up with the same data in every cell (which has 4x16 bits total). Having no memory-compute barrier should allow for pushing data through a program, in a physical sense. Like an FPGA, but easier to program. Imagine such a chip at a few Ghz, delivering one output per clock cycle for a given task.

jeffreygoesto 2y ago
In my experience the step of leaving von Neumann architecture and programming that data flows across the compute engines is the more difficult step than the crappy FPGA tools. I tell people that with an FPGA you teach the computer what to be, not what to do. A bit like functional vs. imperative programming. Therefore I can't yet imagine how to easier onboard people who learned on a "classical CPU". Can you go a bit deeper in what makes your system easier to program?
- mikewarot 2y ago
  FPGAs are heterogeneous, and the tools are all about getting the maximum utilization of a given chip's features while also routing signals to get the fastest reliable cycle times. Compile times sometimes stretch out into overnight sessions. By making each cell identical you remove most of the floor planning concerns. By latching the data from each cell, you remove timing concerns, and deliver deterministic performance. This has other benefits like being possible to route around a bad cell, or to build a "wall" around a given section of code. The emulator will always give correct output baring hardware failure. I hope to get through tiny Tapeout and have a chip before the end of the year. As for programming, the code has to compile to a directed acyclic graph of boolean logical operations. I suspect that it would be fairly easy to use as a backend for tinygrad or perhaps LLVM as a stretch goal.
  - jeffreygoesto 2y ago
    Ah thanks! Reminds me of systolic arrays somehow. Good luck and hopefully there are no bugs in your design!
TheMode 2y ago
Are you speaking about a sort of (multi-state) cellular automata?
- mikewarot 2y ago
  No, each cell has it's own 4x4 bit look up table (64 bits of "program" or "data" depending on the mode of that cell), so it's more like an FPGA with the routing fabric stripped out.
  - nynx 2y ago
    This sounds like just an fpga. What do you mean by “routing fabric stripped out?” How does data get from one cell to another?
    - mikewarot 2y ago
      Each cell only hooks to it's neighbors, up, down, left, right... so no "fast" wires across the chip, etc.
      - jldugger 2y ago
        This sounds like the Systolic Array (https://en.wikipedia.org/wiki/Systolic_array https://en.wikipedia.org/wiki/Systolic_array) design google based its TensorFlow Processing Units on.
      - squantor 2y ago
        This really sounds like the Xilinx XC6200 series FPGA. It has a few routing hierarchies but does strip out most of the routing a classical (for that time) FPGA has. I heard numbers that 70% of a FPGA is routing. The XC6200 became news once as they[1] used an evolutionary algorithm to create a configuration that can detect a single frequency tone. The resulting configuration was determined that it was impossible that it could work, but it did. Placing the configuration in a different part of the FPGA broke it. It seemed that algorithm used the analog properties of the FPGA in that specific location to get to a smaller result. [1] https://www.idi.ntnu.no/emner/tdt22/2011/Thompsonieeeehw.pdf https://www.idi.ntnu.no/emner/tdt22/2011/Thompsonieeeehw.pdf

TomK32 2y ago

Trains in general a great, why do people insist on sitting without any company in an expensive car which they even have to drive themselves?

alwa 2y ago
The gp comment speaks more of freight than of passengers. But the same analogy applies… trains are great when you need to transport a whole bunch of people exactly the same way, but they’re less useful if every person needs to start and end at a totally different place. In the same sense, parallelization (and computing on a GPU) are great when you need to perform the same computation on a huge number of data. When each computation is more independent, it might make more sense to perform it through the CPU.
deleted 2y ago
[deleted]

amanaplanacanal 2y ago

Trucks are better at branching, but also heavily subsidized. In the 19th century when trains were being run everywhere, they were heavily subsidized too.