5 ms·

At the lower end of your range this sounds somewhat like a Xeon Phi - up to 72 cores at 1.5 GHz with 512 kb L2 cache per core. Intel has discontinued the Xeon P

At the lower end of your range this sounds somewhat like a Xeon Phi - up to 72 cores at 1.5 GHz with 512 kb L2 cache per core. Intel has discontinued the Xeon Phi line, presumably in part because there were limited applications where such an architecture proved compelling. What advantages would you see your FPGA based architecture having over something like the Xeon Phi that might lead it to have more success?

zackmorris 7y ago
That's a good question, and I'm honestly not sure. I think highly parallel, general purpose and customizable computers have uses outside the mainstream. So they would make great ray tracing video cards and platforms for physics simulations for science research, stuff like that. I think most problems in computer science today can be traced to the original sin of choosing Von Neumann architecture and imperative programming over reprogrammable hardware and functional programming. That's why we think of things like machine learning as rather difficult problems, even though they are just simple algorithms that are compute-bound. Basically my complaint is that this stuff has progressed in spite of our choices, not because of them. The proprietary and closed-source hardware, drivers and programming paradigms we have to deal with today have set us back at least 20 years. Probably more like 40 years if you include some of the shenanigans that happened between IBM, Intel, Microsoft and Apple in the 80s and 90s. They solved personal computer problems wonderfully, but inadvertently hindered the progress of computer science in general. I don't have an answer for why the market tends to reject the stuff I talk about. I just know that mathematically it's very first-principles stuff that I really hoped would have caught on by now. To me it feels like these ideas keep getting sabotaged for economic and political reasons rather than scientific ones. Edit: I forgot to mention that FPGAs solve this problem by providing generic reprogrammable hardware whose use can't be monopolized the way that single-purpose hardware is. Something like open source FPGAs (that don't burn up when they're programmed wrong) would allow us to explore many more problem spaces in computer science than we can currently.
- mattnewport 7y ago
  Xeon Phi evolved from Larrabee which was Intel's attempt to make a GPU based around highly parallel general purpose x86 cores. Many smart people thought it was a good idea and worked on it but it failed. Xeon Phi in theory might be good for ray tracing but all the action in ray tracing acceleration today is with custom hardware added to traditional GPUs. Intel poured a lot of resources into this approach and it didn't find success so I'm not really sure you can say it was sabotaged for economic or political reasons, rather it failed to find a market. I think the reasons it failed are complex and perhaps it's an idea whose time has just yet to come rather than being a bad idea but it has been tried without much success more than once. I backed the Parallella Kickstarter and that didn't really go anywhere either.
  - zackmorris 7y ago
    Ya I actually bought an Nvidia GeForce RTX 2070, specifically to play around with its ray tracing capabilities. I kind of wish I could play around with a Google TensorFlow processor (TPU): https://en.wikipedia.org/wiki/Tensor_processing_unit https://en.wikipedia.org/wiki/Tensor_processing_unit It's an ok chip and has some of the ideas that I mentioned (like an 8 bit design that could possibly emulate 64 bit and higher via microcode or software) but I'm afraid that it's too narrowly scoped to be of much use in other domains. Companies seem to be exploring branches of the parallel/DSP problem space rather than providing general solutions like the Xeon Phi. I'm not totally sold on high multicore (say 64 or more cores), but it's the only way that I can see forward since Moore's law ended. It would be much better to have increased single-threaded performance since that can emulate multicore. Multicore can't emulate single-threaded above Amdahl's limit of about 4x, which is why very few chips today have more than 4 or 8 cores. That said, most of the problems that I find interesting are also embarrassingly parallel. And since there is currently no consumer high-multicore processor, I can't explore those problem spaces. From first principles, it should have gone: 1) Single core: 6502, z80, 286 2) Low multicore: Pentium, POWER 3) High multicore (MIMD): Xeon Phi, Parallella 4) Optimized vector processing (SIMD): GPU, TPU Without a viable #3, we have no way of knowing if #4 is an optimal solution or an evolutionary dead end. We have to drink the kool-aid and proceed out of trust. That's the part that I find extremely disappointing about the last 20-25 years. Another way to say that is: if there is no market fit for ever-faster processors at cheaper prices.. that consumers would rather have low-cost mobile processors and regressively rent time on data centers.. that they'd rather have photorealistic rasterized video games than true supercomputing at home, then computer science is at the very least wounded (and I would argue, slowly dying). This is a nuanced issue that the mainstream isn't even aware of. I'm excited to see some people trying to get projects like Parallella off the ground. From what I can tell, they are one of the only chips working from first principles: https://en.wikipedia.org/wiki/Adapteva https://en.wikipedia.org/wiki/Adapteva But as you pointed out, it may take more than enlightened engineering to bridge the gap to next-gen computing. That's the reason why we are only today seeing advances in machine learning that were fairly well understood by 2000. I just don't want to wait around another 20 years for the other good stuff to get here. Which is why I think it might take an open source design on inexpensive FPGA hardware to show people what they're missing.

zackmorris 7y ago

That's a good question, and I'm honestly not sure. I think highly parallel, general purpose and customizable computers have uses outside the mainstream. So they would make great ray tracing video cards and platforms for physics simulations for science research, stuff like that. I think most problems in computer science today can be traced to the original sin of choosing Von Neumann architecture and imperative programming over reprogrammable hardware and functional programming. That's why we think of things like machine learning as rather difficult problems, even though they are just simple algorithms that are compute-bound. Basically my complaint is that this stuff has progressed in spite of our choices, not because of them. The proprietary and closed-source hardware, drivers and programming paradigms we have to deal with today have set us back at least 20 years. Probably more like 40 years if you include some of the shenanigans that happened between IBM, Intel, Microsoft and Apple in the 80s and 90s. They solved personal computer problems wonderfully, but inadvertently hindered the progress of computer science in general. I don't have an answer for why the market tends to reject the stuff I talk about. I just know that mathematically it's very first-principles stuff that I really hoped would have caught on by now. To me it feels like these ideas keep getting sabotaged for economic and political reasons rather than scientific ones. Edit: I forgot to mention that FPGAs solve this problem by providing generic reprogrammable hardware whose use can't be monopolized the way that single-purpose hardware is. Something like open source FPGAs (that don't burn up when they're programmed wrong) would allow us to explore many more problem spaces in computer science than we can currently.

mattnewport 7y ago
Xeon Phi evolved from Larrabee which was Intel's attempt to make a GPU based around highly parallel general purpose x86 cores. Many smart people thought it was a good idea and worked on it but it failed. Xeon Phi in theory might be good for ray tracing but all the action in ray tracing acceleration today is with custom hardware added to traditional GPUs. Intel poured a lot of resources into this approach and it didn't find success so I'm not really sure you can say it was sabotaged for economic or political reasons, rather it failed to find a market. I think the reasons it failed are complex and perhaps it's an idea whose time has just yet to come rather than being a bad idea but it has been tried without much success more than once. I backed the Parallella Kickstarter and that didn't really go anywhere either.
- zackmorris 7y ago
  Ya I actually bought an Nvidia GeForce RTX 2070, specifically to play around with its ray tracing capabilities. I kind of wish I could play around with a Google TensorFlow processor (TPU): https://en.wikipedia.org/wiki/Tensor_processing_unit https://en.wikipedia.org/wiki/Tensor_processing_unit It's an ok chip and has some of the ideas that I mentioned (like an 8 bit design that could possibly emulate 64 bit and higher via microcode or software) but I'm afraid that it's too narrowly scoped to be of much use in other domains. Companies seem to be exploring branches of the parallel/DSP problem space rather than providing general solutions like the Xeon Phi. I'm not totally sold on high multicore (say 64 or more cores), but it's the only way that I can see forward since Moore's law ended. It would be much better to have increased single-threaded performance since that can emulate multicore. Multicore can't emulate single-threaded above Amdahl's limit of about 4x, which is why very few chips today have more than 4 or 8 cores. That said, most of the problems that I find interesting are also embarrassingly parallel. And since there is currently no consumer high-multicore processor, I can't explore those problem spaces. From first principles, it should have gone: 1) Single core: 6502, z80, 286 2) Low multicore: Pentium, POWER 3) High multicore (MIMD): Xeon Phi, Parallella 4) Optimized vector processing (SIMD): GPU, TPU Without a viable #3, we have no way of knowing if #4 is an optimal solution or an evolutionary dead end. We have to drink the kool-aid and proceed out of trust. That's the part that I find extremely disappointing about the last 20-25 years. Another way to say that is: if there is no market fit for ever-faster processors at cheaper prices.. that consumers would rather have low-cost mobile processors and regressively rent time on data centers.. that they'd rather have photorealistic rasterized video games than true supercomputing at home, then computer science is at the very least wounded (and I would argue, slowly dying). This is a nuanced issue that the mainstream isn't even aware of. I'm excited to see some people trying to get projects like Parallella off the ground. From what I can tell, they are one of the only chips working from first principles: https://en.wikipedia.org/wiki/Adapteva https://en.wikipedia.org/wiki/Adapteva But as you pointed out, it may take more than enlightened engineering to bridge the gap to next-gen computing. That's the reason why we are only today seeing advances in machine learning that were fairly well understood by 2000. I just don't want to wait around another 20 years for the other good stuff to get here. Which is why I think it might take an open source design on inexpensive FPGA hardware to show people what they're missing.