5 ms·

Kinda disappointed that Tenstorrent is pushing one of the world's most boring architectures. Its a regular chip with a few extra units bolted on the end, thats

Kinda disappointed that Tenstorrent is pushing one of the world's most boring architectures. Its a regular chip with a few extra units bolted on the end, thats been copy and pasted in a grid. We've seen the same stuff from folks like Meta, even Tesla.

mk_stjames 3y ago
I noted how, when they first started, they talked mostly about developing 'AI accelerators' and it felt mostly like they were talking about big, GPGPU style chips to go head to head with nVidia. Thousands of small SIMD cores doing matrix multiplies, with fast memory and pcie. Maybe something halfway between Cerebras size and and Nvidia Hopper. A tall order but something really needed. Then at some point it feels like Jim got hooked on the idea of RISC-V everything, and they pivoted their messaging to talking about these more CPU-like chips with a main R-V64 8-wide decoding, state of the art OoO execution, etc. That sounds more like a RISC-V competitor to AMD Zen instead of a competitor to an nVidia GPU. And they they talk about that just being the interface to the AI chip later but... it really feels like that saw 'hey we can get all this RISC-V stuff for free essentially, and really take over the development of the spec, and that is easier than figuring out how to develop an GP AI chip and a stack that competes with CUDA to go with it, so that is easier to start...' I'm totally a non-expert though and the preceding is just what I've picked up from watching interviews with Jim (who I just find awesome to listen to).
- mtkhaos 3y ago
  Logic is simple inline with reducing power draw from a simplistic instruction set. Move into a space where we have rapid manufacturing for Specialized chips. Alongside the concept inherit to Nvidia's DPU and you have something very Interesting.
- kevin_thibedeau 3y ago
  It makes more business sense to have more general purpose hardware that can be pivoted to other applications. Lots of AI ASIC vendors are going to go belly up in the coming years as their platforms fail to attract customers. Carving out a tiny niche with limited demand and no IP moat is very risky in the IC world.
- panick21_ 3y ago
  Fast CPU performance is necessary for AI workloads too. You need a fast CPU and combine that with lots of Vector or Tensor processing. Lots of applications need both. They have done both for a while.
- blihp 3y ago
  In the interview he runs down the issues they encountered going down the pure AI accelerator path. It sounds like they've decided the opportunity wasn't there (i.e. too hard) so they've pivoted.
DesiLurker 3y ago
sounds like a variation of that old MIT startup called Tilera, very lightweight CPUs with high performance interconnect fabric. at that time I remember thinking it was a solution looking for a problem. IDK whats wrong with that architecture for AI/ML but I feel too much overhead in full on CPUs. I guess thats where lightweight risc cores come in. Personally what I'd like to see is a clever architecture utilizing a grid architecture with a stack based process for communicating with local nodes using a bare minimal language like forth so extremely light 'nodes' to do matrix math & nothing else.
- baybal2 3y ago
  [dead]
- my123 3y ago
  Tilera itself is owned by NVIDIA nowadays.
ftxbro 3y ago
I think boring can be good, I'm not an expert but 'extra units copy and pasted in a grid' is exactly how I imagine a hardware AI accelerator to be like
- zackmorris 3y ago
  I agree, and I've wanted a grid of 1000+ cores for 25 years now, once I realized that the only bottleneck in computing is the bus between CPU and memory back in the late 90s. The only chip that comes even close to what I want for a reasonable price is Apple's M1 line, but they added GPU and AI cores which defeat the purpose. The churn with GPUs and now AI cores is too much for me. I just can't waste time manually annotating the code I want to run concurrently, on GPU, on AI core, whatever. To me, it looks like everyone is putting 10-100 times more work into their code than what should be needed. I see the same pattern repeated with web development and the explosion of walled garden platforms for mobile and smart devices. So much work for so long for so few results. Just give me a big dumb boring grid of cores and a self-parallelizing language to program them. Stuff like Julia/Clojure/MATLAB/Erlang/Go come close, but each have poison pills that make reaching the mainstream untenable. Someday I want to write a language that does what I need, but that day will likely never come, because every day is Quantum Leap for me, I just go to work to make rent, pushing the rock up that hill like Sisyphus, only to watch it roll down and have to start all over again. Inadequate tooling has come to totally dominate every aspect of my work, but there may never be time to write better stuff.
  - ftxbro 3y ago
    > "Stuff like Julia/Clojure/MATLAB/Erlang/Go come close, but each have poison pills that make reaching the mainstream untenable." they have 'mojo' now but it might be proprietary like matlab or mathematica
    - snvzz 3y ago
      Still waiting for them to explain what "poison pills" are there. These projects (edit: save MATLAB) are open source, they have the four freedoms.
comboy 3y ago
I think you are oversimplifying. You've chosen arbitrary abstraction layer. It's like saying it's just transistors again. Or it's just machine learning. It matters what they can deliver and Jim's track record is best in the whole industry. I think they have a pretty good understanding what's the best approach to bring outstanding results given technology available plus I think currently they want to deliver something to get connected with decent clients and then be able to optimize for actual real world use cases. I'd love to see chips made out of millions of small computing blocks, neuron alike, without a common clock, with local memory, maybe even with some analog electronics involved. But I'm pretty sure people who are actually working on this kind of stuff could provide me a list of reasons why it's silly (at least given current technology limitations).
- frozenport 3y ago
  Lol. If they knew they wouldn’t be pivoting. Jim doesn’t come out a technical person, he seems mostly about realizing other peoples ideas or IP. Would have been great to see him leading something revolutionary.
  - comboy 3y ago
    > Jim doesn’t come out a technical person, Hhahahahahahahahhaahahaha
    - B1FF_PSUVM 3y ago
      https://news.ycombinator.com/lowlights https://news.ycombinator.com/lowlights needed (I recently found https://news.ycombinator.com/highlights https://news.ycombinator.com/highlights )

mk_stjames 3y ago

I noted how, when they first started, they talked mostly about developing 'AI accelerators' and it felt mostly like they were talking about big, GPGPU style chips to go head to head with nVidia. Thousands of small SIMD cores doing matrix multiplies, with fast memory and pcie. Maybe something halfway between Cerebras size and and Nvidia Hopper. A tall order but something really needed. Then at some point it feels like Jim got hooked on the idea of RISC-V everything, and they pivoted their messaging to talking about these more CPU-like chips with a main R-V64 8-wide decoding, state of the art OoO execution, etc. That sounds more like a RISC-V competitor to AMD Zen instead of a competitor to an nVidia GPU. And they they talk about that just being the interface to the AI chip later but... it really feels like that saw 'hey we can get all this RISC-V stuff for free essentially, and really take over the development of the spec, and that is easier than figuring out how to develop an GP AI chip and a stack that competes with CUDA to go with it, so that is easier to start...' I'm totally a non-expert though and the preceding is just what I've picked up from watching interviews with Jim (who I just find awesome to listen to).

mtkhaos 3y ago
Logic is simple inline with reducing power draw from a simplistic instruction set. Move into a space where we have rapid manufacturing for Specialized chips. Alongside the concept inherit to Nvidia's DPU and you have something very Interesting.
kevin_thibedeau 3y ago
It makes more business sense to have more general purpose hardware that can be pivoted to other applications. Lots of AI ASIC vendors are going to go belly up in the coming years as their platforms fail to attract customers. Carving out a tiny niche with limited demand and no IP moat is very risky in the IC world.
panick21_ 3y ago
Fast CPU performance is necessary for AI workloads too. You need a fast CPU and combine that with lots of Vector or Tensor processing. Lots of applications need both. They have done both for a while.
blihp 3y ago
In the interview he runs down the issues they encountered going down the pure AI accelerator path. It sounds like they've decided the opportunity wasn't there (i.e. too hard) so they've pivoted.

DesiLurker 3y ago

sounds like a variation of that old MIT startup called Tilera, very lightweight CPUs with high performance interconnect fabric. at that time I remember thinking it was a solution looking for a problem. IDK whats wrong with that architecture for AI/ML but I feel too much overhead in full on CPUs. I guess thats where lightweight risc cores come in. Personally what I'd like to see is a clever architecture utilizing a grid architecture with a stack based process for communicating with local nodes using a bare minimal language like forth so extremely light 'nodes' to do matrix math & nothing else.

baybal2 3y ago
[dead]
my123 3y ago
Tilera itself is owned by NVIDIA nowadays.

ftxbro 3y ago

I think boring can be good, I'm not an expert but 'extra units copy and pasted in a grid' is exactly how I imagine a hardware AI accelerator to be like

zackmorris 3y ago
I agree, and I've wanted a grid of 1000+ cores for 25 years now, once I realized that the only bottleneck in computing is the bus between CPU and memory back in the late 90s. The only chip that comes even close to what I want for a reasonable price is Apple's M1 line, but they added GPU and AI cores which defeat the purpose. The churn with GPUs and now AI cores is too much for me. I just can't waste time manually annotating the code I want to run concurrently, on GPU, on AI core, whatever. To me, it looks like everyone is putting 10-100 times more work into their code than what should be needed. I see the same pattern repeated with web development and the explosion of walled garden platforms for mobile and smart devices. So much work for so long for so few results. Just give me a big dumb boring grid of cores and a self-parallelizing language to program them. Stuff like Julia/Clojure/MATLAB/Erlang/Go come close, but each have poison pills that make reaching the mainstream untenable. Someday I want to write a language that does what I need, but that day will likely never come, because every day is Quantum Leap for me, I just go to work to make rent, pushing the rock up that hill like Sisyphus, only to watch it roll down and have to start all over again. Inadequate tooling has come to totally dominate every aspect of my work, but there may never be time to write better stuff.
- ftxbro 3y ago
  > "Stuff like Julia/Clojure/MATLAB/Erlang/Go come close, but each have poison pills that make reaching the mainstream untenable." they have 'mojo' now but it might be proprietary like matlab or mathematica
  - snvzz 3y ago
    Still waiting for them to explain what "poison pills" are there. These projects (edit: save MATLAB) are open source, they have the four freedoms.

comboy 3y ago

I think you are oversimplifying. You've chosen arbitrary abstraction layer. It's like saying it's just transistors again. Or it's just machine learning. It matters what they can deliver and Jim's track record is best in the whole industry. I think they have a pretty good understanding what's the best approach to bring outstanding results given technology available plus I think currently they want to deliver something to get connected with decent clients and then be able to optimize for actual real world use cases. I'd love to see chips made out of millions of small computing blocks, neuron alike, without a common clock, with local memory, maybe even with some analog electronics involved. But I'm pretty sure people who are actually working on this kind of stuff could provide me a list of reasons why it's silly (at least given current technology limitations).

frozenport 3y ago
Lol. If they knew they wouldn’t be pivoting. Jim doesn’t come out a technical person, he seems mostly about realizing other peoples ideas or IP. Would have been great to see him leading something revolutionary.
- comboy 3y ago
  > Jim doesn’t come out a technical person, Hhahahahahahahahhaahahaha
  - B1FF_PSUVM 3y ago
    https://news.ycombinator.com/lowlights https://news.ycombinator.com/lowlights needed (I recently found https://news.ycombinator.com/highlights https://news.ycombinator.com/highlights )