5 ms·

I'm not any kind of expert in the field, but trading single chip speed for more chips surely has it's downsides, which aren't mentioned in the article at all.

jmercouris 5y ago
Of course it does, usually systems have slower and faster compute units to compensate for performance penalty of non parallelisable operations
mirker 5y ago
Yes, but it’s meant to do ML inference, which can be parallelized decently. On those workloads, you can use GPUs, which are also composed of thousands of “wimpy” cores.
- R0b0t1 5y ago
  A full CPU is useful for decision intensive or time series intensive data. Normal ML inference is not necessarily either of those. You could have more complicated neurons (or just make normal compute tiles which they may be doing).
  - goldenkey 5y ago
    I thought the same thing back in 2015 considering the way GPUs supposedly handle branches with warps. However, my stock trading simulator ran way better on 3 GTX Titans rather than the Intel "Knights Many Cores" Phi preview I had exclusively been able to obtain. I was excited because it had something like 100 pentium 4 cores on it, and was supposed to be much faster than a GPU for logical code. Dissapointment set in when the GPU stomped it performance wise. I still don't even understand why but I do know now that the whole "GPUs can't handle branching performantly" is a bit overstated. Intel discontinued their Phi which I can only gander was due to its lack of competitiveness.
    - sdenton4 5y ago
      A standard way to handle branching in gpu code is with masking, like so (where x is a vector, and broadcasting is implied): M = x > 0 y = M * f(x) + (1-M) * g(x) So you end up evaluating both sides of the decision branch and adding the results. But this is fine if you've got a dumb number of cores. And often traditional cpus wind up evaluating both branches anyway.
      - monocasa 5y ago
        > And often traditional cpus wind up evaluating both branches anyway. That's actually really overstated. Evaluating both sides isn't really something CPUs tend to do, but instead predict one path and roll back on mispredict. This is because the out of order hardware isn't a tree fundamentally, but generally better thought of a ring buffer where uncommitted state is what's between the head and tail. Storing diverging paths is incredibly expensive there. I'm not going to say something as strong as "it's never been done", but but I certainly don't know of an general purpose CPU arch that'll compute both sides of an architectural branch, instead relying an making good predictions down one instruction stream, then rolling back and restarting when it's clear you mispredicted.
        jasonwatkinspdx 5y ago
        It's not even about the expense of implementing diverging paths in hardware. This concept, of exploring like a tree vs a path was explored under the name Disjoint Eager Execution. You know what killed it? Branch predictors. In a world where branch predictors are maybe only 75% effective, DEE could make sense. We live in a world where branch predictors are far better than that. So it just isn't worth speculating off the predicted most likely path.
        monocasa 5y ago
        What killed it was more the effectiveness of Tomasulo style out of order machinery, and the real problem not being control hazards, but data hazards. DEE was thought of in a day where memory was about as fast as the processor. That's why it's always being compared with cores like R3000.
- monocasa 5y ago
  Sort of. GPU "cores" in the CPU space would be called SIMD lanes. Apples to apples GPU cores using the CPU terminology would put an Nvidia 3060 at 28 cores and a 3090 at 82 cores.
cogman10 5y ago
Biggest that comes up quick is memory bandwidth. The more cores you have, the more memory is needed to keep the chips processing.
- touisteur 5y ago
  Well, it really depends on the computational intensity your algorithm needs. I've stumbled upon things of beauty porting things to GPUs, especially if you're going to perform huge amount of operations based on a very small amount of data. As long as you don't have too much intermediate data, register spilling, etc. these GPU things do fly. They're also very impressive on NN-based workloads... Even something 2 or 3 gens behind can be game changing, with some optimization effort. Tensor libraries leave a lot on the floor to pick up, especially if you're not using the canned 'competition winning' networks.
  - cogman10 5y ago
    Nature of the workload certainly matters a lot, and for a lot of work GPUs do memory bandwidth isn't always the limiting factor (though it often is, that's why consumer grade GPUs have 12GB of ram and beefier systems grade GPGPUs/TPUs have 40+GB of memory). Datastructure and locality matters a lot for GPGPU programming. While PCI-Express is really fast, it's got a lot of latency and is limited.
    - mirker 5y ago
      Nit but bandwidth isn’t related to memory capacity. Bandwidth is a function of cache/access pattern and is independent of total memory size.
      - touisteur 5y ago
        The generous interpretation here would be that having huge amounts of ram that you can't access quickly enough wouldn't be that useful. I'm curious though, apart from huge NNs and raytracing scenes, what use cases call for so much RAM. I mean, apart from content lookup tables?
Mikeb85 5y ago
Read the article. It's about ML workloads which scale well across many cores. It's also being compared to GPUs. The whole point of what they're doing to is to be able to pack more cores versus a CPU but with a larger instruction set than a GPU core.
- zozbot234 5y ago
  I like ML but it's not a very good language for this highly parallel HPC'ish stuff. We'll see how Rust does, it should be a lot closer to what's actually needed here.
  - medo-bear 5y ago
    ML as in machine learning
klyrs 5y ago
Sure, but this is a coprocessor on an expansion card, similar to a GPU. I've worked on a few systolic algorithms and this kind of chip has massive potential in that space. TPUs have been a big letdown in that regard, as they don't even have the comparison operation needed for the matrix-based shortest-path algorithm.
freemint 5y ago
I would not take much to acknowledge them, just a "fastest RISC-V chip in many core workloads" would a long way. I personally think those chips would be an absolute monster for solving MILP problems as they tend to have enormous parallelism and a lot of linear algebra (in particular simplex iterations). However there is no hype for funding in Mixed-Integer Linear Programming so machine learning it is.

jmercouris 5y ago

Of course it does, usually systems have slower and faster compute units to compensate for performance penalty of non parallelisable operations

mirker 5y ago

Yes, but it’s meant to do ML inference, which can be parallelized decently. On those workloads, you can use GPUs, which are also composed of thousands of “wimpy” cores.

R0b0t1 5y ago
A full CPU is useful for decision intensive or time series intensive data. Normal ML inference is not necessarily either of those. You could have more complicated neurons (or just make normal compute tiles which they may be doing).
- goldenkey 5y ago
  I thought the same thing back in 2015 considering the way GPUs supposedly handle branches with warps. However, my stock trading simulator ran way better on 3 GTX Titans rather than the Intel "Knights Many Cores" Phi preview I had exclusively been able to obtain. I was excited because it had something like 100 pentium 4 cores on it, and was supposed to be much faster than a GPU for logical code. Dissapointment set in when the GPU stomped it performance wise. I still don't even understand why but I do know now that the whole "GPUs can't handle branching performantly" is a bit overstated. Intel discontinued their Phi which I can only gander was due to its lack of competitiveness.
  - sdenton4 5y ago
    A standard way to handle branching in gpu code is with masking, like so (where x is a vector, and broadcasting is implied): M = x > 0 y = M * f(x) + (1-M) * g(x) So you end up evaluating both sides of the decision branch and adding the results. But this is fine if you've got a dumb number of cores. And often traditional cpus wind up evaluating both branches anyway.
    - monocasa 5y ago
      > And often traditional cpus wind up evaluating both branches anyway. That's actually really overstated. Evaluating both sides isn't really something CPUs tend to do, but instead predict one path and roll back on mispredict. This is because the out of order hardware isn't a tree fundamentally, but generally better thought of a ring buffer where uncommitted state is what's between the head and tail. Storing diverging paths is incredibly expensive there. I'm not going to say something as strong as "it's never been done", but but I certainly don't know of an general purpose CPU arch that'll compute both sides of an architectural branch, instead relying an making good predictions down one instruction stream, then rolling back and restarting when it's clear you mispredicted.
      - jasonwatkinspdx 5y ago
        It's not even about the expense of implementing diverging paths in hardware. This concept, of exploring like a tree vs a path was explored under the name Disjoint Eager Execution. You know what killed it? Branch predictors. In a world where branch predictors are maybe only 75% effective, DEE could make sense. We live in a world where branch predictors are far better than that. So it just isn't worth speculating off the predicted most likely path.
        monocasa 5y ago
        What killed it was more the effectiveness of Tomasulo style out of order machinery, and the real problem not being control hazards, but data hazards. DEE was thought of in a day where memory was about as fast as the processor. That's why it's always being compared with cores like R3000.
monocasa 5y ago
Sort of. GPU "cores" in the CPU space would be called SIMD lanes. Apples to apples GPU cores using the CPU terminology would put an Nvidia 3060 at 28 cores and a 3090 at 82 cores.

cogman10 5y ago

Biggest that comes up quick is memory bandwidth. The more cores you have, the more memory is needed to keep the chips processing.

touisteur 5y ago
Well, it really depends on the computational intensity your algorithm needs. I've stumbled upon things of beauty porting things to GPUs, especially if you're going to perform huge amount of operations based on a very small amount of data. As long as you don't have too much intermediate data, register spilling, etc. these GPU things do fly. They're also very impressive on NN-based workloads... Even something 2 or 3 gens behind can be game changing, with some optimization effort. Tensor libraries leave a lot on the floor to pick up, especially if you're not using the canned 'competition winning' networks.
- cogman10 5y ago
  Nature of the workload certainly matters a lot, and for a lot of work GPUs do memory bandwidth isn't always the limiting factor (though it often is, that's why consumer grade GPUs have 12GB of ram and beefier systems grade GPGPUs/TPUs have 40+GB of memory). Datastructure and locality matters a lot for GPGPU programming. While PCI-Express is really fast, it's got a lot of latency and is limited.
  - mirker 5y ago
    Nit but bandwidth isn’t related to memory capacity. Bandwidth is a function of cache/access pattern and is independent of total memory size.
    - touisteur 5y ago
      The generous interpretation here would be that having huge amounts of ram that you can't access quickly enough wouldn't be that useful. I'm curious though, apart from huge NNs and raytracing scenes, what use cases call for so much RAM. I mean, apart from content lookup tables?

Mikeb85 5y ago

Read the article. It's about ML workloads which scale well across many cores. It's also being compared to GPUs. The whole point of what they're doing to is to be able to pack more cores versus a CPU but with a larger instruction set than a GPU core.

zozbot234 5y ago
I like ML but it's not a very good language for this highly parallel HPC'ish stuff. We'll see how Rust does, it should be a lot closer to what's actually needed here.
- medo-bear 5y ago
  ML as in machine learning

klyrs 5y ago

Sure, but this is a coprocessor on an expansion card, similar to a GPU. I've worked on a few systolic algorithms and this kind of chip has massive potential in that space. TPUs have been a big letdown in that regard, as they don't even have the comparison operation needed for the matrix-based shortest-path algorithm.

freemint 5y ago

I would not take much to acknowledge them, just a "fastest RISC-V chip in many core workloads" would a long way. I personally think those chips would be an absolute monster for solving MILP problems as they tend to have enormous parallelism and a lot of linear algebra (in particular simplex iterations). However there is no hype for funding in Mixed-Integer Linear Programming so machine learning it is.