4 ms·

Its 2023 and I still don't understand the point of CPU speeds, even as a dev doing both single and multithreading. AI is continuing to put priority on GPUs. In

Its 2023 and I still don't understand the point of CPU speeds, even as a dev doing both single and multithreading. AI is continuing to put priority on GPUs. In the worst case, I wait an extra fraction of a second with a bad CPU. With a bad GPU, you are basically excluded from the latest technology. I'm sure ARM really wants to advertise that CPU is relevant, but it continues to be nearly irrelevant when you buy something a laptop with a dedicated GPU. (Its good enough)

oneplane 3y ago
I suppose it depends on what performance bracket you are targeting. Scripting and interpreted languages, JIT next and VM after that (i.e. JVM, CLR), native code after that and optimised low-level code after that would all have very different reactions to 'good CPU' vs. 'bad CPU'. A 'slow' process (±10s) taking 1 second longer won't seem that much slower, but a fast process (±100ms) taking ten times as long does definitely become problematic. As for AI, it's mostly irrelevant what your local machines does, unless you are an end-user (i.e. you are running a model from a vendor on a local accelerator for local processing instead of remote processing). Development happens on remote farms and on-line usage also happens on remote farms.
- hospitalJail 3y ago
  I need to do everything locally. Air gapped computer with confidential stuff.
  - oneplane 3y ago
    I that case your situation is likely irrelevant for the masses. It also still matters what performance bracket you're targeting.
    - hospitalJail 3y ago
      Maybe. I can picture wanting to do nsfw stuff locally... But also this is a fortune 100 company that has the airgapped computer with the confidential stuff. Its big bucks.
imtringued 3y ago
Try encoding video in real time or NP hard optimization problems (e.g. generating bitstreams for FPGAs) instead of running something as trivial as a database and a Webserver.
LoganDark 3y ago
I'm sitting here with "only" 12GB of VRAM and I can't train even the bigger RWKV models, and that's a freaking RNN, it's not even a transformer! And that's highly disappointing for someone who's been working as a contributor to one of the inference libraries, because the fruits of my labor aren't even going to benefit me that much. Bleh~ I (somewhat) lie, all my experience working on rwkv.cpp does gain me knowledge and also contribute to the community. I'm just sort of sad that I don't have enough VRAM to finetune a bigger model to fit something like a chat format, or learn a certain vocabulary or whatever.
rsynnott 3y ago
After some years of a 15" 2016 MacBook Pro (perhaps the worst laptop Apple ever made) at work, the M1 Max that replaced it was a revelation. I have no use for a fast GPU (in fact one of my pet peeves about the 2016 MBP was that they eliminated the no-discrete-GPU option that prior 15"s had, resulting in far worse thermals for no gains for many users). Your milage may, of course, vary, but some of us appreciate a fast CPU and have no use for a fast GPU (the M1 Max is, I gather, tolerably fast in this department, but the poor thing never gets to do anything other than displaying GUIs in my case...) RE ML stuff, do people actually do the heavy lifting for that _on laptops_ much?
- hospitalJail 3y ago
  >the poor thing never gets to do anything other than displaying GUIs in my case... This is kind of my point. What actual CPU stuff is done in 2023? Its why Apple's marketing focuses so much on battery life. For a year, they won a competition that no one had cared to win.
  - rsynnott 3y ago
    Sorry, I mean that the GPU never does anything much. The CPU is fairly busy.
  - c-cube 3y ago
    CPU stuff, let's see... For a dev, maybe running their own program, compilers (sometimes with a lot of optimization passes), interpreters for python/ruby, an IDE with indexing over the whole project, running the OS itself, tests, browser tabs... None of that runs on a GPU! (except part of the browser rendering). CPU is fairly important.

oneplane 3y ago

I suppose it depends on what performance bracket you are targeting. Scripting and interpreted languages, JIT next and VM after that (i.e. JVM, CLR), native code after that and optimised low-level code after that would all have very different reactions to 'good CPU' vs. 'bad CPU'. A 'slow' process (±10s) taking 1 second longer won't seem that much slower, but a fast process (±100ms) taking ten times as long does definitely become problematic. As for AI, it's mostly irrelevant what your local machines does, unless you are an end-user (i.e. you are running a model from a vendor on a local accelerator for local processing instead of remote processing). Development happens on remote farms and on-line usage also happens on remote farms.

hospitalJail 3y ago
I need to do everything locally. Air gapped computer with confidential stuff.
- oneplane 3y ago
  I that case your situation is likely irrelevant for the masses. It also still matters what performance bracket you're targeting.
  - hospitalJail 3y ago
    Maybe. I can picture wanting to do nsfw stuff locally... But also this is a fortune 100 company that has the airgapped computer with the confidential stuff. Its big bucks.

imtringued 3y ago

Try encoding video in real time or NP hard optimization problems (e.g. generating bitstreams for FPGAs) instead of running something as trivial as a database and a Webserver.

LoganDark 3y ago

I'm sitting here with "only" 12GB of VRAM and I can't train even the bigger RWKV models, and that's a freaking RNN, it's not even a transformer! And that's highly disappointing for someone who's been working as a contributor to one of the inference libraries, because the fruits of my labor aren't even going to benefit me that much. Bleh~ I (somewhat) lie, all my experience working on rwkv.cpp does gain me knowledge and also contribute to the community. I'm just sort of sad that I don't have enough VRAM to finetune a bigger model to fit something like a chat format, or learn a certain vocabulary or whatever.

rsynnott 3y ago

After some years of a 15" 2016 MacBook Pro (perhaps the worst laptop Apple ever made) at work, the M1 Max that replaced it was a revelation. I have no use for a fast GPU (in fact one of my pet peeves about the 2016 MBP was that they eliminated the no-discrete-GPU option that prior 15"s had, resulting in far worse thermals for no gains for many users). Your milage may, of course, vary, but some of us appreciate a fast CPU and have no use for a fast GPU (the M1 Max is, I gather, tolerably fast in this department, but the poor thing never gets to do anything other than displaying GUIs in my case...) RE ML stuff, do people actually do the heavy lifting for that _on laptops_ much?

hospitalJail 3y ago
>the poor thing never gets to do anything other than displaying GUIs in my case... This is kind of my point. What actual CPU stuff is done in 2023? Its why Apple's marketing focuses so much on battery life. For a year, they won a competition that no one had cared to win.
- rsynnott 3y ago
  Sorry, I mean that the GPU never does anything much. The CPU is fairly busy.
- c-cube 3y ago
  CPU stuff, let's see... For a dev, maybe running their own program, compilers (sometimes with a lot of optimization passes), interpreters for python/ruby, an IDE with indexing over the whole project, running the OS itself, tests, browser tabs... None of that runs on a GPU! (except part of the browser rendering). CPU is fairly important.