5 ms·

The shocking thing about these M series comparisons is never "the M series is fast as the GIANT NVIDIA THING!" it's always "Man, the M series is 70% as fast wit

The shocking thing about these M series comparisons is never "the M series is fast as the GIANT NVIDIA THING!" it's always "Man, the M series is 70% as fast with like 1/4 the power."

kllrnohj 3y ago
It's not really that shocking. Power consumption is non-linear with respect to frequency and you see this all the time in high end CPU & GPU parts. Look at something like the eco modes on the Ryzen 7xxx for a great example. The 7950X stock pulls something like 260w on an all core workload at 5.1ghz. Yet enable the 105w eco mode and that power consumption plummets to 160w at 4.8ghz. That means the last 300mhz of performance, which is borderline inconsequential (~6% performance loss), costs 100W. The 65W option then cuts that in half almost again down to 88w (at 4ghz now), for a "mere" 20% reduction in performance. Or phrased differently, for 1/3rd the power the 7950X will give you 75% of the performance of a 7950X. Matching performance while using less power is impressive. Using less power while also being slower not so much, though.
- nottorp 3y ago
  > Or phrased differently, for 1/3rd the power the 7950X will give you 75% of the performance of a 7950X. So where is the - presumably much cheaper - 65 W 7950X?
  - dotnet00 3y ago
    Why would it be much cheaper? The chips are intentionally clocked higher than the most efficient point because the point of the CPU is raw speed, not power consumption, especially since 7950x is a desktop chip. His point is that M-CPUs being somewhat competitive but much more efficient is not as stunning since you're comparing a CPU tuned to be at the most efficient speed to a CPU tuned to be at the highest speed. Similarly a 4090's power consumption drops dramatically if you underclock or undervolt even slightly, but what's the point? You're almost definitely buying a 4090 for its raw speed.
    - nottorp 3y ago
      Because they would be able to sell some rejects. And because I don't want a software limit that may or may not work.
      - wtallis 3y ago
        AMD's chiplet-based design means they have plenty of other ways to make good use of parts that cannot hit the highest clock speeds. They have very little reason to do a 16-core low-clock part for their consumer desktop platform. And your concerns about "a software limit that may or may not work" are completely at odds with how their power management works.
      - dotnet00 3y ago
        It isn't a "software limit" beyond just being controlled by writing to certain CPU registers via software. It's very much a feature of the hardware, the same feature that allows for overclocking the chips.
      - kllrnohj 3y ago
        That's not how binning works. Quality silicon is also the ones that are more efficient. A flagship 65W part would be just as expensive as a result, it's the same-ish quality of parts.
  - coder543 3y ago
    Every 7950X offers a 65W mode. It’s not a separate SKU. It’s a choice each user can make if they care more about efficiency. Tasks take longer to complete, but the total energy consumed for the completion of the task is dramatically less.
    - nottorp 3y ago
      You think? I think it's a choice that very few technical users even know about. And of those, 90% don't care about efficiency. The 250 W space heater vacuum cleaner soundtrack mode should be opt in rather than opt out. Same for video cards.
      - coder543 3y ago
        Of course most users don’t pick the 65W option. They want maximum performance, and the cost of electricity is largely negligible to most people buying a 7950X. AMD isn’t going to offer a huge discount for a 65W 7950X for the reasons discussed elsewhere: they don’t need to.
        nottorp 3y ago
        > and the cost of electricity is largely negligible to most people buying a 7950X. “I can afford it” is still wasteful. Most people wouldn’t even notice the speed difference in 65 W mode. Even if they can afford the space heater mode.
lern_too_spel 3y ago
As others have pointed out, it is nowhere near even 1/4 as fast.
- etchalon 3y ago
  > The result for a 10 Minute audio is 0:03:36.296329 (216 seconds). Compare that to 0:03:06.707770 (186 seconds) on my Nvidia 4090. The 2000 € GPU is still 30 seconds or ~ 16% faster. All graphics core where fully utilized during the run and I quit all programs, disabled desktop picture or similar for that run.
  - lern_too_spel 3y ago
    What other people have mentioned is that there are multiple implementations for Nvidia GPUs that are many times faster than whisper.cpp.
    - etchalon 3y ago
      And my comment was about M1 comparison articles, of which this was one. And which exhibited the property I mentioned.
      - lern_too_spel 3y ago
        It does this poorly. They compared the currently most optimized whisper implementation for M1 against an implementation that is far from the best currently available for the Nvidia GPU. They cannot make a claim of reaching 70% speed at 25% power usage. Edit: The article has been updated to compare against a faster implementation for the Nvidia GPU and no longer makes that claim.

The shocking thing about these M series comparisons is never "the M series is fast as the GIANT NVIDIA THING!" it's always "Man, the M series is 70% as fast with like 1/4 the power."

kllrnohj 3y ago

It's not really that shocking. Power consumption is non-linear with respect to frequency and you see this all the time in high end CPU & GPU parts. Look at something like the eco modes on the Ryzen 7xxx for a great example. The 7950X stock pulls something like 260w on an all core workload at 5.1ghz. Yet enable the 105w eco mode and that power consumption plummets to 160w at 4.8ghz. That means the last 300mhz of performance, which is borderline inconsequential (~6% performance loss), costs 100W. The 65W option then cuts that in half almost again down to 88w (at 4ghz now), for a "mere" 20% reduction in performance. Or phrased differently, for 1/3rd the power the 7950X will give you 75% of the performance of a 7950X. Matching performance while using less power is impressive. Using less power while also being slower not so much, though.

nottorp 3y ago
> Or phrased differently, for 1/3rd the power the 7950X will give you 75% of the performance of a 7950X. So where is the - presumably much cheaper - 65 W 7950X?
- dotnet00 3y ago
  Why would it be much cheaper? The chips are intentionally clocked higher than the most efficient point because the point of the CPU is raw speed, not power consumption, especially since 7950x is a desktop chip. His point is that M-CPUs being somewhat competitive but much more efficient is not as stunning since you're comparing a CPU tuned to be at the most efficient speed to a CPU tuned to be at the highest speed. Similarly a 4090's power consumption drops dramatically if you underclock or undervolt even slightly, but what's the point? You're almost definitely buying a 4090 for its raw speed.
  - nottorp 3y ago
    Because they would be able to sell some rejects. And because I don't want a software limit that may or may not work.
    - wtallis 3y ago
      AMD's chiplet-based design means they have plenty of other ways to make good use of parts that cannot hit the highest clock speeds. They have very little reason to do a 16-core low-clock part for their consumer desktop platform. And your concerns about "a software limit that may or may not work" are completely at odds with how their power management works.
    - dotnet00 3y ago
      It isn't a "software limit" beyond just being controlled by writing to certain CPU registers via software. It's very much a feature of the hardware, the same feature that allows for overclocking the chips.
    - kllrnohj 3y ago
      That's not how binning works. Quality silicon is also the ones that are more efficient. A flagship 65W part would be just as expensive as a result, it's the same-ish quality of parts.
- coder543 3y ago
  Every 7950X offers a 65W mode. It’s not a separate SKU. It’s a choice each user can make if they care more about efficiency. Tasks take longer to complete, but the total energy consumed for the completion of the task is dramatically less.
  - nottorp 3y ago
    You think? I think it's a choice that very few technical users even know about. And of those, 90% don't care about efficiency. The 250 W space heater vacuum cleaner soundtrack mode should be opt in rather than opt out. Same for video cards.
    - coder543 3y ago
      Of course most users don’t pick the 65W option. They want maximum performance, and the cost of electricity is largely negligible to most people buying a 7950X. AMD isn’t going to offer a huge discount for a 65W 7950X for the reasons discussed elsewhere: they don’t need to.
      - nottorp 3y ago
        > and the cost of electricity is largely negligible to most people buying a 7950X. “I can afford it” is still wasteful. Most people wouldn’t even notice the speed difference in 65 W mode. Even if they can afford the space heater mode.

lern_too_spel 3y ago

As others have pointed out, it is nowhere near even 1/4 as fast.

etchalon 3y ago
> The result for a 10 Minute audio is 0:03:36.296329 (216 seconds). Compare that to 0:03:06.707770 (186 seconds) on my Nvidia 4090. The 2000 € GPU is still 30 seconds or ~ 16% faster. All graphics core where fully utilized during the run and I quit all programs, disabled desktop picture or similar for that run.
- lern_too_spel 3y ago
  What other people have mentioned is that there are multiple implementations for Nvidia GPUs that are many times faster than whisper.cpp.
  - etchalon 3y ago
    And my comment was about M1 comparison articles, of which this was one. And which exhibited the property I mentioned.
    - lern_too_spel 3y ago
      It does this poorly. They compared the currently most optimized whisper implementation for M1 against an implementation that is far from the best currently available for the Nvidia GPU. They cannot make a claim of reaching 70% speed at 25% power usage. Edit: The article has been updated to compare against a faster implementation for the Nvidia GPU and no longer makes that claim.