11 ms·

The security implications of this as commented on the article seems pretty bad. Wonder if any microcode fix can work or it would need a silicon level fix. The s

by tmd83 10y ago

The security implications of this as commented on the article seems pretty bad. Wonder if any microcode fix can work or it would need a silicon level fix. The second I presume would be very bad? -- Keeping my earlier comment but it seems there's a microcode fix in testing currently. http://forum.hwbot.org/showpost.php?p=480922&postcount=30 http://forum.hwbot.org/showpost.php?p=480922&postcount=30

zelon88 10y ago
It's happened to AMD before, and it was pretty bad for their brand. http://www.anandtech.com/show/2477/2 http://www.anandtech.com/show/2477/2
- raverbashing 10y ago
  Intel had serious problems as well, let's not forget: the Pentium fp bug and the F00F bug (though those are more than a decade old at this point)
  - msbarnett 10y ago
    Haswell and early Broadwell had a bug in its transactional memory extensions so serious that the microcode "fix" simply permanently disabled the instructions. CPU errata like this aren't uncommon.
    - m45t3r 10y ago
      While those are the most remembered bugs, almost every CPU have some kinda of CPU errata that can lock up the system in really weird and specific ways. For example, my Intel Ivy Bridge CPU running Linux: $ dmesg | grep -i microcode [ 0.000000] microcode: microcode updated early to revision 0x1c, date = 2015-02-26 [ 1.415548] microcode: sig=0x306a9, pf=0x10, revision=0x1c [ 1.415665] microcode: Microcode Update Driver: v2.2. The fact that most users does not hit those bugs is because modern OS already patches those microcode before execution (like the case above). Of course, if AMD can't fixes those bugs without performance regressions (remember the infamous TLB Bug from earlier Phenoms?) it can be pretty bad. However I don't think the majority of users needs to be too cautious about it.
      - yuhong 10y ago
        Some of which are quite serious: http://yuhongbao.blogspot.ca/2015/06/why-your-core-2-processor-appear-to-not.html http://yuhongbao.blogspot.ca/2015/06/why-your-core-2-process...
  - deleted 10y ago
    [deleted]
  - rincebrain 10y ago
    Don't forget the (other) f00f bug in the Intel Quark![1] [1] - https://en.wikipedia.org/wiki/Intel_Quark#Segfault_bug https://en.wikipedia.org/wiki/Intel_Quark#Segfault_bug
    - makomk 10y ago
      Yeah, that was kind of unfortunate as it made the Intel Quark incompatible with all normal Linux distros. I don't think it even had any kind of microcode update facility, being effectively a slightly updated 486.
      - rincebrain 10y ago
        Even with a runtime microcode update facility, no BIOS update equivalent would still make it pretty fatal for packaging except as an entirely distinct build target, because otherwise you'd need to do something kinky like always first booting a custom kernel+initrd target built to involve no locks, load the microcode update, then kexec into the real kernel.
- neogodless 10y ago
  I don't know if this bug affects Phenom II chips, but I've been running a Phenom II for 8 straight years in my primary desktop with nary a hiccup. Rock solid reliable. Guess I haven't been running the "correct conditions" for a crash?
  - rincebrain 10y ago
    You could also have a BIOS update with the workaround already baked in, or a chip that has the later silicon without the bug... (I also had a Phenom II for many years without any mysterious crashing problems, even though the chip was the correct revision and there was no BIOS erratum workaround enabled.)
    - informedperson 10y ago
      Windows 7 and later distribute and activate microcode updates through Windows Update.
      - rincebrain 10y ago
        It seems like it's not a microcode update fix, but an MSR killbit flip from the BIOS? [1] [1] - http://support.amd.com/TechDocs/41322_10h_Rev_Gd.pdf http://support.amd.com/TechDocs/41322_10h_Rev_Gd.pdf page 66
        yuhong 10y ago
        Yes, which reduces performance if I remember correctly. It was quite famous. They eventually released a fixed stepping, and Phenom II is not affected.
        informedperson 10y ago
        True, my mistake. (While it doesn't apply for this specific example, it's still the reason why one generally will see microcode-fixed errata go away rather quickly, independently of BIOS updates)
  - 10y ago
pierrebai 10y ago
AMD already has a fix, it requires a BIOS update for the motherboard. They have not given details, but some experts think the fix is merely an increase in the power delivered to that part of the chip. Their conclusion stems from teh fact that the FMA3 bug disappeared when the chip was over-clocked because the over-clocking required increasing the voltage to the chip. So the BIOS update probably adjust the voltages up a bit. http://www.fudzilla.com/news/processors/43166-amd-confirms-ryzen-screen-freeze-fix http://www.fudzilla.com/news/processors/43166-amd-confirms-r...
- mickronome 10y ago
  Since they are pushing how efficiently the chip are regulating power to various parts of the chip to keep power draw to a minimum it sounds like a likely cause. Assuming most of the testing budget is spent on the most common and likely instruction sequence patterns is probably not unrealistic. Still a bit odd that such a short sequence of repeats of the same instruction failed. Which got me thinking. At more than 2000 opcodes for x86-64 and legacy opcodes, testing all 3-sequences with one input is more than 8G sequences, to cover that and (guesstimated average) 64 bits of input per opcode is a staggering 10^29 number of instructions, or 2T CPU*year at 4Gops/s. Not easy to test all that, even if it would be off by a couple of magnitudes in the right direction! Might be that valid 2-sequences is the most that can hope to be even close to exhaustively tested while at the same time covering some significant fraction of the operand space ? The fact that modern CPU's work at all given their complexity in so many different areas looks pretty darn close to magic, even from a pretty close distance.
  - deleted 10y ago
    [deleted]
  - flamedoge 10y ago
    At least we try, with SMT solvers and such. Surprise surprise, math works!
- informedperson 10y ago
  A power issue would also plausibly explain why disabling SMT avoids the problem, since disabling SMT powers off a bunch of stuff (a whole thread context does need a bit of power), and, in a multi-threaded scenario, tends to reduce core usage thereby reducing average (! so likely not relevant) power draw as well.
__david__ 10y ago
If it's not fixable in microcode, then yeah, I'd say it's pretty bad. Makes it useless as a cloud machine, if anyone can cause a DOS and take out every other VM on a host. For a personal machine, it's probably not terrible if you are using linux or something else where you can compile everything yourself. But running binaries built by someone else would be a crapshoot. I wonder how many games (of the Windows, AAA variety) use these multiply instructions? There's also not a lot of detail in the article—like if the data has to be specific or just the combination of instructions is enough to crash it.
- zigzigzag 10y ago
  Cloud doesn't have to mean virtualised and shared. There are plenty of operations that run on dedicated hardware.
  - bonzini 10y ago
    For many cloud vendors, including EC2, "dedicated hardware" just means yours will be the only virtual machine on that computer. But they're still running your image under a hypervisor.
    - zigzigzag 10y ago
      Yes but if you're worried about attacks from co-tenants, that's OK (the cloud provider can always access your hardware anyway)
      - bonzini 10y ago
        It would still be a pain for cloud providers if the tenant can unwillingly DoS the hypervisor. Sure they have watchdogs in place, but it gets more complicated if the DoS can be non-malicious.
  - __david__ 10y ago
    I suspect Amazon (for instance) has far more shared machines than dedicated machines. If something isn't viable for shared machines then a huge section of the market is ripped away. Still seems catastrophic to me.
theandrewbailey 10y ago
> Flops is only affected when the SMT is enabled, so disabling the SMT can be used as a temporary work-around (until the actual fix arrives). Does this mean disabling SMT will also fix the bug, or is that specific to this app? I've heard that some things improve performance on Ryzen with SMT off, but I've heard that's because OS level task schedulers need to be optimized better. But I still wonder if Ryzen's SMT implementation is on par with Intel's first implementations.

by tmd83 10y ago

zelon88 10y ago

It's happened to AMD before, and it was pretty bad for their brand. http://www.anandtech.com/show/2477/2 http://www.anandtech.com/show/2477/2

raverbashing 10y ago
Intel had serious problems as well, let's not forget: the Pentium fp bug and the F00F bug (though those are more than a decade old at this point)
- msbarnett 10y ago
  Haswell and early Broadwell had a bug in its transactional memory extensions so serious that the microcode "fix" simply permanently disabled the instructions. CPU errata like this aren't uncommon.
  - m45t3r 10y ago
    While those are the most remembered bugs, almost every CPU have some kinda of CPU errata that can lock up the system in really weird and specific ways. For example, my Intel Ivy Bridge CPU running Linux: $ dmesg | grep -i microcode [ 0.000000] microcode: microcode updated early to revision 0x1c, date = 2015-02-26 [ 1.415548] microcode: sig=0x306a9, pf=0x10, revision=0x1c [ 1.415665] microcode: Microcode Update Driver: v2.2. The fact that most users does not hit those bugs is because modern OS already patches those microcode before execution (like the case above). Of course, if AMD can't fixes those bugs without performance regressions (remember the infamous TLB Bug from earlier Phenoms?) it can be pretty bad. However I don't think the majority of users needs to be too cautious about it.
    - yuhong 10y ago
      Some of which are quite serious: http://yuhongbao.blogspot.ca/2015/06/why-your-core-2-processor-appear-to-not.html http://yuhongbao.blogspot.ca/2015/06/why-your-core-2-process...
- deleted 10y ago
  [deleted]
- rincebrain 10y ago
  Don't forget the (other) f00f bug in the Intel Quark![1] [1] - https://en.wikipedia.org/wiki/Intel_Quark#Segfault_bug https://en.wikipedia.org/wiki/Intel_Quark#Segfault_bug
  - makomk 10y ago
    Yeah, that was kind of unfortunate as it made the Intel Quark incompatible with all normal Linux distros. I don't think it even had any kind of microcode update facility, being effectively a slightly updated 486.
    - rincebrain 10y ago
      Even with a runtime microcode update facility, no BIOS update equivalent would still make it pretty fatal for packaging except as an entirely distinct build target, because otherwise you'd need to do something kinky like always first booting a custom kernel+initrd target built to involve no locks, load the microcode update, then kexec into the real kernel.
neogodless 10y ago
I don't know if this bug affects Phenom II chips, but I've been running a Phenom II for 8 straight years in my primary desktop with nary a hiccup. Rock solid reliable. Guess I haven't been running the "correct conditions" for a crash?
- rincebrain 10y ago
  You could also have a BIOS update with the workaround already baked in, or a chip that has the later silicon without the bug... (I also had a Phenom II for many years without any mysterious crashing problems, even though the chip was the correct revision and there was no BIOS erratum workaround enabled.)
  - informedperson 10y ago
    Windows 7 and later distribute and activate microcode updates through Windows Update.
    - rincebrain 10y ago
      It seems like it's not a microcode update fix, but an MSR killbit flip from the BIOS? [1] [1] - http://support.amd.com/TechDocs/41322_10h_Rev_Gd.pdf http://support.amd.com/TechDocs/41322_10h_Rev_Gd.pdf page 66
      - yuhong 10y ago
        Yes, which reduces performance if I remember correctly. It was quite famous. They eventually released a fixed stepping, and Phenom II is not affected.
      - informedperson 10y ago
        True, my mistake. (While it doesn't apply for this specific example, it's still the reason why one generally will see microcode-fixed errata go away rather quickly, independently of BIOS updates)
- 10y ago

pierrebai 10y ago

AMD already has a fix, it requires a BIOS update for the motherboard. They have not given details, but some experts think the fix is merely an increase in the power delivered to that part of the chip. Their conclusion stems from teh fact that the FMA3 bug disappeared when the chip was over-clocked because the over-clocking required increasing the voltage to the chip. So the BIOS update probably adjust the voltages up a bit. http://www.fudzilla.com/news/processors/43166-amd-confirms-ryzen-screen-freeze-fix http://www.fudzilla.com/news/processors/43166-amd-confirms-r...

mickronome 10y ago
Since they are pushing how efficiently the chip are regulating power to various parts of the chip to keep power draw to a minimum it sounds like a likely cause. Assuming most of the testing budget is spent on the most common and likely instruction sequence patterns is probably not unrealistic. Still a bit odd that such a short sequence of repeats of the same instruction failed. Which got me thinking. At more than 2000 opcodes for x86-64 and legacy opcodes, testing all 3-sequences with one input is more than 8G sequences, to cover that and (guesstimated average) 64 bits of input per opcode is a staggering 10^29 number of instructions, or 2T CPU*year at 4Gops/s. Not easy to test all that, even if it would be off by a couple of magnitudes in the right direction! Might be that valid 2-sequences is the most that can hope to be even close to exhaustively tested while at the same time covering some significant fraction of the operand space ? The fact that modern CPU's work at all given their complexity in so many different areas looks pretty darn close to magic, even from a pretty close distance.
- deleted 10y ago
  [deleted]
- flamedoge 10y ago
  At least we try, with SMT solvers and such. Surprise surprise, math works!
informedperson 10y ago
A power issue would also plausibly explain why disabling SMT avoids the problem, since disabling SMT powers off a bunch of stuff (a whole thread context does need a bit of power), and, in a multi-threaded scenario, tends to reduce core usage thereby reducing average (! so likely not relevant) power draw as well.

__david__ 10y ago

If it's not fixable in microcode, then yeah, I'd say it's pretty bad. Makes it useless as a cloud machine, if anyone can cause a DOS and take out every other VM on a host. For a personal machine, it's probably not terrible if you are using linux or something else where you can compile everything yourself. But running binaries built by someone else would be a crapshoot. I wonder how many games (of the Windows, AAA variety) use these multiply instructions? There's also not a lot of detail in the article—like if the data has to be specific or just the combination of instructions is enough to crash it.

zigzigzag 10y ago
Cloud doesn't have to mean virtualised and shared. There are plenty of operations that run on dedicated hardware.
- bonzini 10y ago
  For many cloud vendors, including EC2, "dedicated hardware" just means yours will be the only virtual machine on that computer. But they're still running your image under a hypervisor.
  - zigzigzag 10y ago
    Yes but if you're worried about attacks from co-tenants, that's OK (the cloud provider can always access your hardware anyway)
    - bonzini 10y ago
      It would still be a pain for cloud providers if the tenant can unwillingly DoS the hypervisor. Sure they have watchdogs in place, but it gets more complicated if the DoS can be non-malicious.
- __david__ 10y ago
  I suspect Amazon (for instance) has far more shared machines than dedicated machines. If something isn't viable for shared machines then a huge section of the market is ripped away. Still seems catastrophic to me.

theandrewbailey 10y ago

> Flops is only affected when the SMT is enabled, so disabling the SMT can be used as a temporary work-around (until the actual fix arrives). Does this mean disabling SMT will also fix the bug, or is that specific to this app? I've heard that some things improve performance on Ryzen with SMT off, but I've heard that's because OS level task schedulers need to be optimized better. But I still wonder if Ryzen's SMT implementation is on par with Intel's first implementations.