12 ms·

Meltdown and Spectre Linux Kernel Status

217 points by dankohn1 9y ago

jo909 9y ago
You can find a "shared by a subscriber" non-paywall link of the lwn.net article mentioned in the article here: https://news.ycombinator.com/item?id=16177525 https://news.ycombinator.com/item?id=16177525
- eddyg 9y ago
  LWN is worth supporting.
  - jo909 9y ago
    Sure it is. They allow the link to be shared nevertheless. https://lwn.net/op/FAQ.lwn#slinks https://lwn.net/op/FAQ.lwn#slinks Where is it appropriate to post a subscriber link? Almost anywhere. Private mail, messages to project mailing lists, and blog entries are all appropriate. As long as people do not use subscriber links as a way to defeat our attempts to gain subscribers, we are happy to see them shared.
    - LeifCarrotson 9y ago
      And as this is a blog entry, I think it's a place where the author should have used the subscriber link. > As always, [lwn.net covers the technical details](https://lwn.net/Articles/744287/ https://lwn.net/Articles/744287/) about the latest state of the kernel patches to resolve the Spectre issues, so please go read that to find out that type of information. The non-subscriber link in the blog entry results in a paywall. I think LWN would prefer that Greg used a subscriber link in this case. You may be being downvoted because people think you're trying to post a paywall workaround, but I agree that this would have been an appropriate place to use a subscriber link.
      - tytso 9y ago
        You can use a subscriber link, but Greg chose not to. Jon Corbet is a mutual friend, and LWN is essentially a two person shop (Jon and Jake do all of the work, and they are not getting rich off of what is essentially a labor of love). So if someone decides they want to use a somewhat more proactive way of encouraging people to subscribe to LWN, personally I think it's appropriate and its his choice. Just as someone who wants to post a subscriber link to Hacker News is also appropriate. (But you should seriously think about subscribing, even at the Starving Hacker level, if at all possible. I subscribe at the highest tier even though my company has a corporate subscription, Just Because.)
  - dankohn1 9y ago
    I've spoken to Jon Corbet, publisher of LWN, about sharing subscriber links on HN. He's fine with it. Here's the link Greg KH references: https://lwn.net/SubscriberLink/744287/ec63c9aa1fe236c7/ https://lwn.net/SubscriberLink/744287/ec63c9aa1fe236c7/ Please get your company to subscribe to LWN. It's an amazing resource and needs your support.
lima 9y ago
RHEL/CentOS state: Red Hat built their own mitigation for Meltdown and Spectre and were the first distribution to have an updated kernel right when the embargo was lifted. As far as I can tell, their meltdown mitigations are similar to what's in the upstream kernel now, and the Spectre mitigations relies on Intel's new MSRs (the microcode update that had since been retracted - new microcode_ctl packages remove it). Interestingly, you can enable/disable the mitigations at runtime with Red Hat, which is useful for performance testing. I found no way to do this with the upstream kernel. I'm not happy that Intel, Google, Red Hat and a few others have known about the bug for months and the upstream kernel devs are scrambling to build mitigations and still don't support the microcode mitigations. Google apparently patched their stuff in September, while everyone else is left out in the cold. This is a real mess. Surely there must be a better way than telling a few select companies? The Red Hat guys in particular surely would've loved to collaborate with the upstream kernel. Must be frustrating not to be able to tell anyone.
- pqh 9y ago
  They really could have done a tiered release. Keep the knowledge of the vulnerability as close as possible for the ~6 months they had it, then release it to a greater (but still restricted) set of people for another ~6 months. the worst case would have been someone in the latter set leaking the info, which is just what we have now. So it could have been potentially better. There was really no reason to drop a bomb like this.
  - rplnt 9y ago
    It didn't survive even the first 6 months and was published shortly before the embargo was over.
    - Aissen 9y ago
      Nitpick: it actually survived 7 months. From June 1st to January ~1st
- tener 9y ago
  I'm afraid this is the kind of problem where there is no easy solution. Is there any specific improvement you would like to propose?
Santosh83 9y ago
Am I correct in stating that the microcode update recently released by Intel was buggy and therefore pulled back by Linux distros and Intel themselves, and also a fixed version of the microcode update has not yet been released by Intel for Linux as well as other OS like Windows?
- j_s 9y ago
  I am curious to know more details on the status of the microcode update specifically when it comes to Windows Update.
  - newman314 9y ago
    I do not believe the Windows patches include microcode at this time.
drofmij 9y ago
This command does not show anything on my ubuntu 16.04 lts machine. grep . /sys/devices/system/cpu/vulnerabilities/* is there an alternate command for ubuntu?
- drofmij 9y ago
  for ubuntu command check here: https://askubuntu.com/questions/992137/how-to-check-that-kpti-is-enabled-on-my-ubuntu https://askubuntu.com/questions/992137/how-to-check-that-kpt... Here is one of the commands recommended in that thread: grep -q "cpu_insecure\|cpu_meltdown\|kaiser" /proc/cpuinfo && echo "patched :)" \ || echo "unpatched :("
  - yorby 9y ago
    This command appears to work in Fedora too...
  - the_af 9y ago
    That line worked for me, but do note the first command listed in the answer you linked to is NOT reliable and is not always consistent with the next two commands.
- deleted 9y ago
  [deleted]
- mnw21cam 9y ago
  Having actually read the article, this would suggest that your kernel is too old to have the fixes.
  - twunde 9y ago
    Not true. Ubuntu packported the kernel patches. 16.04 should be updated automatically as long as you've restarted your server
    - newman314 9y ago
caio1982 9y ago
What a mess this has been so far... I appreciate all the effort kernel folks had put in it until now but the whole tier 1 companies cabal thing is just disgusting and it made the problem worse with the early disclosure (not to say "leak" if you count LKML diffs flying around). If at least all the cabal parties were equally covered and protected by now, but nope nope nope. If I were a kernel developer not working for any cabal party I would be very pissed off.
- DoofusOfDeath 9y ago
  I recall Herb Sutter saying that companies collaborating on the C++ standards had to do it via an organization like the ISO to avoid the risk of antitrust(?) charges. I wonder if the "cabal" you mentioned actually broke those laws?
  - Skunkleton 9y ago
    I know it sucks, but I think the intentions were good here. By managing the disclosure in this way, the majority of users _should_ have been protected before public disclosure. Of course that didn't happen, but that was the intent IMO.
- TorKlingberg 9y ago
  How would you do it then? Publish as soon as it's discovered, and let everyone scramble for fixes as its being exploited?
  - caio1982 9y ago
    0. fucking put the pressure on Intel and other chip makers to have microcode updates ready by NYE, like, you know, 6 months after the first report 1. make sure all tier 1 companies involved are equally patched and on the same page as far as mitigations go before EOY, then involve tier 2 companies (see news on "the impromptu war room") a full month before the embargo deadline 2. not commit or patch anything or fly commented diffs around in the freaking public LKML; git branches to be tested and secondary mailing lists can be private if only for a brief period of time That's definitely not what happened, and probably the dirty details have not surfaced yet. I can't wait for this to be reasonably over so we can start to read post mortem on the whole process.
eigengrau 9y ago
I’m confused… The article led me to recheck whether I’m loading the newest Intel ucode update during early boot, and it turns out I am, but the latest ucode revision for my Ivybridge machine is from 2015. Didn’t Intel push microcode updates for all affected systems yet, in preparation for more spectre-related mitigations?
- remael 9y ago
  Lenovo mentions [1] a "target availability" of a bios update for my x230 with ivybridge for 2nd of February. Intel didn't update the microcode for those chips yet. [1] https://datacentersupport.lenovo.com/de/de/products/storage/lenovo-storage/v3700v2/product_security/ps500151 https://datacentersupport.lenovo.com/de/de/products/storage/...
  - deleted 9y ago
    [deleted]
- SolarNet 9y ago
  Intel is behind on the patches... 6 months is not enough time apparently.
blauditore 9y ago
So, apparently my system is unpatched. What is the proper thing to do now? Should I immediately update my BIOS (would this also patch my CPU's firmware?) or just wait for mitigations arriving with OS updates?
- cesarb 9y ago
  > Should I immediately update my BIOS (would this also patch my CPU's firmware?) The current recommendation is "no", since some of the microcode updates are known to cause stability problems. It's better to wait for Intel to fix that first, then make sure your BIOS update has a microcode version without the stability problems, then update. Yes, it's a mess. > or just wait for mitigations arriving with OS updates? Even after you update the microcode, you still need a OS update to enable the microcode mitigations, so you have to wait for a OS update anyway.
  - danieldk 9y ago
    The current recommendation is "no", since some of the microcode updates are known to cause stability problems. It's better to wait for Intel to fix that first, then make sure your BIOS update has a microcode version without the stability problems, then update. Yes, it's a mess. Note that the microcode update does not have to be delivered through a BIOS update. Linux can also update the microcode during early boot: https://wiki.archlinux.org/index.php/microcode https://wiki.archlinux.org/index.php/microcode
tomxor 9y ago
If you also want to know if intel has released a microcode update for your CPU yet (regardless of whether or not it's available in your distro yet): Download the latest archive from Intel at https://downloadcenter.intel.com/download/27431/Linux-Processor-Microcode-Data-File https://downloadcenter.intel.com/download/27431/Linux-Proces... (currently microcode-20180108.tgz) then: tar -xzf microcode-20180108.tgz sudo apt install iucode-tool # or whatever your package manager is iucode-tool -tb -lS ./intel-ucode/* This will crosscheck the microcode binaries with the signature of your CPU and output something like: 001: sig 0x00010676, pf mask 0x01, 2010-09-29, rev 0x060f, size 4096 002: sig 0x00010676, pf mask 0x04, 2010-09-29, rev 0x060f, size 4096 003: sig 0x00010676, pf mask 0x10, 2010-09-29, rev 0x060f, size 4096 004: sig 0x00010676, pf mask 0x40, 2010-09-29, rev 0x060f, size 4096 005: sig 0x00010676, pf mask 0x80, 2010-09-29, rev 0x060f, size 4096 As you can see Intel hasn't released any updates for my old CPU (latest is from 2010)
- mahrain 9y ago
  So this microcode goes in /lib/firmware/intel-ucode (in ubuntu) will this then firmware update the CPU with the latest microcode, or only use this like a driver leaving the CPU alone?
  - tomxor 9y ago
    Microcode is loaded into some on-die volatile RAM every boot by the CPU... there is a CPU instruction to do it. AFAIK: First the CPU loads an on die ROM of the microcode it was released with on it's own. Second the BIOS/UEFI may load an update into the CPU before boot if it has a newer one. Third linux and some other OS will load any newer microcode into the CPU at the bootloader stage before the kernel loads. This is also commonly the case for other modern hardware, not just CPUs... although I expect it doesn't need to happen before the kernel loads and is probably bundled into their respective drivers.
  - deleted 9y ago
    [deleted]
- AsyncAwait 9y ago
eis 9y ago
> And yes, it is behind a paywall for a few more weeks, you should be buying a subscription to get this type of thing, go do that! LWN has tons of great content and deserves any support they can get. But I strongly suggest that in this case, there shouldn't be a paywall. The current state of Meltdown/Spectre fixes is of high public interest. And people who are normally not into kernel development should be able to read this information without having to pay (or find workarounds).

217 points by dankohn1 9y ago

You can find a "shared by a subscriber" non-paywall link of the lwn.net article mentioned in the article here: https://news.ycombinator.com/item?id=16177525 https://news.ycombinator.com/item?id=16177525

eddyg 9y ago
LWN is worth supporting.
- jo909 9y ago
  Sure it is. They allow the link to be shared nevertheless. https://lwn.net/op/FAQ.lwn#slinks https://lwn.net/op/FAQ.lwn#slinks Where is it appropriate to post a subscriber link? Almost anywhere. Private mail, messages to project mailing lists, and blog entries are all appropriate. As long as people do not use subscriber links as a way to defeat our attempts to gain subscribers, we are happy to see them shared.
  - LeifCarrotson 9y ago
    And as this is a blog entry, I think it's a place where the author should have used the subscriber link. > As always, [lwn.net covers the technical details](https://lwn.net/Articles/744287/ https://lwn.net/Articles/744287/) about the latest state of the kernel patches to resolve the Spectre issues, so please go read that to find out that type of information. The non-subscriber link in the blog entry results in a paywall. I think LWN would prefer that Greg used a subscriber link in this case. You may be being downvoted because people think you're trying to post a paywall workaround, but I agree that this would have been an appropriate place to use a subscriber link.
    - tytso 9y ago
      You can use a subscriber link, but Greg chose not to. Jon Corbet is a mutual friend, and LWN is essentially a two person shop (Jon and Jake do all of the work, and they are not getting rich off of what is essentially a labor of love). So if someone decides they want to use a somewhat more proactive way of encouraging people to subscribe to LWN, personally I think it's appropriate and its his choice. Just as someone who wants to post a subscriber link to Hacker News is also appropriate. (But you should seriously think about subscribing, even at the Starving Hacker level, if at all possible. I subscribe at the highest tier even though my company has a corporate subscription, Just Because.)
- dankohn1 9y ago
  I've spoken to Jon Corbet, publisher of LWN, about sharing subscriber links on HN. He's fine with it. Here's the link Greg KH references: https://lwn.net/SubscriberLink/744287/ec63c9aa1fe236c7/ https://lwn.net/SubscriberLink/744287/ec63c9aa1fe236c7/ Please get your company to subscribe to LWN. It's an amazing resource and needs your support.

lima 9y ago

RHEL/CentOS state: Red Hat built their own mitigation for Meltdown and Spectre and were the first distribution to have an updated kernel right when the embargo was lifted. As far as I can tell, their meltdown mitigations are similar to what's in the upstream kernel now, and the Spectre mitigations relies on Intel's new MSRs (the microcode update that had since been retracted - new microcode_ctl packages remove it). Interestingly, you can enable/disable the mitigations at runtime with Red Hat, which is useful for performance testing. I found no way to do this with the upstream kernel. I'm not happy that Intel, Google, Red Hat and a few others have known about the bug for months and the upstream kernel devs are scrambling to build mitigations and still don't support the microcode mitigations. Google apparently patched their stuff in September, while everyone else is left out in the cold. This is a real mess. Surely there must be a better way than telling a few select companies? The Red Hat guys in particular surely would've loved to collaborate with the upstream kernel. Must be frustrating not to be able to tell anyone.

pqh 9y ago
They really could have done a tiered release. Keep the knowledge of the vulnerability as close as possible for the ~6 months they had it, then release it to a greater (but still restricted) set of people for another ~6 months. the worst case would have been someone in the latter set leaking the info, which is just what we have now. So it could have been potentially better. There was really no reason to drop a bomb like this.
- rplnt 9y ago
  It didn't survive even the first 6 months and was published shortly before the embargo was over.
  - Aissen 9y ago
    Nitpick: it actually survived 7 months. From June 1st to January ~1st
tener 9y ago
I'm afraid this is the kind of problem where there is no easy solution. Is there any specific improvement you would like to propose?

Santosh83 9y ago

Am I correct in stating that the microcode update recently released by Intel was buggy and therefore pulled back by Linux distros and Intel themselves, and also a fixed version of the microcode update has not yet been released by Intel for Linux as well as other OS like Windows?

j_s 9y ago
I am curious to know more details on the status of the microcode update specifically when it comes to Windows Update.
- newman314 9y ago
  I do not believe the Windows patches include microcode at this time.

drofmij 9y ago

This command does not show anything on my ubuntu 16.04 lts machine. grep . /sys/devices/system/cpu/vulnerabilities/* is there an alternate command for ubuntu?

drofmij 9y ago
for ubuntu command check here: https://askubuntu.com/questions/992137/how-to-check-that-kpti-is-enabled-on-my-ubuntu https://askubuntu.com/questions/992137/how-to-check-that-kpt... Here is one of the commands recommended in that thread: grep -q "cpu_insecure\|cpu_meltdown\|kaiser" /proc/cpuinfo && echo "patched :)" \ || echo "unpatched :("
- yorby 9y ago
  This command appears to work in Fedora too...
- the_af 9y ago
  That line worked for me, but do note the first command listed in the answer you linked to is NOT reliable and is not always consistent with the next two commands.
deleted 9y ago
[deleted]
mnw21cam 9y ago
Having actually read the article, this would suggest that your kernel is too old to have the fixes.
- twunde 9y ago
  Not true. Ubuntu packported the kernel patches. 16.04 should be updated automatically as long as you've restarted your server
  - newman314 9y ago

caio1982 9y ago

What a mess this has been so far... I appreciate all the effort kernel folks had put in it until now but the whole tier 1 companies cabal thing is just disgusting and it made the problem worse with the early disclosure (not to say "leak" if you count LKML diffs flying around). If at least all the cabal parties were equally covered and protected by now, but nope nope nope. If I were a kernel developer not working for any cabal party I would be very pissed off.

DoofusOfDeath 9y ago
I recall Herb Sutter saying that companies collaborating on the C++ standards had to do it via an organization like the ISO to avoid the risk of antitrust(?) charges. I wonder if the "cabal" you mentioned actually broke those laws?
- Skunkleton 9y ago
  I know it sucks, but I think the intentions were good here. By managing the disclosure in this way, the majority of users _should_ have been protected before public disclosure. Of course that didn't happen, but that was the intent IMO.
TorKlingberg 9y ago
How would you do it then? Publish as soon as it's discovered, and let everyone scramble for fixes as its being exploited?
- caio1982 9y ago
  0. fucking put the pressure on Intel and other chip makers to have microcode updates ready by NYE, like, you know, 6 months after the first report 1. make sure all tier 1 companies involved are equally patched and on the same page as far as mitigations go before EOY, then involve tier 2 companies (see news on "the impromptu war room") a full month before the embargo deadline 2. not commit or patch anything or fly commented diffs around in the freaking public LKML; git branches to be tested and secondary mailing lists can be private if only for a brief period of time That's definitely not what happened, and probably the dirty details have not surfaced yet. I can't wait for this to be reasonably over so we can start to read post mortem on the whole process.

eigengrau 9y ago

I’m confused… The article led me to recheck whether I’m loading the newest Intel ucode update during early boot, and it turns out I am, but the latest ucode revision for my Ivybridge machine is from 2015. Didn’t Intel push microcode updates for all affected systems yet, in preparation for more spectre-related mitigations?

remael 9y ago
Lenovo mentions [1] a "target availability" of a bios update for my x230 with ivybridge for 2nd of February. Intel didn't update the microcode for those chips yet. [1] https://datacentersupport.lenovo.com/de/de/products/storage/lenovo-storage/v3700v2/product_security/ps500151 https://datacentersupport.lenovo.com/de/de/products/storage/...
- deleted 9y ago
  [deleted]
SolarNet 9y ago
Intel is behind on the patches... 6 months is not enough time apparently.

blauditore 9y ago

So, apparently my system is unpatched. What is the proper thing to do now? Should I immediately update my BIOS (would this also patch my CPU's firmware?) or just wait for mitigations arriving with OS updates?

cesarb 9y ago
> Should I immediately update my BIOS (would this also patch my CPU's firmware?) The current recommendation is "no", since some of the microcode updates are known to cause stability problems. It's better to wait for Intel to fix that first, then make sure your BIOS update has a microcode version without the stability problems, then update. Yes, it's a mess. > or just wait for mitigations arriving with OS updates? Even after you update the microcode, you still need a OS update to enable the microcode mitigations, so you have to wait for a OS update anyway.
- danieldk 9y ago
  The current recommendation is "no", since some of the microcode updates are known to cause stability problems. It's better to wait for Intel to fix that first, then make sure your BIOS update has a microcode version without the stability problems, then update. Yes, it's a mess. Note that the microcode update does not have to be delivered through a BIOS update. Linux can also update the microcode during early boot: https://wiki.archlinux.org/index.php/microcode https://wiki.archlinux.org/index.php/microcode

tomxor 9y ago

If you also want to know if intel has released a microcode update for your CPU yet (regardless of whether or not it's available in your distro yet): Download the latest archive from Intel at https://downloadcenter.intel.com/download/27431/Linux-Processor-Microcode-Data-File https://downloadcenter.intel.com/download/27431/Linux-Proces... (currently microcode-20180108.tgz) then: tar -xzf microcode-20180108.tgz sudo apt install iucode-tool # or whatever your package manager is iucode-tool -tb -lS ./intel-ucode/* This will crosscheck the microcode binaries with the signature of your CPU and output something like: 001: sig 0x00010676, pf mask 0x01, 2010-09-29, rev 0x060f, size 4096 002: sig 0x00010676, pf mask 0x04, 2010-09-29, rev 0x060f, size 4096 003: sig 0x00010676, pf mask 0x10, 2010-09-29, rev 0x060f, size 4096 004: sig 0x00010676, pf mask 0x40, 2010-09-29, rev 0x060f, size 4096 005: sig 0x00010676, pf mask 0x80, 2010-09-29, rev 0x060f, size 4096 As you can see Intel hasn't released any updates for my old CPU (latest is from 2010)

mahrain 9y ago
So this microcode goes in /lib/firmware/intel-ucode (in ubuntu) will this then firmware update the CPU with the latest microcode, or only use this like a driver leaving the CPU alone?
- tomxor 9y ago
  Microcode is loaded into some on-die volatile RAM every boot by the CPU... there is a CPU instruction to do it. AFAIK: First the CPU loads an on die ROM of the microcode it was released with on it's own. Second the BIOS/UEFI may load an update into the CPU before boot if it has a newer one. Third linux and some other OS will load any newer microcode into the CPU at the bootloader stage before the kernel loads. This is also commonly the case for other modern hardware, not just CPUs... although I expect it doesn't need to happen before the kernel loads and is probably bundled into their respective drivers.
- deleted 9y ago
  [deleted]
AsyncAwait 9y ago

eis 9y ago

> And yes, it is behind a paywall for a few more weeks, you should be buying a subscription to get this type of thing, go do that! LWN has tons of great content and deserves any support they can get. But I strongly suggest that in this case, there shouldn't be a paywall. The current state of Meltdown/Spectre fixes is of high public interest. And people who are normally not into kernel development should be able to read this information without having to pay (or find workarounds).