16 ms·
Meltdown Update Kernel doesnt boot
- ask098 9y agoit's the fault of Intel, why don't they recall all the CPU? just like vehicle company
- hodgesrm 9y agoThat might just create yet more problems. You can't just plug a new CPU into your motherboard--it has to match the socket, chipset, memory, installed OS, etc.
- StillBored 9y agoWell if they didn't change the socket every year... Realistically though, they might be able to do it for cpu's that aren't soldered on (think just about every laptop) made in the last year or two, but would they really fab new versions of 10 year old cores? Its not like many of those lines are even running anymore, so they would basically have to redesign/layout and reverify everything. Probably easier/cheaper just to send everyone a new machine.
- hodgesrm 9y agoOr give a price break on future hardware. The fix is turning out to be incredibly expensive for ordinary users, virtualization vendors, hardware vendors, OS providers, cloud providers, etc. This incident demonstrates why you really don't want catastrophic bugs in the CPU. The fact that the hardware vendors missed this one makes you wonder what else is out there.
- gruez 9y ago>Or give a price break on future hardware feels like this would happen: >intel agrees to give consumers a $30 price break in response to meltdown/spectre >in other news, intel raises the prices of next generation CPUs by $30
- Dylan16807 9y agoMake it $60 and I'll take that, all the better for them to get undercut by AMD.
- bitL 9y agoAnd how would they replace the bad part when a good one doesn't exist?
- QAPereo 9y agoWith money, and a handwritten apology soaked in the tears of their C-levels? Or just the money actually. If you can’t replace my broken item, a refund is always appreciated.
- imglorp 9y agoBryan Cantrill refrence has been noted.
- bcantrill 9y agoThat was a reference to me? If so -- I'm flattered. I haven't yet asked Intel to soak a written apology with their tears, but it's an excellent idea! (I have, however, given them many other candid thoughts on how they can improve their handling of Spectre and Meltdown -- but so far, to no avail.)
- JonRB 9y agoDoesn't this affect all of their CPUs going a long way back? And how do you recall embedded or laptop CPUs, which are often soldered in-place? A recall would be great, but there's no way they'd be able to do it. Vehicle recalls are a bit different because they impact physical safety. Digital safety doesn't get the same priority.
- QAPereo 9y agoHow about they go into bankruptcy with most of the world’s computer users as their creditors? Maybe not, but it’s terrifying that you can avoid responsibility by fucking up on a larger scale than most.
- semi-extrinsic 9y agoAlso vehicle recalls are usually done by fixing stuff next time the vehicle comes in for regularly scheduled service. How often does your computer get those?
- en4bz 9y agoA * B * C = X
- george3383 9y agoIs this a Fight Club reference?
- deleted 9y ago[deleted]
- valarauca1 9y agoThey really should. The problem is this nearly every single processor Intel shipped for a decade so Intel doesn't have the cash flow to RMA that many replacements. They're going to fight tooth and nail to avoid this.
- MBCook 9y agoNot a decade or so. Since the Pentium Pro. So since ‘95 or ‘96.
- lunorian 9y agoWait if a court forces Intel to replace processors with a safe one does that mean I get new macbook?
- StillBored 9y agoI can't imagine a court doing anything. The cost to intel is simply to high, and its not like they were negligent given that ARM/POWER/etc are all vulnerable to some extent too. This whole thing is the equivalent of discovering that if someone throws enough nails on the road your car will blow a tire, spin out of control and kill you. With the kneejerk reaction of trying to fill everyone's tires with concrete to avoid the tires blowing out rather than trying to figure out how to keep people from throwing nails on the road (with the idea that spiteful users are more likely on toll roads) in the first place.
- CogitoCogito 9y agoI agree with you that the courts probably won't do much, but you should not group all the CPU manufacturers together. Meltdown mainly (only?) affects Intel. It's spectre that affects basically everyone.
- derekp7 9y agoActually, it is more like that someone can throw a rock through a car window and then compromise the car door lock. Or, probably more accurately, that a hitchhiker can knock you out of the car, and drive off with it. Spectre/Meltdown ONLY is an issue if you run some untrusted code on your system. If you avoid this, there is not problem. Yes, we like to be able to run untrusted code (such as in a web browser / javascript), but that is not the fault of a car manufacturer that you like to pick up hitchhikers.
- willk 9y agoIs this endangering the life of anyone? A car manufacturer doesn't do a recall if personal safety isn't at risk.
- dmschulman 9y agoThey will often refuse to do a recall even if personal safety is at risk. The real quantifier is how much financial damage the auto company will endure in the event they do or do not do a recall
- cr0sh 9y ago> They will often refuse to do a recall even if personal safety is at risk. Ford Pinto, anyone... Another one that many people don't know about is a problem with old 2-door Tahoes; a bracket on the driver's side seat likes to fail upon quick acceleration - such as when getting on the freeway, for instance. One minute you're upright, pushing the pedal to get up to speed, the next - whoops! - flat on your back! If you're lucky, you live to tell the tale... AFAIK, GM never issued a recall about that one (it caused me to pass on a really nice lifted 4wd Tahoe a few years back)...
- brewdad 9y agoEh, sometimes they do for the sake of brand preservation. If personal safety is at risk they will do a recall, either voluntary or mandated by the NHTSA.
- derekp7 9y agoBut they have yet to do any recalls for cars that are susceptible to being broken into using a slim jim.
- tchaffee 9y agoVolkswagen emissions recall.
- _Nat_ 9y agoAre you suggesting that we recall the great bulk of modern CPU's? Like, literally gut everyone's computers, including those in data centers and running critical infrastructure, until replacements are eventually manufactured? Or did you mean something else?
- Consultant32452 9y agoBoxed Intel processors carry a 3 year warranty. It certainly seems reasonable for everyone who bought a CPU within the last 3 years to expect a warranty replacement with the manufacturer defect fixed. In the EU virtually every product comes with a 2 year warranty. So every CPU sold in the EU in the last two years should be replaced for free by Intel, even through OEMs. I wonder what potential class action lawsuits Intel might be facing.
- rlanday 9y agoAny sufficiently complex CPU surely contains some number of defects, perhaps even serious security defects, just as any sufficiently complex piece of software contains bugs and security holes. I wouldn't be surprised if someone tries to sue Intel over this, or even if they win, but this is way outside the scope of what a warranty would traditionally cover, which in the case of a CPU would be hardware failure. If a warranty had to cover every possible defect, a bunch of people would be constantly trying to get free CPUs out of Intel every time they updated their errata: https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/7th-gen-core-family-spec-update.pdf https://www.intel.com/content/dam/www/public/us/en/documents... Note that the cost of overly onerous regulation (e.g. requiring that every computer manufacturer replace these chips even though the problems can largely be worked around in software) is of course passed onto consumers.
- Consultant32452 9y agoOf course the cost of producing products that actually perform at the level they're advertised to perform is passed onto the consumer, regardless of regulation.
- nkkollaw 9y agoI want to be there for question 099.
- myth_drannon 9y agoFaulty vehicle it's a potential accident with multiple casualities vs a hacked computer with potential loss of your private emails or whatever.
- tchaffee 9y agoOr just a case of emissions violation?
- timonovici 9y agoWhat if that computer is running something critical, say a reactor, and elevator or some device in a hospital? Computers are everywhere these days... Todays proof-of-concept becomes tommorow's metasploit module - and could cause large damage even in incompetent people's hands.
- pmontra 9y agoAnd replace them with what? They don't have any current processor that doesn't have the same bugs. It will be years before they design and make one. The best we can do is some class action and get a refund, but not too much or they'll go broke and close.
- tchaffee 9y ago> And replace them with what? An AMD CPU?
- pmontra 9y agoWhich doesn't have Meltdown but still has Spectre. Furthermore you have to replace at least the whole motherboard on a desktop or probably all of your laptop except the discs and maybe the RAM.
- tchaffee 9y ago> still has Spectre I didn't realize that when I made the comment, and I agree my suggestion falls flat now that I know. > You have to replace at least the whole motherboard on a desktop or probably all of your laptop except the discs and maybe the RAM. I'm ok with putting that responsibility on Intel to remedy the situation, even if it deeply hurt them financially or put them out of business. If you sell a faulty product that doesn't live up to its description, yes you risk actually going out of business. But with the fact that AMD has Spectre this idea of replacement no longer makes much sense and your original idea of a partial refund makes the most sense.
- dbenhur 9y agoMost vehicle recalls are not of the form: return your vehicle, we give you a new fixed one; but rather: bring your vehicle to one of our dealers and they'll perform some action to repair the defect. The later is pretty analogous to issuing firmware and OS patches to mitigate the flaw.
- 1stranger 9y agoWhat if the fix to the car reduced your MPG by 30% to address a safety issue? This seems somewhat analogous to the number being bandied about as the CPU performance decrease. (Depends on workload, etc, etc.) I think most car owners would expect some kind of compensation for a product that no longer has the same efficiency as when they bought it.
- zoe1337 9y agoThis issue is similar to what happened with the Volkswagen emission cheat. After they fix the ECU to have the advertised NOx emissions, the car lost peak power and fuel efficiency.
- noncoml 9y agoWhat was the problem? All the notes say is that 109 fixes it.
- jonathonf 9y agohttps://launchpad.net/ubuntu/+source/linux/4.4.0-109.132 https://launchpad.net/ubuntu/+source/linux/4.4.0-109.132 : linux (4.4.0-109.132) xenial; urgency=low * linux: 4.4.0-109.132 -proposed tracker (LP: #1742252) * Kernel trace with xenial 4.4 (4.4.0-108.131, Candidate kernels for PTI fix) (LP: #1741934) - SAUCE: kaiser: fix perf crashes - fix to original commit diff'ing the two changes it was this: > diff -u linux-4.4.0/arch/x86/events/intel/ds.c linux-4.4.0/arch/x86/events/intel/ds.c > --- linux-4.4.0/arch/x86/events/intel/ds.c > +++ linux-4.4.0/arch/x86/events/intel/ds.c > @@ -415,7 +415,6 @@ > return; > > per_cpu(cpu_hw_events, cpu).ds = NULL; > - kfree(ds); > } > > void release_ds_buffers(void) plus it's here: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1741934/comments/17 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1741934...
- AdmiralAsshat 9y agoThey consider a bug that renders the OS unable to boot a "low" urgency!?!
- revelation 9y agoThe last time I used Ubuntu, they hadn't yet realized what vertical display synchronization is, and nobody had explained to them that you don't do your rendering on the framebuffer you're currently scanning out. So an occasional boot of the recovery kernel truly vanishes behind the plain broken display they expect you to put up with.
- lunorian 9y agoSorry fam - security issues are more important ¯\_(ツ)_/¯
- 9y ago
- user5994461 9y agoDid ubuntu botched an update again? Or is it the upstream kernel?
- eecc 9y agoHas Ubutu botched or did Ubutu botch... please ;)
- deleted 9y ago[deleted]
- ksenzee 9y agoMuphry's Law strikes again!
- jwilk 9y agohttps://en.wikipedia.org/wiki/Muphry%27s_law https://en.wikipedia.org/wiki/Muphry%27s_law If you write anything criticizing editing or proofreading, there will be a fault of some kind in what you have written.
- nkkollaw 9y agoCan't spell Ubuntu right, though.
- eecc 9y agoWhatever, it’s a commercial name (might have even been the autocorrect.) I recommended — nicely, no offense intended — the correct grammatical form. Stop staring at my finger. Please ;)
- gkya 9y agoWould this entire Meltdown/Spectre thing count as the biggest mess-up of computing history? When yesterday the PoC repo was posted here, the spy.mp4 demo video gave me some chills. And now I can't update my OS before making an installation USB because Canonical can't just follow Linus' releases. Thanks.
- aaomidi 9y agoLink to the demo?
- elihu 9y agofrom https://github.com/IAIK/meltdown/ https://github.com/IAIK/meltdown/: https://cdn.rawgit.com/IAIK/meltdown/master/videos/spy.mp4 https://cdn.rawgit.com/IAIK/meltdown/master/videos/spy.mp4
- bringtheaction 9y agoYouTube mirror for mobile users anyone?
- gregsadetsky 9y agohttps://www.youtube.com/watch?v=RbHbFkh6eeE https://www.youtube.com/watch?v=RbHbFkh6eeE
- bArray 9y ago>Would this entire Meltdown/Spectre thing count as the biggest mess-up of computing history? When yesterday the PoC repo was posted here, the spy.mp4 demo video gave me some chills. It must be up there amongst the greats, probably with the "halt and catch fire" op code. Normally they just patch this stuff with microcode and never really tell anybody, this time that won't work. I'm not entirely convinced it was a mistake at all (dons tin foil hat), Intel have been making suspicious design decisions to their chips for a well now (think this, Intel Me, hidden instructions, etc). It seems clear to me that this security by obscurity approach is quite frankly crap. >And now I can't update my OS before making an installation USB because Canonical can't just follow Linus' releases. Thanks. Linus' releases need some sort of buffering before they can be considered stable, often distributions will apply their own patches on top. Also consider the scenario where Linus releases a bad kernel and no testing has been performed before rolling out to all Linux users.
- lunorian 9y agoSee this is why you wait a day or two before patching :)
- Whitestrake 9y agoIf everyone waited a day or two before patching, this bug would simply be opened a day or two later than it was.
- snuxoll 9y agoHow hard is it to just boot an older kernel and rollback the default? Before I even thought about patching sensitive systems tonight the first thing our IT director asked was if I had a rollback plan. The answer? "Yes, boot old kernel, yum history undo [transaction id], reboot". Always have a backout plan when doing upgrades, I'm just glad EL and derived distributions have an easy way to do it with yum's transaction history.
- tripzilch 9y ago(tinfoil hat) That may be this kernel boot bug exists--the agencies are just trying to squeeze a few more days in of extracting data from prime targets, conveniently under the guise of public knowledge about exploiting it.
- hansendc 9y agoThis arch/x86/events/intel/ds.c fix is unlikely to have rendered too many things unbootable. I missed ds.c entirely when doing the original implementation. I can't fault the nice folks at Canonical for mis-merging a tiny hunk like this. It really only affects pretty specific hardware anyway.
- MBCook 9y agoWhat hardware does it effect?
- JackNichelson 9y agoPretty specific one.
- ASalazarMX 9y agoTake my hand, I'll guide you back to Reddit.
- fancyfacebook 9y agoFrom the bug it looks like a smattering of Xeons, Celerons, i5s, not sure what's "specific" about it from first glance.
- mycpuorg 9y agoPlease look under "Meltdown - x86" section in GKH's (The Stable Kernel Maintainer) blog: http://www.kroah.com/log/ http://www.kroah.com/log/
- nabilt 9y agoI just updated my Dell, but haven't restarted. Do we know how widespread the problem is and should I roll back the update?
- bArray 9y agoI would suspend your machine until we find out more, I have the same problem.
- qznc 9y agoFor me the 108 kernel boots fine, but breaks suspend. :D
- Nacraile 9y agoYou all do realize that (at least by default) ubuntu will keep old kernel versions around, and you can choose to boot them in GRUB, don't you? This is certainly a pain, but it's hardly the first time a broken kernel has shipped. Reasonable recovery mechanisms are in place.
- jlgaddis 9y agoThere's a -109 out now. I'd install it before rebooting and probably expunge -108 entirely.
- krutzger 9y agoI was under the impression that Ubuntu would automatically revert to last good kernel of the new one fails to boot. Was I mistaken?
- bproven 9y agoAFAIK worse case you should be able to select the previous kernel in grub2 upon reboot. It just hangs after grub on this (bad) kernel I think...
- ams6110 9y agoYeah but if you have a couple of hundred machines that aren't booting.... That's pretty worst case.
- bproven 9y agoyeah - I hear you. :( Still better than being totally hosed...at least there is some option
- jlgaddis 9y agoThat is pretty worst case. OTOH, it teaches the lesson about testing updates on a small number of hosts before rolling 'em out globally. One has to learn that lesson at some point.
- myth_drannon 9y agoNo, you have to manually restart, boot into grub and select the one that works
- michaelt 9y agoThere's something close-but-not-quite-the-same: by default the grub menu is hidden with an instant timeout (IIRC) but if a boot fails to complete, on the subsequent boot the menu won't be hidden. Google 'recordfail' if you're interested in the details.
- trendia 9y ago(Copying my instructions from another post). If kernel 4.4 doesn't work, I recommend compiling the 4.15 kernel. (Note, however, that you may need to apply a patch to NVIDIA drivers). I've done this on Ubuntu 16.04 LTS, 17.10, and Debian 8 so far this week. To compile, set CONFIG_PAGE_TABLE_ISOLATION=y. That is: sudo apt-get build-dep linux sudo apt-get install gcc-6-plugin-dev libelf-dev libncurses5-dev cd /usr/src wget https://git.kernel.org/torvalds/t/linux-4.15-rc7.tar.gz tar -xvf linux-4.15-rc7.tar.gz cd linux-4.15-rc7 cp /boot/config-`uname -r` .config make CONFIG_PAGE_TABLE_ISOLATION=y deb-pkg
- lathiat 9y agoI wouldn't really recommend doing this, but if you really want to do this, it would probably be easier just to use the pre-spun mainline kernels: https://wiki.ubuntu.com/Kernel/MainlineBuilds https://wiki.ubuntu.com/Kernel/MainlineBuilds
- deleted 9y ago[deleted]
- gphreak 9y agoExactly, and use 4.14. According to a recent comment from a kernel dev both kernels use the same patch approach. 4.4 and 4.9 are using a different approach that’s less ideal, less complete and apparently less tested.
- trendia 9y agoI can somewhat understand where you're coming from if the person doing it only wants something that works. But for those who would be willing to risk breaking a few things to try something out, building a kernel is a worthwhile effort, and the meltdown / spectre bugs provide a perfect excuse to do it.
- agumonkey 9y agoany report of windows update messing with the bios rendering motherboard non bootable (powers on , but no post, not even an error beep)
- otakucode 9y agoFor AMD chips, yes. Yesterday Microsoft announced they were suspending rolling out updates to certain AMD chips because it was resulting in non-bootable systems. I didn't read the technical details so I can not say whether it was specifically BIOS-related. Both a total non-booting state and BSODs were mentioned in the article I saw (from general press, so might have been garbage, sorry).
- rincebrain 9y agoMicrosoft's claim was "Microsoft has determined that some AMD chipsets do not conform to the documentation previously provided to Microsoft to develop the Windows operating system mitigations to protect against the chipset vulnerabilities known as Spectre and Meltdown"[1], and their docs simply suggest asking AMD for more details[2]. So it sounds like it was probably specific chipsets and not CPUs, but who knows. [1] - https://www.engadget.com/2018/01/09/microsoft-halts-meltdown-spectre-amd-patches/ https://www.engadget.com/2018/01/09/microsoft-halts-meltdown... [2] - https://support.microsoft.com/en-us/help/4056892/windows-10-update-kb4056892 https://support.microsoft.com/en-us/help/4056892/windows-10-...
- cpncrunch 9y agoIt happened to one of my HP boxes that has an AMD chipset. After installing the update, windows 10 just hangs at the blue windows logo. Only solution is to turn the machine off and on twice, which then results in it undoing the update. The Microsoft link you provide says "Microsoft is working with AMD to resolve this issue", so they're not just brushing it off and telling customers to contact AMD.
- lurr 9y agoSo do we make snide remarks about fixes not being tested like we did when microsoft also had issues fixing CPU level bugs?
- del_operator 9y agoSounds like one way to stop the bug. :P
- jnordwick 9y agoI will not be updating. I have yet to see this mythic JavaScript exploit, and I see too many other ways I, as an end user, can be affected. I haven't even seen a proof of concept exploit that has the same conditions as in the wild. All the POC exploits seems to have been given some assistance in various ways (such as being given root perms or a preknown memory address). Does anybody have an example of this JavaScript exploit or any exploit that would work in the wild?
- nykolasz 9y agoWaiting a few days to patch my own servers... Not sure what is more dangerous right now: applying these rushed patches or the vuln itself.
- snuxoll 9y agoWhat distro are you running? I trust Red Hat to get kernel updates right the first time, I just patched externally facing servers and systems that handle PHI tonight with no issues (outside of one of my PostgreSQL servers showing a non-neglible increase in CPU usage, damnit Intel). Of course, I also go into any updates with a rollback plan. ITIL sucks, but one thing it taught me was the value of well documented plans any time you make changes to production systems.
- gphreak 9y agoSame. RHEL/CentOS went without a hitch. The age of the kernel start’s to concern me, though. According to the top comment in one of the posts in HN even 4.9 and 4.4 use a less ideal patch: https://news.ycombinator.com/item?id=16085672 https://news.ycombinator.com/item?id=16085672 I can’t really judge how much RH engineers are capable of fixing that kind of stuff in a kernel that’s officially out of support upstream. Based on the general quality of RHEL/RHV I trust them to do the right thing, but I have no insight whatsoever in how kernel development actually works.
- snuxoll 9y agoRed Hat pays the salary of a couple kernel developers, backporting security fixes is a pretty big part of their job. Keep in mind, RHEL/CentOS 7 doesn't even use something newer like 4.4 - it's still on 3.10 because Red Hat guarantees a stable kABI throughout the lifetime of a release
- IceyEC 9y agoThis seems like one of those things that is very hard to get right: https://forums.aws.amazon.com/thread.jspa?messageID=823033 https://forums.aws.amazon.com/thread.jspa?messageID=823033
- nkkollaw 9y agoThis is a little bit sensationalist (is that a word?). It's not like Windows that bricks your laptop. It's a handful of hardware config, and you can easily boot with an older kernel.