7 ms·
This is targeted at ML, which means this is targeted at Linux. Which means us system administrators have to deal with the piece of shit that is the Nvidia bin
by bryanlarsen 9y ago
This is targeted at ML, which means this is targeted at Linux. Which means us system administrators have to deal with the piece of shit that is the Nvidia binary driver.
- it doesn't work with Wayland
- it doesn't work in UEFI mode with CSM enabled. There are motherboards available that won't boot in UEFI mode with CSM disabled without a monitor attached.
- it doesn't work with Secure Boot
- about half of their releases fail to work with at least one of the pieces of software we try to use it with. The current release 384.90-0ubuntu0.16.04.2 in particular crashes our software.
And for some reason Nvidia has the reputation for being good at drivers. FirePro was also really bad, but I haven't had to deal with that for years now...
- bryanlarsen 9y agoAnd in case this is getting visibility at Nvidia, this is also another major source of frustration: https://github.com/NVIDIA/nvidia-docker/issues/11 https://github.com/NVIDIA/nvidia-docker/issues/11 Some of the software we use is linked against GL to display results when run on the desktop. It doesn't actually use GL when run on the server, but it's still linked against GL so we need it on the server.
- angry_octet 9y agoCan't you just bundle some GL libs in a library search path on the server? Or does it init the GL context?
- bryanlarsen 9y agoWhat we do is we install the nVidia binary driver package inside the docker container, then delete some files that overlap with those that nvidia-docker injects.
- bhouston 9y agoAnd the last major stable release for Ubuntu broke egl completely.
- blakesterz 9y agoIs there a generally accepted "best" manufacturer for Linux cards? I keep reading to avoid Nvidia, I'm not quite sure what to get though.
- baobrien 9y agoAMD's open source drivers on recent are pretty solid, AFAIK, and the proprietary ones don't suck.
- baldfat 9y agoAMD had released the most documentation and yet the community loves to pile on AMD. I think there is no company that could have done better than what they did in 2014 Linux Driver announcement https://www.phoronix.com/scan.php?page=article&item=amd_catalyst_kernel&num=1 https://www.phoronix.com/scan.php?page=article&item=amd_cata...
- Spivak 9y agoBut it has to be recent. If the card you're looking at runs the radeon driver then run for the hills.
- dragontamer 9y agoAMD also has the ROCm commitment going on. All the ROCm drivers are fully open source on GitHub. ROCm doesn't have display drivers, but the compute-drivers are more important for well... compute tasks. (You'll probably be doing development on another machine, transfer code to the ROCm server box and then execute it through SSH or something). So forward looking, one can reasonably expect that AMD's Open Source drivers will be superior. But this only applies to RX 4xx series and newer, and not even to all those cards. (Some cards like RX 550 just don't work right now, despite claimed support on marketing pages).
- angry_octet 9y agoI'm, buy NVIDIA, they are the best? You can run cuda on them and not anything else? Don't buy NVIDIA if you don't care about performance of OpenGL/etc.
- jrs95 9y agoInteresting. I had assumed most people just did ML on Windows specifically because of graphics driver issues. Even if this list weren’t a problem, Windows drivers tend to outperform Mac and Linux drivers by a significant margin.
- IshKebab 9y agoML frameworks are written by Unixy types. Torch doesn't even run on Windows, and it's clearly a second-tier platform for Tensorflow.
- dangerbird2 9y agoFrom my experience, the main pain point with Linux GPU drivers is for standard "desktop" tasks, like not breaking when Ubuntu switches to Wayland... A server designated for GPGPU applications just need the basic configuration to get the Nvidia toochain for CUDA up and running
- bryanlarsen 9y agoIt'd be nice if their CUDA libraries worked with the open source "nouveau" library. All the driver does is init the card and do some resource management; all the secret sauce is in the CUDA libraries.
- angry_octet 9y agoI have way more graphics/cuda driver issues on Windows... I'm not seeing a performance delta that extreme either. If you are comparing AAA games, the driver does complete code rewrites on anything important, that's an apples/oranges comparison.
- jamesfmilne 9y agoThe NVIDIA driver is most certainly not a piece of shit. NVIDIA is a bit behind on things like Wayland, but it has been way ahead of AMD for the last 10+ years on Linux for people running mission-critical software that requires good OpenGL drivers. Try running with AMD hardware on RHEL/CentOS 6/7 (the OS many major VFX facilities use), you'll have a Bad Time (tm). I am speaking as someone who has been maintaining software & hardware running Linux making heavy use of NVIDIA hardware for the last ten years. The NVIDIA driver on macOS on the other hand, now there is a lamentable situation...
- bryanlarsen 9y agoRHEL 6 was released in 2010. At that time Nvidia was way better than AMD. In the last 7 years AMD has improved dramatically, but nVidia has gotten worse. Ubuntu and Red Hat will probably have new LTS versions using Wayland coming out in 2018. Which means that 2017 is the time to start testing for them. And it's not really a glowing endorsement if Nvidia is only the second worst driver on Linux, not the absolute worst.
- LunaSea 9y agoSo NVIDIA's drivers are good because AMD's are worse?
- CreepGin 9y agoJust not a piece of shit
- bryanlarsen 9y agoThat choice of terms was my frustration leaking through. Those drivers wasted 4 hours of my day yesterday, and several whole days this last month.
- moonbug22 9y agoWhatchya doin updating them in the first place? Pick a version, install it against dkms and forget about it.
- cobookman 9y agoSounds like a great opportunity for microsoft to pitch Windows Server.
- ordinaryperson 9y agoNvidia 384 drivers on Ubuntu 17.10 have left me unable to reboot or shut down, I have to manually hard cycle off the power. And this is on Xorg, not Wayland. And good luck getting Nvidia "Optimus" properly configured and running smoothly.
- erik_seaberg 9y agoHow would a sysadmin use Wayland? I would expect lack of remoting to be a showstopper.
- earenndil 9y agoFreebsd? AFAIK they have much better drivers there.