4 ms·
Kind of disappointing that the Guix pytorch package has no GPU support: https://guix.gnu.org/en/packages/python-pytorch-1.12.0/ https://guix.gnu.org/en/package
by amelius 4y ago
Kind of disappointing that the Guix pytorch package has no GPU support:
https://guix.gnu.org/en/packages/python-pytorch-1.12.0/ https://guix.gnu.org/en/packages/python-pytorch-1.12.0/
This makes me wonder if Guix is ready for the real world.
- grnmamba 4y agoCan't say the N word in Guix I think Guix is more than ready for providing development environments, even for proprietary apps on open source technology stacks (but Nix will probably be easier to pitch). As an operating system much less so.
- zetalyrae 4y agoIf Guix uses Nix under the hood, is it "just" Nix with an S-expression syntax (admittedly much better than Nix syntax) and fewer packages? If so, is there a really compelling reason to prefer Guix over NixOS? The only substantive difference I'm aware of is the use of GNU Shepherd instead of systemd.
- fiddlerwoaroof 4y agoAs far as I know it doesn't use nix under the hood, it's a reimplementation of nix in guile scheme.
- zetalyrae 4y agoAh, my mistake. I was certain it wrapped around Nix for some reason.
- cygx 4y agoSame backend, but different frontend, so to speak. Hence, it used to be possible to have nix and guix share a store and local state directory. Not sure if that's still possible today, though...
- filterfish 4y agoIt isn't.
- yjftsjthsd-h 4y ago> This makes me wonder if Guix is ready for the real world. ML and GPUs are a smallish subset of the real world, and a subset that's particularly plagued by proprietary software (especially the GPU side); I don't think it's a good litmus test for "is guix useful for real work".
- amelius 4y agoML and GPUs are on HN's front page almost every day ...
- nextos 4y agoYou can use this channel to get non-GNU sanctioned software, including kernels and drivers: https://gitlab.com/nonguix/nonguix https://gitlab.com/nonguix/nonguix Furthermore, you can install Nix in Guix: https://guix.gnu.org/packages/nix-2.5.1 https://guix.gnu.org/packages/nix-2.5.1
- behnamoh 4y agothen why not just install Nix in the first place??
- nextos 4y agoI use NixOS, but GuixSD users might find this option useful.
- ParetoOptimal 4y agoYou prefer using guix but want a package Nix has already packaged and don't have the time and knowledge to package it yourself.
- dan-robertson 4y agoThere are a bunch of things that come up regularly on HN that aren’t particularly relevant to the real world so I don’t think this logic is good. Though I think you are correct that ML is relevant to the real world. However when you wrote ‘the real world’ you didn’t mean the thing for which ML is relevant but rather ‘being a general purpose operating system that is useful [to software developers]’ and for this I think the lack of hardware-accelerated PyTorch isn’t great but I don’t think training/running ML models is particularly relevant to being a general purpose OS (even for software developers). I’m not sure it’s even relevant to ML practitioners as I would expect they’d be farming out training to a dedicated compute cluster. One needs to jump through a bunch of hoops to get hardware-accelerated PyTorch on Apple’s latest computers. Does that mean they aren’t ready for the real world?
- dannymi 4y agoI tried to change Guix to enable vulkan in pytorch and I get a build error. Since I don't use pytorch I didn't investigate further. If you do know the build options to make this work, Guix will gladly consider your patch. The in-progress diff would be, diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index e702e499fc..73af29487d 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -101,6 +101,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages swig) #:use-module (gnu packages tls) #:use-module (gnu packages video) + #:use-module (gnu packages vulkan) #:use-module (gnu packages web) #:use-module (gnu packages xml) #:use-module (gnu packages xorg) @@ -2897,6 +2898,10 @@ (define-public python-pytorch (build-system python-build-system) (arguments '(#:phases (modify-phases %standard-phases + (add-before 'build 'use-vulkan + (lambda _ + (setenv "USE_VULKAN" "1") + (setenv "USE_VULKAN_SHADERC_RUNTIME" "1"))) (add-before 'build 'use-system-libraries (lambda\* (#:key outputs #:allow-other-keys) ;; Tell 'setup.py' to let 'CMakeLists.txt' know that we @@ -2973,7 +2978,11 @@ (define-public python-pytorch pthreadpool protobuf pybind11 + shaderc sleef + glslang spirv-headers ;spirv-tools ; not sure why this is needed + vulkan-headers + vulkan-loader xnnpack zstd)) (propagated-inputs Error messages: FAILED: bin/scalar_test : && /gnu/store/069aq2v993kpc41yabp5b6vm4wb9jkhg-gcc-10.3.0/bin/c++ -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_QNNPACK -DUSE_PYTORCH_QN NPACK -DUSE_XNNPACK -DUSE_VULKAN -DUSE_VULKAN_API -DUSE_VULKAN_SHADERC_RUNTIME -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing -field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno -error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wn o-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -DHAVE_AVX512_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -rdynamic - pthread caffe2/CMakeFiles/scalar_test.dir/__/aten/src/ATen/test/scalar_test.cpp.o -o bin/scalar_test -Wl,-rpath,/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib: -lgtest_main -lgtest /gnu/store/0 94bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/libgomp.so /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libpthread.so -Wl,--no-as-needed,"/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/bui ld/lib/libtorch.so" -Wl,--as-needed -Wl,--no-as-needed,"/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so" -Wl,--as-needed /gnu/store/9pyydl5w9xnz1qm56sxn1zh4qny6fkxz-protobuf-3.17.3 /lib/libprotobuf.so lib/libc10.so -pthread && : ld: /tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so: undefined reference to `glslang::TShader::setNanMinMaxClamp(bool)' ld: /tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so: undefined reference to `spvtools::CreateCompactIdsPass()' [...] The reason is because pytorch 1.12.0 depends on shaderc-2020.4/lib/libshaderc_combined.a--and static libraries do not have automatic dependency resolution, so spirv, glsl etc is never linked.
- pxc 4y agoGuix already sees use in the 'real world' in a number of HPC environments.