4 ms·
I tried to change Guix to enable vulkan in pytorch and I get a build error. Since I don't use pytorch I didn't investigate further. If you do know the build op
by dannymi 4y ago
I tried to change Guix to enable vulkan in pytorch and I get a build error. Since I don't use pytorch I didn't investigate further.
If you do know the build options to make this work, Guix will gladly consider your patch.
The in-progress diff would be,
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index e702e499fc..73af29487d 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -101,6 +101,7 @@ (define-module (gnu packages machine-learning)
#:use-module (gnu packages swig)
#:use-module (gnu packages tls)
#:use-module (gnu packages video)
+ #:use-module (gnu packages vulkan)
#:use-module (gnu packages web)
#:use-module (gnu packages xml)
#:use-module (gnu packages xorg)
@@ -2897,6 +2898,10 @@ (define-public python-pytorch
(build-system python-build-system)
(arguments
'(#:phases (modify-phases %standard-phases
+ (add-before 'build 'use-vulkan
+ (lambda _
+ (setenv "USE_VULKAN" "1")
+ (setenv "USE_VULKAN_SHADERC_RUNTIME" "1")))
(add-before 'build 'use-system-libraries
(lambda\* (#:key outputs #:allow-other-keys)
;; Tell 'setup.py' to let 'CMakeLists.txt' know that we
@@ -2973,7 +2978,11 @@ (define-public python-pytorch
pthreadpool
protobuf
pybind11
+ shaderc
sleef
+ glslang spirv-headers ;spirv-tools ; not sure why this is needed
+ vulkan-headers
+ vulkan-loader
xnnpack
zstd))
(propagated-inputs
Error messages:
FAILED: bin/scalar_test
: && /gnu/store/069aq2v993kpc41yabp5b6vm4wb9jkhg-gcc-10.3.0/bin/c++ -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_QNNPACK -DUSE_PYTORCH_QN
NPACK -DUSE_XNNPACK -DUSE_VULKAN -DUSE_VULKAN_API -DUSE_VULKAN_SHADERC_RUNTIME -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing
-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno
-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wn
o-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -DHAVE_AVX512_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -rdynamic -
pthread caffe2/CMakeFiles/scalar_test.dir/__/aten/src/ATen/test/scalar_test.cpp.o -o bin/scalar_test -Wl,-rpath,/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib: -lgtest_main -lgtest /gnu/store/0
94bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/libgomp.so /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libpthread.so -Wl,--no-as-needed,"/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/bui
ld/lib/libtorch.so" -Wl,--as-needed -Wl,--no-as-needed,"/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so" -Wl,--as-needed /gnu/store/9pyydl5w9xnz1qm56sxn1zh4qny6fkxz-protobuf-3.17.3
/lib/libprotobuf.so lib/libc10.so -pthread && :
ld: /tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so: undefined reference to `glslang::TShader::setNanMinMaxClamp(bool)'
ld: /tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so: undefined reference to `spvtools::CreateCompactIdsPass()'
[...]
The reason is because pytorch 1.12.0 depends on shaderc-2020.4/lib/libshaderc_combined.a--and static libraries do not have automatic dependency resolution, so spirv, glsl etc is never linked.
- dannymi 4y agoJust tried--same problem with pytorch 1.12.1. So that's why it's not enabled.
- dannymi 4y agopytorch is using shaderc_combined.a. There's a shaderc_combined.pc in shaderc, but it doesn't lists glslang or spirv as dependencies, even though shaderc uses glslang and spirv data structures and functions. Errrrr. How's that supposed to work?