3 ms·
The complexity of building NumPy or SciPy is pretty much par for the course in HPC. Most of the libraries it is based on are battle-hardened and ridiculously we
by gh02t 11y ago
The complexity of building NumPy or SciPy is pretty much par for the course in HPC. Most of the libraries it is based on are battle-hardened and ridiculously well tested to a degree not seen in many places, but they date from a different era, including libraries where originally "building" it involved a hole puncher and a bucket of coffee. Getting all those libraries to play nicely together at all is a genuine miracle.
Most any significant scientific package I've ever had to build (which is quite a few) had a complex build system that felt like it was built on top of shaky glass pillars. It's just the nature of what's involved and it's a fundamentally hard problem; it's difficult to replace those libraries because the extreme amount of verification/validation and optimization that have gone into something like LAPACK makes it economically undesirable.
I'd say also that this is part of the appeal of NumPy and SciPy, which is they glue together a huge number of tools that are otherwise disjoint. You don't have to worry (most of the time) about matrix ordering, calling conventions, those mysterious "workspace" variables in LAPACK etc.