3 ms·

Over the years, the merits of Intel’s 512-bit Advanced Vector eXtensions (AVX-512) have been extensively debated. Introduced in 2014, CPUs took time to offer ro

Over the years, the merits of Intel’s 512-bit Advanced Vector eXtensions (AVX-512) have been extensively debated. Introduced in 2014, CPUs took time to offer robust support. In parallel, Arm Scalable Vector Extensions (SVE), targeting Arm servers, has only gained traction in recent times. Today, the landscape has shifted significantly with Intel’s Sapphire Rapids CPUs on one flank and the AWS Graviton 3 and Ampere Altra chips on the other. Here are compelling reasons to opt for these over the traditional AVX2 and NEON extensions: 1. Masked Loads: Efficient data processing by selectively loading data. 2. Half-Precision Floating Point Math: Accelerated computations with reduced memory footprint. These features proved invaluable for the latest SimSIMD release. The software now processes vector similarities up to 300x faster using NEON, SVE, AVX2, and AVX-512 extensions across Inner Product, Euclidean, Angular, Hamming, and Jaccard distances. It outstrips commonly used libraries like NumPy and SciPy, famously built on BLAS and LAPACK. Check out the post for some cool tricks, clarifications on AVX-512 and SVE advantages, and benchmark numbers :) Here is the repo: https://github.com/ashvardanian/simsimd https://github.com/ashvardanian/simsimd

westurner 3y ago
I encourage one to merge into e.g. {NumPy, SciPy, }; are there PRs? Though SymPy.physics only yet supports X,Y,Z vectors and doesn't mention e.g. "jaccard"?, FWIW: https://docs.sympy.org/latest/modules/physics/vector/vectors.html https://docs.sympy.org/latest/modules/physics/vector/vectors... https://docs.sympy.org/latest/modules/physics/vector/fields.html#curl https://docs.sympy.org/latest/modules/physics/vector/fields.... #cfd include/simsimd/simsimd.h: https://github.com/ashvardanian/SimSIMD/blob/main/include/simsimd/simsimd.h https://github.com/ashvardanian/SimSIMD/blob/main/include/si... conda-forge maintainer docs > Switching BLAS implementation: https://conda-forge.org/docs/maintainer/knowledge_base.html#switching-blas-implementation https://conda-forge.org/docs/maintainer/knowledge_base.html#... : conda install "libblas=*=*mkl" conda install "libblas=*=*openblas" conda install "libblas=*=*blis" conda install "libblas=*=*accelerate" conda install "libblas=*=*netlib" numpy-feedstock: https://github.com/conda-forge/numpy-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/numpy-feedstock/blob/main/rec... scipy-feedstock: https://github.com/conda-forge/scipy-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/scipy-feedstock/blob/main/rec... pysimdjson-feedstock: https://github.com/conda-forge/pysimdjson-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/pysimdjson-feedstock/blob/mai... simdjson-feedstock: https://github.com/conda-forge/simdjson-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/simdjson-feedstock/blob/main/... mkl_random-feedstock: https://github.com/conda-forge/mkl_random-feedstock https://github.com/conda-forge/mkl_random-feedstock https://github.com/google/paranoid_crypto/tree/main/paranoid_crypto/lib/randomness_tests https://github.com/google/paranoid_crypto/tree/main/paranoid... : > NumPy-based implementation of random number generation sampling using Intel (R) Math Kernel Library, mirroring numpy.random, but exposing all choices of sampling algorithms available in MKL blas: https://github.com/conda-forge/blas-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/blas-feedstock/blob/main/reci... xtensor-blas-feedstock: https://github.com/conda-forge/xtensor-blas-feedstock https://github.com/conda-forge/xtensor-blas-feedstock xtensor-fftw (FFT with xtensor (c++)) could probably be AVX-512 and SVE -optimized as well? https://github.com/xtensor-stack/xtensor-fftw https://github.com/xtensor-stack/xtensor-fftw ggml_cpu_has_avx512() https://github.com/search?q=repo%3Aggerganov%2Fggml%20AVX&type=code https://github.com/search?q=repo%3Aggerganov%2Fggml%20AVX&ty... https://github.com/search?q=repo%3Aggerganov%2Fllama.cpp%20avx&type=code https://github.com/search?q=repo%3Aggerganov%2Fllama.cpp%20a... CuPy would also be an impactful place to merge and defend these optimizations; though no GPUs have AVX-512 or SVE? cupyx.scipy.spatial.distance: https://docs.cupy.dev/en/stable/reference/scipy_spatial_distance.html https://docs.cupy.dev/en/stable/reference/scipy_spatial_dist... https://docs.cupy.dev/en/stable/reference/comparison.html https://docs.cupy.dev/en/stable/reference/comparison.html
- ashvardanian 3y ago
  Hey, thanks for recommendations! Yes, I’m definitely open to contributing there.
  - westurner 3y ago
    Np. Thanks for the optimizations. From "PostgresML is 8-40x faster than Python HTTP microservices" (2023) https://news.ycombinator.com/item?id=33270638 https://news.ycombinator.com/item?id=33270638 : > Apache Ballista and Polars do Apache Arrow and SIMD. > The Polars homepage links to the "Database-like ops benchmark" of {Polars, data.table, DataFrames.jl, ClickHouse, cuDF, spark, (py)datatable, dplyr, pandas, dask, Arrow, DuckDB, Modin,} but not yet PostgresML? https://h2oai.github.io/db-benchmark/ https://h2oai.github.io/db-benchmark/ * LLM -> Vector database: https://en.wikipedia.org/wiki/Vector_database https://en.wikipedia.org/wiki/Vector_database /? inurl:awesome site:github.com "vector database" https://www.google.com/search?q=inurl%253Aawesome+site%253Agithub.com+%22vector+database%22 https://www.google.com/search?q=inurl%253Aawesome+site%253Ag... : https://github.com/dangkhoasdc/awesome-vector-database https://github.com/dangkhoasdc/awesome-vector-database , https://github.com/mileszim/awesome-vector-database https://github.com/mileszim/awesome-vector-database , https://github.com/currentslab/awesome-vector-search https://github.com/currentslab/awesome-vector-search /? "vector database" "duckdb" https://www.google.com/search?q=+%22vector+database%22+%22duckdb%22 https://www.google.com/search?q=+%22vector+database%22+%22du... ... pgvector pgvector/pgvector/src/vector.c: vector_spherical_distance https://github.com/pgvector/pgvector/blob/master/src/vector.c#L694C1 https://github.com/pgvector/pgvector/blob/master/src/vector.... postgresml/postgresml: /? distance https://github.com/search?q=repo%3Apostgresml%2Fpostgresml%20distance&type=code https://github.com/search?q=repo%3Apostgresml%2Fpostgresml%2...

westurner 3y ago

I encourage one to merge into e.g. {NumPy, SciPy, }; are there PRs? Though SymPy.physics only yet supports X,Y,Z vectors and doesn't mention e.g. "jaccard"?, FWIW: https://docs.sympy.org/latest/modules/physics/vector/vectors.html https://docs.sympy.org/latest/modules/physics/vector/vectors... https://docs.sympy.org/latest/modules/physics/vector/fields.html#curl https://docs.sympy.org/latest/modules/physics/vector/fields.... #cfd include/simsimd/simsimd.h: https://github.com/ashvardanian/SimSIMD/blob/main/include/simsimd/simsimd.h https://github.com/ashvardanian/SimSIMD/blob/main/include/si... conda-forge maintainer docs > Switching BLAS implementation: https://conda-forge.org/docs/maintainer/knowledge_base.html#switching-blas-implementation https://conda-forge.org/docs/maintainer/knowledge_base.html#... : conda install "libblas=*=*mkl" conda install "libblas=*=*openblas" conda install "libblas=*=*blis" conda install "libblas=*=*accelerate" conda install "libblas=*=*netlib" numpy-feedstock: https://github.com/conda-forge/numpy-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/numpy-feedstock/blob/main/rec... scipy-feedstock: https://github.com/conda-forge/scipy-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/scipy-feedstock/blob/main/rec... pysimdjson-feedstock: https://github.com/conda-forge/pysimdjson-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/pysimdjson-feedstock/blob/mai... simdjson-feedstock: https://github.com/conda-forge/simdjson-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/simdjson-feedstock/blob/main/... mkl_random-feedstock: https://github.com/conda-forge/mkl_random-feedstock https://github.com/conda-forge/mkl_random-feedstock https://github.com/google/paranoid_crypto/tree/main/paranoid_crypto/lib/randomness_tests https://github.com/google/paranoid_crypto/tree/main/paranoid... : > NumPy-based implementation of random number generation sampling using Intel (R) Math Kernel Library, mirroring numpy.random, but exposing all choices of sampling algorithms available in MKL blas: https://github.com/conda-forge/blas-feedstock/blob/main/recipe/meta.yaml https://github.com/conda-forge/blas-feedstock/blob/main/reci... xtensor-blas-feedstock: https://github.com/conda-forge/xtensor-blas-feedstock https://github.com/conda-forge/xtensor-blas-feedstock xtensor-fftw (FFT with xtensor (c++)) could probably be AVX-512 and SVE -optimized as well? https://github.com/xtensor-stack/xtensor-fftw https://github.com/xtensor-stack/xtensor-fftw ggml_cpu_has_avx512() https://github.com/search?q=repo%3Aggerganov%2Fggml%20AVX&type=code https://github.com/search?q=repo%3Aggerganov%2Fggml%20AVX&ty... https://github.com/search?q=repo%3Aggerganov%2Fllama.cpp%20avx&type=code https://github.com/search?q=repo%3Aggerganov%2Fllama.cpp%20a... CuPy would also be an impactful place to merge and defend these optimizations; though no GPUs have AVX-512 or SVE? cupyx.scipy.spatial.distance: https://docs.cupy.dev/en/stable/reference/scipy_spatial_distance.html https://docs.cupy.dev/en/stable/reference/scipy_spatial_dist... https://docs.cupy.dev/en/stable/reference/comparison.html https://docs.cupy.dev/en/stable/reference/comparison.html

ashvardanian 3y ago
Hey, thanks for recommendations! Yes, I’m definitely open to contributing there.
- westurner 3y ago
  Np. Thanks for the optimizations. From "PostgresML is 8-40x faster than Python HTTP microservices" (2023) https://news.ycombinator.com/item?id=33270638 https://news.ycombinator.com/item?id=33270638 : > Apache Ballista and Polars do Apache Arrow and SIMD. > The Polars homepage links to the "Database-like ops benchmark" of {Polars, data.table, DataFrames.jl, ClickHouse, cuDF, spark, (py)datatable, dplyr, pandas, dask, Arrow, DuckDB, Modin,} but not yet PostgresML? https://h2oai.github.io/db-benchmark/ https://h2oai.github.io/db-benchmark/ * LLM -> Vector database: https://en.wikipedia.org/wiki/Vector_database https://en.wikipedia.org/wiki/Vector_database /? inurl:awesome site:github.com "vector database" https://www.google.com/search?q=inurl%253Aawesome+site%253Agithub.com+%22vector+database%22 https://www.google.com/search?q=inurl%253Aawesome+site%253Ag... : https://github.com/dangkhoasdc/awesome-vector-database https://github.com/dangkhoasdc/awesome-vector-database , https://github.com/mileszim/awesome-vector-database https://github.com/mileszim/awesome-vector-database , https://github.com/currentslab/awesome-vector-search https://github.com/currentslab/awesome-vector-search /? "vector database" "duckdb" https://www.google.com/search?q=+%22vector+database%22+%22duckdb%22 https://www.google.com/search?q=+%22vector+database%22+%22du... ... pgvector pgvector/pgvector/src/vector.c: vector_spherical_distance https://github.com/pgvector/pgvector/blob/master/src/vector.c#L694C1 https://github.com/pgvector/pgvector/blob/master/src/vector.... postgresml/postgresml: /? distance https://github.com/search?q=repo%3Apostgresml%2Fpostgresml%20distance&type=code https://github.com/search?q=repo%3Apostgresml%2Fpostgresml%2...