3 ms·

This is where the power and expressiveness of kdb+ shines. It has SIMD primitives out of the box and can optimize your code based on data types to take advantag

This is where the power and expressiveness of kdb+ shines. It has SIMD primitives out of the box and can optimize your code based on data types to take advantage of it. https://kx.com/blog/what-makes-time-series-database-kdb-so-fast/ https://kx.com/blog/what-makes-time-series-database-kdb-so-f...

MangoToupe 1y ago
Time series is vector processing on easy mode, though. The hard part is applying SIMD to problems that aren't shaped to be easily processed in parallel.
- jgalt212 1y ago
  Fine. What's canonical or most basic example of where SIMD should be applied, but isn't because it's too tricky to do so? In our shop, we never look to vectorize any function or process unless it's called inside a loop many times.
  - MangoToupe 1y ago
    > What's canonical or most basic example of where SIMD should be applied, but isn't because it's too tricky to do so? There is none. That's a contradiction in terms. SIMD either fits the shape or it doesn't.
    - krapht 1y ago
      Variable length parallelism is hard. You can go to highload.fun (SIMD competition site) for problems that are only parallelized after significant effort. Try problem #1, parsing numbers.
  - ashvardanian 1y ago
    Text processing. Loading/branching/storing content 1 byte at a time is the CPUs worst nightmare, but most text processing is quite tricky in SIMD.
    - MangoToupe 1y ago
      > where SIMD should be applied That seems to disqualify your example
      - AlotOfReading 1y ago
        There can be huge advantages to text processing in SIMD if you figure it out. Example, simdjson: https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson
        ashvardanian 1y ago
        Maybe even better examples would be <https://github.com/intel/hyperscan https://github.com/intel/hyperscan>, <https://github.com/simdutf/simdutf https://github.com/simdutf/simdutf>, and my own <https://github.com/ashvardanian/StringZilla https://github.com/ashvardanian/StringZilla> :)

MangoToupe 1y ago

Time series is vector processing on easy mode, though. The hard part is applying SIMD to problems that aren't shaped to be easily processed in parallel.

jgalt212 1y ago
Fine. What's canonical or most basic example of where SIMD should be applied, but isn't because it's too tricky to do so? In our shop, we never look to vectorize any function or process unless it's called inside a loop many times.
- MangoToupe 1y ago
  > What's canonical or most basic example of where SIMD should be applied, but isn't because it's too tricky to do so? There is none. That's a contradiction in terms. SIMD either fits the shape or it doesn't.
  - krapht 1y ago
    Variable length parallelism is hard. You can go to highload.fun (SIMD competition site) for problems that are only parallelized after significant effort. Try problem #1, parsing numbers.
- ashvardanian 1y ago
  Text processing. Loading/branching/storing content 1 byte at a time is the CPUs worst nightmare, but most text processing is quite tricky in SIMD.
  - MangoToupe 1y ago
    > where SIMD should be applied That seems to disqualify your example
    - AlotOfReading 1y ago
      There can be huge advantages to text processing in SIMD if you figure it out. Example, simdjson: https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson
      - ashvardanian 1y ago
        Maybe even better examples would be <https://github.com/intel/hyperscan https://github.com/intel/hyperscan>, <https://github.com/simdutf/simdutf https://github.com/simdutf/simdutf>, and my own <https://github.com/ashvardanian/StringZilla https://github.com/ashvardanian/StringZilla> :)