4 ms·

You may be interested in bitfields and JS optimised vector operations, the @thi.ng universe has some useful implementations (also a WASM simd one): [0]: https

You may be interested in bitfields and JS optimised vector operations, the @thi.ng universe has some useful implementations (also a WASM simd one): [0]: https://github.com/thi-ng/umbrella/tree/develop/packages/vectors#basic-usage https://github.com/thi-ng/umbrella/tree/develop/packages/vec... [1]: https://github.com/thi-ng/umbrella/tree/develop/packages/vector-pools#api https://github.com/thi-ng/umbrella/tree/develop/packages/vec... [2]: https://github.com/thi-ng/umbrella/tree/develop/packages/bitfield#api https://github.com/thi-ng/umbrella/tree/develop/packages/bit... [3]: https://github.com/thi-ng/umbrella/tree/develop/packages/simd#api https://github.com/thi-ng/umbrella/tree/develop/packages/sim... There's also Structura, which implements a few performant data structures: [4]: https://github.com/zandaqo/structurae/blob/master/README.md#bitarray https://github.com/zandaqo/structurae/blob/master/README.md#...

dleeftink 2y ago
I also did some experimenting with number packing and ended up creating a QuickSet implementation[0]. However, it turned out that operating on TypedArrays proved more performant, which I settled on in the end. I've collected some related packages here: [0]: https://github.com/dleeftink/QuickSet https://github.com/dleeftink/QuickSet [1]: https://github.com/dleeftink/QuickSet?tab=readme-ov-file#see-also https://github.com/dleeftink/QuickSet?tab=readme-ov-file#see... Of note is FastIntSet, which uses the technique you described, but I think is only able to store 4 unsigned integers as one JS value (I might be wrong). [2]: https://github.com/TheLucifurry/fast-int-set/blob/main/src/sets/uintSet.ts https://github.com/TheLucifurry/fast-int-set/blob/main/src/s... This notebook demonstrate a MVP bitpacking technique using BigInts (see the 'bitpack' and 'pack' cells): [3]: https://observablehq.com/@dleeftink/array-bitbuffers https://observablehq.com/@dleeftink/array-bitbuffers SimSIMD offers various similarity measures and quantisation levels for TypedArrays: [4]: https://github.com/ashvardanian/simsimd#using-simsimd-in-javascript https://github.com/ashvardanian/simsimd#using-simsimd-in-jav...

dleeftink 2y ago

I also did some experimenting with number packing and ended up creating a QuickSet implementation[0]. However, it turned out that operating on TypedArrays proved more performant, which I settled on in the end. I've collected some related packages here: [0]: https://github.com/dleeftink/QuickSet https://github.com/dleeftink/QuickSet [1]: https://github.com/dleeftink/QuickSet?tab=readme-ov-file#see-also https://github.com/dleeftink/QuickSet?tab=readme-ov-file#see... Of note is FastIntSet, which uses the technique you described, but I think is only able to store 4 unsigned integers as one JS value (I might be wrong). [2]: https://github.com/TheLucifurry/fast-int-set/blob/main/src/sets/uintSet.ts https://github.com/TheLucifurry/fast-int-set/blob/main/src/s... This notebook demonstrate a MVP bitpacking technique using BigInts (see the 'bitpack' and 'pack' cells): [3]: https://observablehq.com/@dleeftink/array-bitbuffers https://observablehq.com/@dleeftink/array-bitbuffers SimSIMD offers various similarity measures and quantisation levels for TypedArrays: [4]: https://github.com/ashvardanian/simsimd#using-simsimd-in-javascript https://github.com/ashvardanian/simsimd#using-simsimd-in-jav...