5 ms·
Yes, that's how it's done. A dense (bitmap) representation of matrix wouldn't fit in memory, would require about a PB of RAM unless my napkin math is off. The
by marginalia_nu 3y ago
Yes, that's how it's done.
A dense (bitmap) representation of matrix wouldn't fit in memory, would require about a PB of RAM unless my napkin math is off. The cardinality of this dataset is in the 100s of millions.
(An additional detail is I'm actually using a tiny fixed width bloom filter to make it go even faster)
- socksy 3y agoI'm curious if you had a look at GraphBLAS (ala Redis Graph etc)?
- marginalia_nu 3y agoNope, did try a few off the shelf approaches, but ultimately figured it was easier to build something myself.