4 ms·
I've run up against this building a gigantic LZ dictionary for full text search[0] and I got around it[1] by using npm module called big associative that provid
by graderjs 5y ago
I've run up against this building a gigantic LZ dictionary for full text search[0] and I got around it[1] by using npm module called big associative that provides dropin big map and big set primitives.
but even in that code I think there was some error or inefficiency that I had to customize to make it usable.
But in the end I was able to get around the 16 million limit on entries and good efficiency.
[0]: https://github.com/i5ik/futzz https://github.com/i5ik/futzz
[1]: https://github.com/i5ik/futzz/blob/master/src/futzz.js#L3 https://github.com/i5ik/futzz/blob/master/src/futzz.js#L3
- sillysaurusx 5y agoDo you have any details on your customizations? That’s interesting.
- graderjs 5y agoLet me take a look and double check that. It might have been a prior library, or maybe I did modify big-associative. I'll investigate. Be back here later edit: So I looked and I think I didn't modify big-associative...could have been a library I used before I found that one. Here's big-associative's repo: https://github.com/samchon/big-associative https://github.com/samchon/big-associative
- tonto 5y agoauthor of blogpost here. interestingly the actual thing (not mentioned in blog post) that I was developing was for text searching too...resulting library I made is here https://github.com/GMOD/ixixx-js/ https://github.com/GMOD/ixixx-js/
- graderjs 5y agoCool, man. I just forked it! Happy to take a look inside...figure out how it works; use the best stuff for mine...kidding. But I ran into issues with using LZ, but also was using a trie for the index structure (or on way to).