4 ms·
Yes, it is a port of that library with some improvements.
by heywhy 5y ago
Yes, it is a port of that library with some improvements.
- dnautics 5y agoYou might want to consider mapping an index to an ets table-based data structure instead of an immutable object managed by a GenServer, it will give you a way to share it between processes without having to awkwardly copy a potentially huge data structure all over the place.
- heywhy 5y agoI do have thoughts about performance too but I was following the "get it working then make improvements" route :). Thank you for the suggestions.
- dnautics 5y agoLove it. You're doing exactly the right thing.
- kuzee 5y agoThis makes sense, and I think you've taken the correct route. I look forward to trying this in one of my projects and comparing to my current postgres-only backed search strategy. For my use case losing the index between restarts isn't a deal breaker, so hopefully I'll have some useful feedback.
- heywhy 5y agoThat's great. I will be looking forward to this.
- linkdd 5y agoI'd say even using mnesia as an option for disc copies.
- dnautics 5y agomnesia had very difficult to debug consistency issues that can crop up. Have these been fixed?
- skrebbel 5y agoI don't understand how this works. Is data read from ETS somehow shared more efficiently than data shared via a regular message? (which iirc is always copied)
- eproxus 5y agoIt’s still copied, but a process can quickly become a bottleneck in parallel code (every request to a process is sequential). An ETS table can be concurrently read (and tweaked even further for that use case if desired).
- heywhy 5y agoLike eproxus mentioned, it's still been shared through normal process messaging but improvements will be made regarding this.
- dnautics 5y agoIt's still copied but if you are using an ets table you're likely only copying a small subset of the data per query instead of schlepping the whole index every time.