5 ms·

I wrote that module as well :) The secondary index module will be more focused on automating more traditional indexing of numbers and simple string values. It

by dvirsky 10y ago

I wrote that module as well :) The secondary index module will be more focused on automating more traditional indexing of numbers and simple string values. It has an SQL-ish language for WHERE expressions, and then the result of that is piped to any redis command you want it to. If you're interested, here's a draft of the syntax (it's changed a bit since but you'll get the idea). https://gist.github.com/dvirsky/3ef73143a6d8212f2b50096a8eb68018 https://gist.github.com/dvirsky/3ef73143a6d8212f2b50096a8eb6...

ddorian43 10y ago
Oh, I should've seen the usernames. I'm not interested on the secondary-indexing, mostly on the full-text-search. It could grow on something big (compared to the benchmarks on redislabs https://redislabs.com/blog/adding-a-search-engine-to-redis-adventures-in-module-land https://redislabs.com/blog/adding-a-search-engine-to-redis-a...)
- dvirsky 10y ago
  Yeah, that blog post is also me :) hehe. Anyway glad to hear you find it interesting. If you want to help out ping me. There's tons of stuff on the roadmap, and currently I'm almost entirely focused on the secondary index.
  - ddorian43 10y ago
    I still see some issues with it: 1. redis being single-thread, where inverted-indexes are ~easily sharable by cores (i think based on what I've read). This making hotspots easier(since it's a single core not single machine) 2. sharing of data(terms) between multiple indexes on 1 server (like in elasticsearch you use _routing, but all things are in 1 index, though some people like separate-per-user like dropbox does) 3. cluster not fully nice yet (losing writes) 4. no option to merge from different nodes (or even redis-api to do so as far as I know)
    - dvirsky 10y ago
      Salvatore just added support for asynchronous operations in side threads in modules. This would allow merging results from the cluster possible, and I want to get to it soon. I'm now implementing it in the client side and it works great, but I want it to be as transparent as in elastic. The recent additions would also allow more parallelization of the indexes for reads. I could create RW mutexes in the engine and allow multiple clients to work on the same term index. It won't be trivial but it's not super hard as well.
    - dvirsky 10y ago
      Anyway, if you want to continue the discussion further, we can take it to reddit (I haven't posted the benchmarks there yet, it's a good idea anyway), or just email me, dvir at redis labs.

by dvirsky 10y ago

ddorian43 10y ago

Oh, I should've seen the usernames. I'm not interested on the secondary-indexing, mostly on the full-text-search. It could grow on something big (compared to the benchmarks on redislabs https://redislabs.com/blog/adding-a-search-engine-to-redis-adventures-in-module-land https://redislabs.com/blog/adding-a-search-engine-to-redis-a...)

dvirsky 10y ago
Yeah, that blog post is also me :) hehe. Anyway glad to hear you find it interesting. If you want to help out ping me. There's tons of stuff on the roadmap, and currently I'm almost entirely focused on the secondary index.
- ddorian43 10y ago
  I still see some issues with it: 1. redis being single-thread, where inverted-indexes are ~easily sharable by cores (i think based on what I've read). This making hotspots easier(since it's a single core not single machine) 2. sharing of data(terms) between multiple indexes on 1 server (like in elasticsearch you use _routing, but all things are in 1 index, though some people like separate-per-user like dropbox does) 3. cluster not fully nice yet (losing writes) 4. no option to merge from different nodes (or even redis-api to do so as far as I know)
  - dvirsky 10y ago
    Salvatore just added support for asynchronous operations in side threads in modules. This would allow merging results from the cluster possible, and I want to get to it soon. I'm now implementing it in the client side and it works great, but I want it to be as transparent as in elastic. The recent additions would also allow more parallelization of the indexes for reads. I could create RW mutexes in the engine and allow multiple clients to work on the same term index. It won't be trivial but it's not super hard as well.
  - dvirsky 10y ago
    Anyway, if you want to continue the discussion further, we can take it to reddit (I haven't posted the benchmarks there yet, it's a good idea anyway), or just email me, dvir at redis labs.