6 ms·
Redis Loadable Modules System
- therein 10y agoSounds exciting. Hopefully it won't be as under-appreciated as Apache Traffic Server's plugin system.
- aaw 10y agoThis looks interesting. I've only glanced at the docs, but it looks like this module system lets you define new commands but not necessarily new data structures (since you'd have to implement RDB/AOF serialization for those). Are there plans for that? It's probably possible to do most of that with this module system but you'd have to serialize the data structure into a string and "type X" wouldn't return your custom data structure type.
- antirez 10y agoDefinitely plans to register new data types with serialize/deserialize callbacks! Thanks.
- deleted 10y ago[deleted]
- maerF0x0 10y agoHe mentioned in the keynote being able to do your own datastructures by raw access to the data and primitives like seeks and next/prev . Sounds like it's at least planned, if not roughed in.
- astrodust 10y agoThere's been a lot of unusual features I've been hoping Redis will support but obviously given their edge-case nature it's unlikely that they'd ever be implemented. This is great to see. Instead of cluttering up the core engine with junk people can mix in their own functionality.
- maerF0x0 10y agothey also announced a couple of "hubs" for hosting/discovering these plugins, plus a vetted one by redislabs
- itamarhaber 10y agoIndeed - http://modules.redis.io http://modules.redis.io will be the open source place and the Redis Labs' Module Hub is at https://redismodules.com https://redismodules.com
- MBCook 10y agoI tried to read the article but I couldn't. It's all one big <pre> block. The word wrapping is wrong on input! Individual paragraphs are groups of individual lines and extra long lines that the browser wraps for you. So you end up with lines in the middle of paragraphs that only have one or two words. Net result is it's very hard to read. I tried Safari's reader mode, but that was even worse. Because all formatting comes from happenstance in the <pre> tag reader mode loses all formatting. The entire article (and code!) becomes one long block of text. I'm continually amazed at people's attempts to reinvent text layout in the browser.
- incepted 10y ago> I'm continually amazed at people's attempts to reinvent text layout in the browser. Agreed. I don't recommend you try to read pg's essays, they will make you very angry.
- manigandham 10y agoWhy is this downvoted? It's true that this blog has bad formatting considering how simple it is, there's no reason to use <pre> for the entire text. Just because the author created Redis doesn't mean the site isn't badly designed.
- MBCook 10y agoI make these kind of comments every once in a while. They're usually downvoted, I assume, for being off-topic or unnecessarily harsh ('We might have cured cancer!' vs 'I can't read the article because of the video behind the text.'). But I hope some people see them and work on the issue. Sometimes, if enough other people agree, these kind of comments get voted up. I guess this issue didn't annoy most people enough to get the up votes.
- shawn-butler 10y agoHow people can code on airplanes amazes me. I have tried but the cramped space, the constant interruptions.. I never get much done.
- guelo 10y agoAgree, I sometimes have problems even fully opening my laptop.
- noir-york 10y agoIt's not the constant interruptions (headphones do a good job of dealing with that), its the lack of internet access that is the showstopper - I don't have all the APIs + docs in my head.
- swaroop 10y agoUse https://kapeli.com/dash https://kapeli.com/dash on Mac or the open source equivalents on other platforms.
- rajivm 10y agoI use https://kapeli.com/dash https://kapeli.com/dash for offline documentation for when I'm on flights. Also, WiFi has been available on more and more flights. I actually have found long flights good for solid focused coding sessions (especially when I already have thought through what I want to build). edit: woops, looks like someone submitted this at the same time. In any case, working offline also forces you to reign in your app dependencies.
- avar 10y agoSince Redis is BSD licensed I wouldn't be surprised if this is the start of popular paid-for proprietary third party modules.
- byroot 10y agoEven if redis was GPL licensed you still could do that. Linux is GPL and there is proprietary driver modules.
- ploxiln 10y agoThere are proprietary modules for linux, and they are distributed and used, but there isn't really a full consensus on whether they are legal. You can certainly do it and get away with it though. http://yarchive.net/comp/linux/gpl_modules.html http://yarchive.net/comp/linux/gpl_modules.html
- angelbob 10y agoI'm hoping there's an easy way to turn this off for the foreseeable future, to avoid all the obvious security holes this is likely to introduce.
- dvirsky 10y agoYou don't have to use it, and you should not use modules you don't trust.
- angelbob 10y agoIn an ops-type capacity, I support developers. I keep up with news like this specifically to avoid winding up with modules somebody else trusts, but didn't inspect.
- dvirsky 10y agoWhat would be a must and should probably be a default, is not allowing to load modules using the network API, only via config file.
- akbar501 10y ago@antirez, is Redis 4.0 going to be part of the more rapid release schedule (will we see it in 3, 6 months) or are major upgrades still annual releases (we'll see 4.0 in 12 months)?
- antirez 10y agoDefinitely part of the short release cycle, we'll probably focus on getting modules done and the features already pending into "unstable", and a few more minor stuff, and release it.
- halayli 10y agoI bet a package manager will follow it. redis install moduleX and then we'll have lib and target dependencies.
- spullara 10y agoI once built something like this for Redis but never really thought he would do it! https://github.com/spullara/redis/tree/plugins https://github.com/spullara/redis/tree/plugins
- BadgerEmeritus 10y agoThis is great and long awaited, please consider some kind of a namespacing scheme for module-defined commands though, or things can easily become messy. In the module example the "hello." prefix is specified for all defined commands - it would probably be better to specify it once on module initialization and have all commands defined by the module prefixed with it automatically, the module registries could then manage the prefix registration/allocation to module authors.
- educar 10y agoThe docs explain the module system (https://github.com/antirez/redis/blob/unstable/src/modules/INTRO.md https://github.com/antirez/redis/blob/unstable/src/modules/I...) but it doesn't quiet explain what the system is for. I couldn't quite get that from reading the blog post or the doc page. Specifically: What kind of modules did you have in mind when designing this? Are any of the below possible? * ldap backend login system * multi-tenant redis * alternate db format/file system. * implement a new redis data type. say built-in date support * can i implement new commands and operators on said data types An of course, thanks for this fantastic software. We use it everyday. Will read in more detail about this over the weekend :-)
- dvirsky 10y agoActually the post explains the motivation well. The basic idea is to add capabilities to redis at native speeds, without having to either convince antirez that they're necessary or forking redis. I work for Redis Labs, and over the course of the past few weeks we've experimented with some modules using the API. Some of the things we did that might answer some of your questions: a PAM based authentication module, which IIRC can plug into LDAP potentially. A bloom filter implementation on top of bitmaps. An inverted index that doesn't use existing data types but rather creates compressed indexes directly using redis strings (I wrote that one personally BTW) An API to store images on redis and retrieve them resized or cropped. So right now the only way to create new data types is to piggyback strings as raw memory, but the final version of the API will include actually registering new data types. You can see the modules we've released here: https://github.com/RedisLabsModules/ https://github.com/RedisLabsModules/
- creshal 10y ago> a PAM based authentication module, which IIRC can plug into LDAP potentially. Yes, pam-ldap / pam-sssd / pam-winbind plug into various LDAP implementations (plain LDAP / ActiveDirectory) and work transparently with all PAM clients.
- vkjv 10y agoMaybe you can relay the information to the appropriate person, but on http://redismodules.com http://redismodules.com, I can neither right click "open in new tab" or ctrl+click to open in a new tab. This breaks my normal browsing habit of opening new tabs as I see something that interests me, but waiting until I finish skimming the list to read.
- derefr 10y agoIf these modules are going to be as fast as the native commands, is there any reason to not lift the entire native command set out into a set of "base" modules?
- dvirsky 10y agoThey're not going to be as fast, but almost as fast. You could do that in theory but IMHO there's not much point
- awinter-py 10y agodatabases (redis included) are awesome. It's always surprising to me that we don't run web apps directly on the database; they're high-performance, durable and parallel, and for some apps are already the perf bottleneck. I think the only reason we don't is that deployment and testing isn't straightforward. Fingers crossed antirez will get this right.
- TheAceOfHearts 10y agoYou might find CouchDB pretty interesting, then! [0] [0] https://couchdb.apache.org/ https://couchdb.apache.org/
- awinter-py 10y agoI know them for data sync. is there a stored proc deployment feature as well?
- manigandham 10y agohttps://github.com/RedisLabsModules/redex https://github.com/RedisLabsModules/redex Shows some sample modules from Redis Labs called RedisEx, pretty useful stuff (although most of these should really be in Redis by now but at least it's an option to include as modules).
- dvirsky 10y ago> most of these should really be in Redis by now This exactly the point of the module system. You don't need to either convince antirez to accept/implement a new feature, or fork redis if he doesn't agree. It will make the dynamics of redis much more interesting IMO
- evantahler 10y agoSo who is adding node.js support?!
- lsiebert 10y agoSuper excited to hear about this at Redis Conf, looking forward to implementing a few ideas.
- MasterScrat 10y agoSomeone should really implement an interval module. This custom implementation was pretty good: http://blog.togo.io/how-to/adding-interval-sets-to-redis/ http://blog.togo.io/how-to/adding-interval-sets-to-redis/