11 ms·
Redis as a JSON store
- colinbartlett 10y agoThe page renders as 100% black and totally blank with uBlock Origin for me. Not complaining as much as pointing it out in hopes the author might see this and adjust to prevent people from bouncing.
- dvirsky 10y agoFor anyone else interested but having rendering problems: A talk by the author on youtube: https://www.youtube.com/watch?v=NLRbq2FtcIk https://www.youtube.com/watch?v=NLRbq2FtcIk The ReJSON website: https://redislabsmodules.github.io/rejson/ https://redislabsmodules.github.io/rejson/
- icebraining 10y agoNo problems here; 7 requests blocked, but page still rendered fine.
- chmike 10y agoSame here. I'm using the default ublock origin configuration. Maybe it's specific rules or not due to ublock origin.
- Ygg2 10y agoWith uMatrix, it looks fine. You need to allow Redislab.
- itamarhaber 10y agoSorry, can't reproduce.
- tehbeard 10y agoRunning uBlock Origin here and it's loading fine in firefox and chrome. I have a default install, if that helps debug why it's rendering blank for you.
- nodja 10y agoSame here. Fanboy’s Annoyance List seems to be one triggering it. Instead of disable all of ublock you can toggle the cosmetic filtering when you click on the ublock icon. http://i.imgur.com/Ne9mHBd.png http://i.imgur.com/Ne9mHBd.png
- floatboth 10y agoWhat's the indexing story? Do it yourself? "Any JSON store implementation will contain a half-specified bug-ridden implementation of half of Postgres's GIN/GiST for jsonb"? :D
- dvirsky 10y agoWe're also developing a secondary indexing module (https://github.com/RedisLabsModules/secondary https://github.com/RedisLabsModules/secondary) and a search module (https://github.com/RedisLabsModules/RediSearch https://github.com/RedisLabsModules/RediSearch), the direction is to integrate one of them into the JSON module.
- traviscj 10y agoWise words -- a good set of indices are the first thing you need for real problems with realistic data sizes.
- dvirsky 10y agoRight now the problem that ReJSON tries to solve is very specific and typical for Redis users: You save JSON blobs on Redis (usually session data), now you want to get just a part of a JSON key, or manipulate just one element in a specific key. Until now you couldn't do it without loading the entire object into memory. ReJSON just allows you to manipulate that object or retrieve just a part of it efficiently. For this use case we do not need and will never need indexing.
- detaro 10y agoBy that logic, redis in general would not be useful for "real problems", whatever that is.
- weego 10y agoIt's Redis...
- gabriele 10y agoThis is a welcome addition. I expected JSON to become a first class citizens in REDISland for quite some time. In fact, REDIS has always been my goto store for JSON data. In 2013 I created a microservice[0] for accessing JSON data stored in native REDIS lists[1] via HTTP(S). [0]https://github.com/boilerplatecode/redis-wsgi https://github.com/boilerplatecode/redis-wsgi [1]https://redis.io/topics/data-types#lists https://redis.io/topics/data-types#lists
- jsemrau 10y agoWe use Redis Queue (RQ) as a listener cache at the Twitter firehose. Currently it is processing 300,000 tweets in JSON format per day and queuing them for further processing. Lovely piece of software (YTD experience ~ 2 months production environment usage)
- macca321 10y ago>Since there does not exist a standard for path syntax, ReJSON implements its own. Well there's this: JavaScript Object Notation (JSON) Pointer RFC https://tools.ietf.org/html/rfc6901 https://tools.ietf.org/html/rfc6901
- grenoire 10y agoLooks like ReJSON just uses periods instead of slashes as the delimiter, except for also using a period for the root object.
- dvirsky 10y agoWe wanted to start with something simple and intuitive, and since we're not doing indexing yet, we figured complex multi-selectors are an overkill at this stage. What we chose is basically the most rudimentary subset of JsonPath. Once we'll add indexing I'm guessing the path expressions will need to become more complicated, and we'll try to choose a "standard" and stick with it.
- itamarhaber 10y agoThank you, I didn't know about that one... good thing HN has comments :) Do you know if anybody is actually using RFC6901?
- lucian1900 10y ago
- chmike 10y agoThe graphic type use to show performance is inappropriate because line slopes have no significant meaning. A bar graph would be more appropriate.
- gtrubetskoy 10y agoFor the life of me I do not understand the JSON obsession. When I was young and naive, JSON indeed looked cool because it is both human and computer readable. Until you run into some of its limitations. For example, AFAIK there is still no agreement on how to handle binary data, e.g. what do you do with: >>> json.dumps(["\xb8\xc3\xb6\xbb"]) Traceback (most recent call last): ... UnicodeDecodeError: 'utf8' codec can't decode byte 0xb8 in position 0: invalid start byte You can base64-encode it or do a number of other things, but then that's outside the JSON standard, and suddenly it isn't the flexible encoding standard it seemed at first...
- maxxxxx 10y agoTotally agree. XML is probably over engineered but JSON is under engineered. No date type, no binaries, no comments, no easy append to existing files. We jumped from one bad standard format to another bad standard format. Other than maybe being a little easier on the eye I don't see how JSON is an improvement over other formats.
- onli 10y agoWhy would you need any of that? A date type is bloat, store an epoch number. Comments have no place in a data format. Append is exactly the same as with XML: You take the part you want and add it at the right place in a list of the other file. I'd argue the lack of namespaces makes it easier than with XML. For the binary data: JSON is not a binary format, one should not try to mix those. To encode it and store it as a string is exactly right. Otherwise a database as blob store is the better place for that.
- maxxxxx 10y agoComments are important ina format that's being used for configuration as JSON is. Commenting out things makes for easier revert if something doesn't work. Having a Line of comment in front of each setting is also a good thing. This is just basic practicality. You can't easily append something to an already existing JSON file. You have to decode the whole thing first and then write it back whole. XML has the same problem. That's why I said we replaced one bad format with another. JSON is not an improvement over XML. It replaced one set of weaknesses with another.
- pfooti 10y agoWow, this looks rad. I'm just now working on a data storage module for my hobby project, and have been pushing strings through JSON.parse to store data in redis. Being able to directly address values deep in the hash is great - when setting foo.bar to 'baz', using a path selector instead of reading, deserializing, modifying, reserializing, and writing is great.
- rizach 10y agoShouldn't storing a json as a string be better performance for redis? All the extra jobb to find subkeys put extra work for redis. While pulling out the str and parse it in application would minimize the time spent in redis. This gets more true as the number of applications against one redis grow.
- dvirsky 10y agoIf you're serializing and saving the entire object always - you don't need this extension. But if you're storing say, a 1K user profile, and most of the time you just want to update the access time, or get some auth token - this both allows atomicity and speeds up what you are doing. The bigger the JSON object is vs. the how small a piece you want to retrieve or manipulate - the more efficient this becomes. Also, manipulating a field in a JSON object atomically is currently possible only with Lua scripts - and even then, Lua will have to parse, manipulate and serialize the entire object. Where as in this case, if you are just updating something - there is zero parsing and zero serialization going on in the module.
- rizach 10y agoShouldn't storing a json as a string be better performance for redis? All the extra jobb to find subkeys put extra work for redis. While pulling out the str and parse it in application would minimize the time spent in redis. This gets more true as the number of applications against one redis grow.
- deleted 10y ago[deleted]
- salmonfamine 10y agoI really could have used this for my last side project. Moving JSON blobs between Python and Redis is pretty error-prone, especially if those blobs contain nested structures.
- aaroninsf 10y agoNaive question: does anyone know enough about this or modules generally to tell me whether ReJSON values are manipulable from Redis Lua? That is, I have some application support from Lua scripts manipulating native types today. But if I can read and write into JSON stored this way from Lua, that might be very handy!
- dvirsky 10y agoYou can use modules from Lua if they are loaded, there is not difference between them and native stuff. You can certainly manipulate ReJSON objects from Lua, parse ReJSON returned objects from Lua, etc. No limitations as long as the module is loaded.
- aaroninsf 10y agoGreat, that is promising... thank you for the clarification!
- gregmac 10y agohttps://redislabsmodules.github.io/rejson/ https://redislabsmodules.github.io/rejson/ > Windows > Yeah, right :) How about just "There are no plans to build a Windows version"?
- wtf_is_up 10y agoHow about not policing someone else's expression?
- rolandnsharp 10y agoI've been doing some pretty heavy caching of my node application with Redis and have just been using JSON.parse and JSON.stringify to cache and retrieve JSON. Is what I'm currently doing bad? what is the main advantage of using this module? Performance?