7 ms·
RethinkDB 1.8: date support, querying nested objects, 8x disk usage improvements
- jonny_eh 13y agoAnother great update. Keep it going!
- leetrout 13y agoKudos to the team; some great releases this summer! (10x insert performance, regex queries, and now date/time and better disk space--all in ~90 days!) I had never looked in to how rdb used disk space and off the cuff attributed its high usage to something similar to Mongo and allocating blocks ahead of usage. This was something that went in to the negative column for me when I evaluated using rdb vs Postgres for a JSON dump. 12k somewhat light (but still larger than 250b) records ate up over 70MB of disk space in just over a couple days and I've not looked back. Time to give it another spin!
- coffeemug 13y agoThere is no technical reason why Rethink needs to have high disk usage. We got much better with this release, and will continue to get even better in the long term. (Squeezing more optimizations does get a bit harder, though)
- continuations 13y agoHow does rethinkDB's disk usage compare to MySQL and Psotgres?
- yogo 13y agoThis is good to hear and will make me try it too. High disk space utilization was something I experienced with mongodb back in the day and that caused me to lose interest (not sure if they're better about it these days). There's probably a trade-off between speed and having blocks allocated but it starts becoming a problem on smaller vps environments.
- werkshy 13y agoMongo got a lot better about reusing disk space in v2.2, if you are deleting from a collection. Still no compression or document field name optimizations though.
- leif 13y agoCouldn't resist: try tokumx if you want "mongodb with compression". There isn't a speed/space usage tradeoff. Our latest blog post is about long field names: http://www.tokutek.com/2013/08/tokumx-tip-create-any-field-name-you-want/ http://www.tokutek.com/2013/08/tokumx-tip-create-any-field-n...
- yogo 13y agoInteresting product. They also have a mariadb engine (different use cases).
- xfour 13y agoWell done guys,I'm especially happy for the Date Object! That was a easy but annoying thing to have to work-around.
- stephanos2k 13y agoCan someone who ditched MongoDB for RethinkDB share his/her experiences?
- xfour 13y agoI did a small project so maybe someone that did something larger can weigh in, but I worked with RethinkDB back in the 1.4 era, so a bit has changed, but the biggest difference IMO is that you have to be prepared to fall back to the RQL syntax to do queries more complex than simple get etc. vs. in Mongo everything was in JSON, and when I was using Mogno you couldn't join tables at all, don't know if that's changed and here you can. Otherwise it is pretty similar, since I didn't do any clustering.
- coffeemug 13y agoHere's a good explanation for why you want to fall back to ReQL rather than write queries in JSON directly: http://rethinkdb.com/docs/introduction-to-reql/ http://rethinkdb.com/docs/introduction-to-reql/
- embwbam 13y agoReQL for the win. It's so expressive, it's composable, I love it. I feel more confident that the m will be easy to scale if/when my app needs it. My code that interacts with the API feels cleaner.
- dodyg 13y agoThey fixed my bug - yay.
- andreypopp 13y agoBeing a heavy PostgreSQL user I really would appreciate a "How RethinkDB compares to PostgreSQL" FAQ entry.
- mglukhovsky 13y agoThat's a great idea, we'll be adding more comparisons in the future-- thanks for the suggestion!
- old-gregg 13y agoMike, I would love to see that too. Can you please include the entry on connection management model. PSQL is known for taking process-per-connection + shared memory approach, knowing the canonical and "mothership approved" approach to handle Rethink connections would be awesome. Thanks!
- coffeemug 13y agoThanks, that in the pipeline. FYI, it's quite similar in Rethink -- a thread/process per connection is the best way to access the db.
- hcarvalhoalves 13y agoDate/time was a hard one but worth the effort. Looking forward to the next updates!
- mapmeld 13y agoI keep thinking that I should try RethinkDB, but most of my projects involve geodata / geospatial indexing. For this use-case PostGIS and MongoDB have benefits which I can't find elsewhere.
- philippelh 13y agoSame thing for me!
- coffeemug 13y agoWe have a github issue open for it FYI (can't find it now because github issue search seems to be down), but unfortunately it won't get done before 2.0.
- e12e 13y agomaybe: https://github.com/rethinkdb/rethinkdb/issues/1158 https://github.com/rethinkdb/rethinkdb/issues/1158 ?
- abc_lisper 13y agoNice. Can you guys please put a wiki page now?
- mglukhovsky 13y agoDo you mean a Wikipedia article? If you mean a wiki for users, we use the Github wiki (https://github.com/rethinkdb/rethinkdb/wiki https://github.com/rethinkdb/rethinkdb/wiki) to track things like community contributions.
- abc_lisper 13y agoYeah, a wikipedia article. That is where I go to first, when I hear about a new technology. I know I am not the only one.
- dkhenry 13y agoGreat Job and this is cool to see such progress. Now the long process of implementing all the new verbs in the drivers begins. I am really looking forwards to when RQL is marked as stable.
- coffeemug 13y agoActually, ReQL is basically stable as of 1.8. There may be very minor changes before 2.0, but there are no big changes planned any more.
- ddorian43 13y agoSo, is there too much overhead for each table (to create per-client tables since custom_field sharding is not yet available) ?
- coffeemug 13y agoTables are a little bit heavy now. I don't know the exact overhead (we should measure), but a table is certainly heavier than a single document. I'm not sure if a table/client is tenable -- if end up doing some tests could you please report the results?
- dkhenry 13y agoSo no new verbs going into the Query Language ? I have been waiting to do much more work around my Java driver until the pace of change slows down a bit.
- andrewflnr 13y agoLook, I don't have enough experience to be sure on this kind of thing, but it seems like RethinkDB is being developed at a blistering pace. Docker is another one like that. Both impress me, both on their vision and rate of improvement. It makes me happy to know that there are successful projects like this out there.
- sahrizv 13y agoYeah, same with the ember.js guys. Churning out releases and RCs under such restricted timelines.
- Orva 13y agoWhat.. version 1.8 and now there is Date support? Shouldn't this kind of stuff be aournd at version 0.01 when we are talking about data stores? You know, the only non-easily swappable piece of our tech stack?
- romland 13y agoIt's really comments like this that make me scared to death of posting a project to HN these days (so I don't). Seems to be so common now. It's getting quite common for me to just skip reading the user comments after reading the article. This was never the case before; the comments were -almost always- better than the article. As a sidenote, I cannot even recall when I actually read the user comments on Slashdot. On the occasion where something pops up in the RSS feed; read TFA and move on to happier things that actually make me feel good. :) I guess I'm not saying anything new. I really wish I had a solution other than asking -you- to please imagine sitting at the receiving end: You get excited to see your project on HN, you click through to comments, and the first thing that was said was ... that. PS. You can call me a chicken for being "afraid" of these things.
- coffeemug 13y agoSometimes people get angry and irritated at stuff. That's ok. The tone is uncalled for, but irritated/angry comments often have legitimate gripes. Rethink should have had dates earlier, we just couldn't get it done. I agree that people should try and be more kind with their tone, but I wanted to try and encourage you not to be afraid of criticism. People have good memory for good work and no memory for bad work. So if you post an atrocious project here, the absolute worst thing that might happen is that some people will write some negative comments and forget about you ten minutes later. It seems like a worst-case scenario most people can live with :)
- coffeemug 13y agoIt should have been, but date support is way, way trickier than it appears. Check this out if you want to learn more -- https://gist.github.com/coffeemug/6168031 https://gist.github.com/coffeemug/6168031. It's bad that dates didn't get in earlier, but it would have been worse if we rushed a bad implementation. Sometimes one needs to pick the least of two evils.
- agildehaus 13y agoAre 2D geospatial indexes/queries planned for 2.0 or do they come later?
- neumino 13y agoIt probably won't get done before 2.0, but I wouldn't be surprised if it will be one of the first features once we get production ready.
- wiremine 13y agoI've been watching the great progress since 1.4, but I continue to be disappointed by the overall performance, makes me wonder if I'm not groking something. Here's a gist of a simple ruby script to insert 1000 trivial docs into rethink: https://gist.github.com/wiremine/6245910 https://gist.github.com/wiremine/6245910 1,000 inserts are taking around 20 seconds. Hardware is a 1.8 Core i5 Mac Air with 4GB Ram and a 128 SSD. Is this sort of performance expected at this point?
- neumino 13y agoBatching inserts make them faster https://gist.github.com/wiremine/6245910#comment-887640 https://gist.github.com/wiremine/6245910#comment-887640 13.618112323 seconds vs 0.591895712 seconds (on my laptop -- X1 Carbon) Using soft durability make things faster too, but that's cheating.
- dkhenry 13y agoYou have something wrong there. I just ranthe test suite I ship with my java driver against and I am able to do 2048 documents in about a second. Most of that time is spend in serializing the data to go over the wire and each insert is being done individually and in raw protobuf form. If you send in JSON and you queued up your inserts I imagine you could do even better.
- coffeemug 13y agoThis sort of thing isn't expected. Please check the following troubleshooting entry for slow inserts: http://rethinkdb.com/docs/troubleshooting/#my-insert-queries-are-slow.-how-can-i-speed-t http://rethinkdb.com/docs/troubleshooting/#my-insert-queries...
- geedy 13y agoKeep up the great work. It's a very interesting project, one I'll continue to keep an eye on. One thing I wish would get added are more aggregators. This would add a lot of valuable use cases, in my opinion. At least one of my mini projects recently would have been greatly simplified had I had the ability to compute max's and min's in the same way as avg's.
- neumino 13y agoYou can currently compute max/min through map/reduce. We will still add a command to compute it since it's a common operation. Thanks for the feedback : )
- ekanna 13y agoLack of native Date support was holding me back previously. Now i don't see any reason for not using rethinkdb. My next project will be using Rethinkdb! It will be nice if you can fix "groupby field names" issue https://github.com/rethinkdb/rethinkdb/issues/386 https://github.com/rethinkdb/rethinkdb/issues/386 in your 1.9 release