leif

13 ms·

91.
by leif 13y ago
If you're on mongo and interested in performance, take a look at tokumx. It's a drop in replacement for mongodb with a better storage engine. http://www.tokutek.com/2013/09/tokumx-vs-mongodb-in-memory-s..
92.
by leif 13y ago
I think you just invented an internet forum with a proof-of-technical-proficiency requirement for each post. Well done, sir.
93.
by leif 13y ago
TokuMX adds compression, fine grained locking, and a lot more to MongoDB. It's a version of MongoDB with the storage code replaced with the same storage core as TokuDB.
94.
by leif 13y ago
Actually, you can do that. I wrote a tool to allow replication from MongoDB to TokuMX: http://www.tokutek.com/2013/07/tokumx-1-0-3-seamless-migrati... . It doesn't allow TokuMX instances to satisfy write c
95.
by leif 13y ago
Such a thing could never exist, because the point of tokumx is to change the storage system, so at some point you have to change the storage over and that's just going to be a rewrite of all your data. It sucks but that's the way
96.
by leif 13y ago
It's not really a drop in replacement for BDB, it's more like a library whose API was inspired by BDB. You're welcome to use it directly, but it doesn't implement all of BDB, we have added a bunch of things (like db->
97.
by leif 13y ago
(Don't worry about off topic I love deep conversations like this. Come to our mailing list if you want to continue more though, we may be exhausting HN)
98.
by leif 13y ago
Agility refers to our online schema change abilities in mysql (hot column add/delete, hot indexing). Our compression technique is naive, you're right (but we don't decompress the whole 4MB for one single point query, as you m
99.
by leif 13y ago
Many primary keys in the wild are sequential, which makes the unique check hit cache and not require any I/O. Let me be clear: in the worst case with a unique index, Fractal Tree indexes are the same speed as B-tree indexes. In most c
100.
by leif 13y ago
I now realize I didn't answer how we check uniqueness. We just do a query like any other normal point query, using a serializable transaction, and if we pass the check, then we do the insert with the same transaction (which took a r
101.
by leif 13y ago
It is production ready, and many people are using it, but as with any deep stack technology, growth is a slow process. From an engineering standpoint, I'd say the data structure wasn't fully mature (ready to replace nearly all us
102.
by leif 13y ago
Also, MVCC snapshots mean that if you're doing a range query, it doesn't matter if someone comes in and injects a message above you while you're scanning a node.
103.
by leif 13y ago
It is highly concurrent but you need some tricks beyond that simple description. Here's how we did it: http://www.tokutek.com/2013/01/concurrency-improvements-in-t... http://www.tokutek.com/20
104.
by leif 13y ago
Actually, you can beat B-trees pretty handily across the board in exactly the scenario you described (a, b, c). The log(N) performance of a B-tree is not just extremely hard to improve on (for searches), it's impossible to improve on.
105.
by leif 13y ago
Couldn't resist: try tokumx if you want "mongodb with compression". There isn't a speed/space usage tradeoff. Our latest blog post is about long field names: http://www.tokutek.com/2013/08/
106.
by leif 13y ago
Especially for us mobile users.
107.
by leif 13y ago
Tokutek is hiring in Lexington, MA - Full Time - Technical Support Engineer Tokutek delivers next generation storage technology for the database world. We develop the open-source, high-performance Fractal Tree indexing library ( http:&#x2F
108.
by leif 13y ago
Jammers. Thanks, iPhone.
109.
by leif 13y ago
I didn't know GPS hammers were a thing but now I want one. Thanks, LSE.
110.
by leif 13y ago
I see rethinkdb as being focused more on the data model and language, and the cluster administration experience. The performance doesn't seem compelling yet, though they are admirably durable by default. You're right, with a relia
111.
by leif 13y ago
Of course they are, the trick is to get the most utility out of each one. I work at Tokutek where we use a data structure that does this, in the sense that when your working set is larger than RAM, we still don't incur very many I&#x2
112.
by leif 13y ago
You can achieve durability with very high performance using write-ahead logging, lots of concurrent writers doing group commit for the log fsyncs, and a write-optimized data structure like an LSM tree or a fractal tree. Maybe not 1 million
113.
by leif 13y ago
Pretty often, if you're doing simple queries. From there, you can either increase the size of the query, or make it more complex (which increases its computational size) to amortize away the overhead. It always depends on the workloa
114.
TokuMX 1.0.3: Seamless Migrations from MongoDB(tokutek.com)
15 points by leif 13y ago|2 comments
115.
by leif 13y ago
Yes, but he has to read a LOT more code (arguably the important part of programming) than you or I on a daily basis.
116.
by leif 13y ago
Surely your SSD has more than 800 IOPS though. Have you done any profiling?
117.
by leif 13y ago
For my curiosity, where is the insert bottleneck now?
118.
by leif 13y ago
What do you see as MongoDB's limitations, and are any/most of them solved by (the product I work on,) TokuMX[1]? [1]: http://www.tokutek.com/products/tokumx-for-mongodb/
119.
by leif 13y ago
The tarball releases are still there...
120.
by leif 13y ago
Lexington, MA - Tokutek ( http://www.tokutek.com/ ) Full-time testing/QA engineer. Use whatever you like to break our software in interesting ways. We want someone who can build out and own testing for the long term.

More ›