6 ms·
Show HN: LevelDB outperforms others on a cheap phone (Microsoft ESE and SQLite)
- cmrx64 11y agoNice demo. If one just needs a KV store, it's good that the engine that only does KV outperforms the more general engines ;) On top of all the engineering that goes into making LevelDB fast.
- hendzen 11y ago500 inserts is way too small for a database benchmark. At that size point you're stressing the in-memory format of the database and not the actual IO performance. I'm guessing LevelDB just wins because it's considerably simpler and doesn't have to go through a schema layer - it's literally just a LSM tree.
- maxpert 11y agoI tried it with 5K to 10K entries and it still out performs SQLite or any other engines. For SQLite the default and most probabilistic way a programmer will implement a key value store. You can try the sample on your own machine as well.
- hendzen 11y agoWell, you're also doing a random insert. That workload will always perform better on an LSM tree when compared to a B/B+ Tree (used by the non-LevelDB engines). Can you post sequential read/write benchmarks in addition?
- maxpert 11y agoI the article I have already covered that LevelDB is known to have better sequential performance anywayz http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html
- pvg 11y agoDo you mean 'likely'? Because 'probabilistic' is something else.
- maxpert 11y agoI would say likely
- asutherland 11y agoThe SQLite numbers are not going to be realistic here given that "no SQLite transactions" are used and the write-ahead-log is probably not used. Assuming defaults are used, that means "PRAGMA synchronous=FULL" and "PRAGMA journal_mode=DELETE" (so no Write-Ahead-Log). This means every mutating statement is potentially going to result in multiple fsync() operations. See https://www.sqlite.org/pragma.html https://www.sqlite.org/pragma.html for context.
- maxpert 11y agoI didn't use any of the sophisticated stuff at all. As article states, I did it like what a typical programmer might have done if he wanted to do some basic key-value storage.
- kbaker 11y agoPlease put a note in your article explaining this. SQLite is literally 10x faster using the WAL vs standard mode. And while you're at it, please consider an index on the SQL server and SQLite tables, so the reads are faster as well. I would love to see the article updated using SQLite's PRAGMA journal_mode = WAL and PRAGMA synchronous = NORMAL. Then it would be a much more fair comparison. Please don't give programmers a license to be lazy and not learn about their tools!!! If this article is trying to inform, or give benchmarks, it should not come to invalid conclusions without explaining the tradeoffs.
- maxpert 11y agoYep have tried it with WAL and it still doesn't beat LevelDB. Thanks for the lazy suggestion I added disclaimer on top of the page right away!
- swsieber 11y agoBut what is the point of your page? If it's to make recommendations then go a step further and include changing a setting or two for better recommendations. Considering your background and endorsement of nosql databases, it comes off as more of a puff piece if you don't actually try to make the other db's run fast. It especially seems that way when you tout it as "blazing fast compared to any other storage solutions", and then proceed to test it in one narrow setup. That noted, you said that you used the default settings, and I'd be curious if there are settings for LevelDB that could be changed to make it faster. I'd be interested if you did more tests explore more than just one particular setup. Edit: I realized that this may have come off a little negative. Overall I think it's nice, but seems less applicable than you make it seem.
- clumsysmurf 11y agoIt would also be interesting to see a LevelDB vs SQLite comparison on Android, but there is some room to improve SQLite on ext4 (Journaling the Journal): https://www.usenix.org/conference/atc13/technical-sessions/presentation/jeong https://www.usenix.org/conference/atc13/technical-sessions/p... SQLite 4 is supposed to have a simplified K/V storage engine. That would probably be a more fair comparison. (Hm, looking at LevelDB I don't see references to Android any more, maybe they dropped support?)
- PeCaN 11y agoSQLite 4 is also designed, in part, to beat LevelDB (i.e. they explicitly benchmark the default storage engine against LevelDB)[1]. SQLite 4 looks pretty darn cool, is it still actively being worked on? Commits seem quite scarce[2]. -- 1. https://sqlite.org/src4/doc/trunk/www/design.wiki https://sqlite.org/src4/doc/trunk/www/design.wiki 2. https://sqlite.org/src4/timeline https://sqlite.org/src4/timeline
- maxpert 11y agoCheckout https://github.com/maxpert/lsm-windows https://github.com/maxpert/lsm-windows for port of the core LSM engine of SQLite4. I might be doing one in one more weekend.
- maxpert 11y agoI already did a port of SQLite4's LSM engine on Universal Windows (https://github.com/maxpert/lsm-windows https://github.com/maxpert/lsm-windows) would have to do some more work before I can do a benchmark. Would love if someone can contribute :)
- ddorian43 11y agoDoesn't leveldb , kinda suck? Meaning every ~serious db doesn't use it or has created a fork of it ? or see the symas-lmdb benchmarks for many different usecases of different kv stores http://symas.com/mdb/#bench http://symas.com/mdb/#bench
- maxpert 11y agoWell LMDB is hands down the fastest of them all. All the benchmarks that I saw with LMDB were mostly on Linux and ext file system. It would be really interesting to see how the memory mapped files perform on a phone though. But doing so requires me to port complete LMDB on phone :P
- catwell 11y agoLMDB already runs on iOS and Android. In my experience you will have issues on some Android devices though. For instance, I recently found out that on some Android devices writes to a mmap-ed file through the usual POSIX interfaces are not immediatly reflected on reads through the mmap, which means that on those devices you MUST use MDB_WRITEMAP. For those who don't know, I/O on Android is a nightmare. On your typical phone, the filesystem you have access to is a FUSE that mirrors to an EXT4 filesystem mounted with crazy device-specific options which include noauto_da_alloc. I don't know any DB which works fast and consistently across all Android phones on the market. SQLite is probably the safest bet for now.
- mtanski 11y ago> LMDB already runs on iOS and Android. In my experience you will have issues on some Android devices though. For instance, I recently found out that on some Android devices writes to a mmap-ed file through the usual POSIX interfaces are not immediatly reflected on reads through the mmap, which means that on those devices you MUST use MDB_WRITEMAP. How is that possible if they are using Linux and UBC (Unified Buffer Cache). It's unlikely they anybody would go undo that from the VM/FS layer. The only think I can think off this behavior happening in a Linux app is if you're creating a private mapping... Private mappings are COW and a write to the global would cause a copy of the old data for you app. They'll be no sycing till you msync()
- akhilcacharya 11y agoHow many applications are really doing intensive read/writes on mobile?
- spotman 11y agoDefine intensive. But any well architected app that wants to work well without Internet will write many things to disk. 500 writes a second is very realistic when downloading some bulk content that will be used to power different states of the app. Now this might not be a continuous burst of 500 operations a second sustained, but many it has a handful of these bursts over several minutes. Given that, I would say that lots of mobile apps can have brief periods of tons of writes.
- habosa 11y agoOn Android most well-architected applications will persist almost everything to disk before showing it to the user. Network directly to UI is too unstable. It's not about intensive reads/writes (dozens of MB) it's about making each individual read or write very fast so you don't drop frames. On Android you have 16ms to draw each frame (60fps). If you have a DB request take 5ms then all of a sudden you're very likely to have jank in your app.
- lern_too_spel 11y agoDon't do I/O on the UI thread. The time it takes to do a write should not affect jank.
- charleslmunger 11y agoYeah, a well architected application doesn't touch disk on the UI thread.
- maxpert 11y agoJust write a chat or email application that persists your conversation on disk. It goes crazy at times.
- spotman 11y agoRealm would also be a great comparison. Albeit the core is not open yet so maybe that's why it was left out? For mobile databases though it really screams for me, when compared to SQLite.
- maxpert 11y agoGood suggestion will add it to benchmarks
- dorfsmay 11y agoThe last time I look at LevelDB it didn't support multiprocess access to the same file (so ended up using SQLite). Ha this change?
- cat-dev-null 11y agoLevelDB also made the Bitcoin client super unstable and prone to dataloss. Use sqlite, because it's bullet-resistent and has competent maintainers.
- halayli 11y agoThat's because leveldb by default writes asynchronously. And probably at the time of reading they were still in RAM/cached.
- maxpert 11y agoLet me quote LevelDB docs here: "By default, each write to leveldb is asynchronous: it returns after pushing the write from the process into the operating system. The transfer from operating system memory to the underlying persistent storage happens asynchronously. The sync flag can be turned on for a particular write to make the write operation not return until the data being written has been pushed all the way to persistent storage. (On Posix systems, this is implemented by calling either fsync(...) or fdatasync(...) or msync(..., MS_SYNC) before the write operation returns.)" and if you see my benchmarks I am flushing each entry to the disk.
- halayli 11y agoSomething is still missing. It's hard to imagine such drastic performance difference in btree implementation. btw I've had better experience using kyoto db than levelDB.
- maxpert 11y agoUpdated the documentation with assumptions.