4 ms·

Given the insert speeds, I'm guessing they're (still) not reusing compiled statements. I mentioned this last time I saw Realm on the front page[1], but their b

by Groxx 12y ago

Given the insert speeds, I'm guessing they're (still) not reusing compiled statements. I mentioned this last time I saw Realm on the front page[1], but their benchmarks are misleading without at least mentioning that they're doing this. I'm still very much interested in Realm, and LOVE having a competitor to SQLite. But don't mislead your customers. [1]: https://news.ycombinator.com/item?id=8043332 https://news.ycombinator.com/item?id=8043332 where seepel finds SQLite to be more than twice as fast as Realm after that one change to the benchmark.

timanglade 12y ago
That was a totally 100% fair point you made when we launched for iOS, and something we had completely overlooked. Your comment immediately prompted us to update our bench code to reuse compiled statements and, we’ve been using updated charts to document our iOS performance since then (e.g.: https://speakerdeck.com/timanglade/realm-overview-at-the-silicon-valley-ios-meetup#17 https://speakerdeck.com/timanglade/realm-overview-at-the-sil...). See other comment (https://news.ycombinator.com/item?id=8390541 https://news.ycombinator.com/item?id=8390541) regarding our performance on Android — we do reuse compiled statements. EDIT: typos. EDIT 2: clearly referencing that this message was about iOS and pointing to the other comment for Android.
- Groxx 12y ago
  Cool, thanks! I've mostly been looking at the iOS announcement page, which hasn't changed - harder to track down other materials you've used for promotion :/ Not that an old blog post is a great barometer of such things, of course. Thanks again for looking into it and updating the material - far too many groups just gloss over (glaring) problems with their micro benchmarks, and it just sours all their future claims. Glad you're not one of them :)
timanglade 12y ago
With those numbers that's an understandable assumption on your part, but as it turns out the dev team did learn its lesson from your helpful pointers around the time we launched for iOS… I just checked our Android code and we are reusing compiled statements in our benchmarks (Inserts only since compiled statements are not available for multi-column queries, afawk). Here’s the code we use for inserts: public void testBatchInserts() throws PerformanceTestException { SQLiteStatement stmt = db.compileStatement("INSERT INTO " + EmployeeDatabaseHelper.TABLE_EMPLOYEES + " VALUES(?1, ?2, ?3, ?4)"); db.beginTransaction(); for (int row = 0; row < getNumInserts(); row++) { stmt.clearBindings(); stmt.bindString(2, getEmployeeName(row)); stmt.bindLong(3, getEmployeeAge(row)); stmt.bindLong(4, getEmployeeHiredStatus(row)); stmt.executeInsert(); } db.setTransactionSuccessful(); db.endTransaction(); } So there you go: no intention to mislead. Us overlooking prepared statements when we originally benched iOS was definitely a regrettable error but we do try to improve. Thanks for helping us along the way.
- Groxx 12y ago
  Glad to hear it! Guess it's more surprises between platforms :| Given that, I'm even more interested in the Android version, and definitely curious what the source code looks like - that's a respectable speedup, you must be doing something right. Would you have the source code for the benchmarks available somewhere? On the iOS post there was a link, but I didn't see it on the android one.
  - timanglade 12y ago
    Yup we’re cleaning up the project right now and will post a link soon. It was in the pile of things we didn’t quite get to do before launch… Sorry about the delay. I haven’t looked too deep personally but I don’t think the speedup comes from anything new we’re doing in the C++ layer. It’s more likely coming from the fact that we tried to find smart ways around Java & the JVM to avoid the kind of copying or hijacking other ORMs do. We did hit some of that stuff again yesterday/today as we just added migrations into the mix but we have some theories on how to get some of that speed back.

by Groxx 12y ago

timanglade 12y ago

That was a totally 100% fair point you made when we launched for iOS, and something we had completely overlooked. Your comment immediately prompted us to update our bench code to reuse compiled statements and, we’ve been using updated charts to document our iOS performance since then (e.g.: https://speakerdeck.com/timanglade/realm-overview-at-the-silicon-valley-ios-meetup#17 https://speakerdeck.com/timanglade/realm-overview-at-the-sil...). See other comment (https://news.ycombinator.com/item?id=8390541 https://news.ycombinator.com/item?id=8390541) regarding our performance on Android — we do reuse compiled statements. EDIT: typos. EDIT 2: clearly referencing that this message was about iOS and pointing to the other comment for Android.

Groxx 12y ago
Cool, thanks! I've mostly been looking at the iOS announcement page, which hasn't changed - harder to track down other materials you've used for promotion :/ Not that an old blog post is a great barometer of such things, of course. Thanks again for looking into it and updating the material - far too many groups just gloss over (glaring) problems with their micro benchmarks, and it just sours all their future claims. Glad you're not one of them :)

timanglade 12y ago

With those numbers that's an understandable assumption on your part, but as it turns out the dev team did learn its lesson from your helpful pointers around the time we launched for iOS… I just checked our Android code and we are reusing compiled statements in our benchmarks (Inserts only since compiled statements are not available for multi-column queries, afawk). Here’s the code we use for inserts: public void testBatchInserts() throws PerformanceTestException { SQLiteStatement stmt = db.compileStatement("INSERT INTO " + EmployeeDatabaseHelper.TABLE_EMPLOYEES + " VALUES(?1, ?2, ?3, ?4)"); db.beginTransaction(); for (int row = 0; row < getNumInserts(); row++) { stmt.clearBindings(); stmt.bindString(2, getEmployeeName(row)); stmt.bindLong(3, getEmployeeAge(row)); stmt.bindLong(4, getEmployeeHiredStatus(row)); stmt.executeInsert(); } db.setTransactionSuccessful(); db.endTransaction(); } So there you go: no intention to mislead. Us overlooking prepared statements when we originally benched iOS was definitely a regrettable error but we do try to improve. Thanks for helping us along the way.

Groxx 12y ago
Glad to hear it! Guess it's more surprises between platforms :| Given that, I'm even more interested in the Android version, and definitely curious what the source code looks like - that's a respectable speedup, you must be doing something right. Would you have the source code for the benchmarks available somewhere? On the iOS post there was a link, but I didn't see it on the android one.
- timanglade 12y ago
  Yup we’re cleaning up the project right now and will post a link soon. It was in the pile of things we didn’t quite get to do before launch… Sorry about the delay. I haven’t looked too deep personally but I don’t think the speedup comes from anything new we’re doing in the C++ layer. It’s more likely coming from the fact that we tried to find smart ways around Java & the JVM to avoid the kind of copying or hijacking other ORMs do. We did hit some of that stuff again yesterday/today as we just added migrations into the mix but we have some theories on how to get some of that speed back.