7 ms·

Realm is a mobile database: an alternative to SQLite and key-value stores

54 points by viebel 10y ago

david-given 10y ago
The documentation links on the github page are broken; see http://realm.io http://realm.io to find out what it is. (AFAICT it's basically Firebase.)
- citruspi 10y ago
  There are two products - Realm Mobile Database[0] and Realm Mobile Platform[1]. The former is, as the HN post suggests, basically a database for mobile applications which you can use in place of SQLite or as a key value store. The latter combines the database software which runs on the mobile devices with the Realm Object Server[2] which runs server side to provide something like Firebase (disclaimer: I've used the Mobile Database but haven't used the Mobile Platform (and Object Server) yet). [0]: https://realm.io/products/realm-mobile-database/ https://realm.io/products/realm-mobile-database/ [1]: https://realm.io/products/realm-mobile-platform/ https://realm.io/products/realm-mobile-platform/ [2]: https://realm.io/docs/realm-object-server https://realm.io/docs/realm-object-server
  - lpga510 10y ago
    What are the differences with firebase? There must be something.
    - bigfish24 10y ago
      The biggest would be the Realm Mobile Platform is not a service, but a server-side application, Realm Object Server, that you can run on your own alongside the mobile client SDKs embedded in your apps. Another key difference, is that with our Node.js SDK you can create a level of integration with any existing backend infrastructure that would be harder with Firebase.
- kristiandupont 10y ago
  We're fixing those, but they should be: https://realm.io/docs/javascript/latest https://realm.io/docs/javascript/latest and https://realm.io/docs/javascript/latest/api https://realm.io/docs/javascript/latest/api
- Grustaf 10y ago
  No, it's (at least) as easy to use as Firebase, but it's much more powerful. It uses actual objects, not json. It's very close to manipulating normal objects in the language, extremely nice to work with. There are limitations, especially with the syncing, but it's by far the nicest database solution I know of.
- HodGreeley 10y ago
  I don't think of Realm as being at all like Firebase, except from a very high level. Realm isn't hosted, it's an object database, until recently didn't do sync, and more.
wmblaettler 10y ago
Realm has been discussed on HN quite a bit for the past 3 years: https://hn.algolia.com/?query=realm&sort=byPopularity&prefix&page=0&dateRange=all&type=story https://hn.algolia.com/?query=realm&sort=byPopularity&prefix... It's been really nice to use so far for me, plus the Realm team puts out a lot of helpful deeper dive blog posts: https://realm.io/news/ https://realm.io/news/
danbruc 10y ago
OT: What happened in the database space and why? In 2000 you would have had a hard time naming enough database systems to run out of fingers for counting. But in recent years database system seem to pop out of the ground like JavaScript frameworks. So what changed? Did the idea of a general purpose database system break because the usage scenarios became so diverse that we can no longer pretend that one size fits almost all? Is this just a perception thing because everybody started uploading his weekend hack toy database to GitHub and blogs about it? And what really irritates me the most, database systems are really complex pieces of software requiring years and years of development time, where does this time come from and who pays for it? Especially having more and more different databases system to choose from does not really promise that developing yet another one will be an easy commercial success.
- jinjin2 10y ago
  Realm has been available for many years and is actually really different from what else is out there by the fact that it is a real object database. It has just been flying under the radar a bit given that it has only been available for mobile devices. I for one think that it is awesome that it is now available for node.js as well. Especially with the new replication functionality that allows you to share live data between node instances running on different machines.
  - lobster_johnson 10y ago
    "Real" object databases — "OODBMS" — uses to be a thing, too. They still survive in some niches, like finance and health, though largely as legacy systems. Products like Objectivity, ObjectStore, POET, Caché (which is built on MUMPS), GemStone (Smalltalk) etc. were once big, expensive products that seemed poised to take over from old-hat SQL databases. Those databases failed not necessarily because the technology was wrong, although they certainly made some choices that didn't help; those databases were often tightly coupled with opaque binary object serialization formats (with no way of looking into a database without going the binary route) that were in turn tightly coupled with the app's class hiearchy. For example, several of these actually post-processed your code (e.g. your Java bytecode) to insert serialization code, so that when you followed a relationship (e.g. "library.books[0].author"), it would automatically query under the hood and return the author as an object. They ended up being clunky to work with as a result; for example, in the beginning, very few of them had a query language, so the only way to query the data was by following relationships through your graph of objects. Realm looks interesting as a new approach to the object database design, but at the same time seems a bit limited. It doesn't seem to have schemas or any kind of role-based permissions, and seems to make the same mistake of not including a query language (so you can't build a REPL for it, and every language binding needs its own query builder). The transaction semantics also seem to be entirely undefined (is it read-committed?). From what I can tell, it's also "offline first", in the sense that you have a local database that can sync to/from a server, but you're technically always working on an offline copy of the master data; while nice to have for a mobile app, it does limit what you can use it for. I also don't see any support for fine-grained updates. Every write you do seems to overwrite, which I assume means the last write always wins; how do you deal with conflicts, or implement things like counters, maps or arrays that need to be incrementally updated?
azinman2 10y ago
I've had some nasty bugs happen in production with Realm, as is expected with any new database store. Newer version seem better in this regard, but if you read the changelogs you might be shocked at how fundamental the fixes can be. It also times some time getting use to -- my biggest gripe is that the notification API won't tell you what actually has been modified, just that an object changed _somehow_. That makes a pure reactive design quite hard if those changes have non-trivial consequences, and not just a quick re-rendering of some element on the screen. Looking forward to finding time to go back to SQLite where I can actually do complicated queries, relational joins, triggers, etc on a fast and mature code base.
- BoorishBears 10y ago
  SQLBrite adds reactivity to SQLite (https://github.com/square/sqlbrite https://github.com/square/sqlbrite) But in general I rely on RxJava to provide fine grained notifications. For example (psudeocode): someUserQueryThatReturnsObservable() .map(user->user.firstName) .distinct() .subscribe(newFirstName->newFirstName) It's messy because you end up having to repeat everything after map for every property, not really an ideal solution
  - unsoundInput 10y ago
    I started using sqlbrite (in combination with sqldelight) and stopped treating the database as an object store. Instead I create an interface for the data the customer needs (e.g. an item interface for elements in a listview), write a select query whose result fulfills this interface, and let the rest happen via databinding. This seems to work fine up to a couple of thousand of elements for me.
  - deleted 10y ago
    [deleted]
- andrewbarba 10y ago
  Re: your gripe, this is solved in their latest release with object-level notifications: https://realm.io/news/realm-objc-swift-2.4/ https://realm.io/news/realm-objc-swift-2.4/ I personally think they did an awesome job with the change api
akeruu 10y ago
I have dabbled a bit with it quite a few times and was quite happy about it, but my main issue is that AFAIK there is not "pure" Java API for Realm. It seems there have been attempts at removing Android dependencies but it seems they were never finished, meaning you can't have it running on desktop apps. This is (sadly) a showstopper for me.
- HodGreeley 10y ago
  Have you looked at Couchbase? NoSQL on-device db, cross-platform for Android, iOS, native Java, and others. (I work for Couchbase.)
- bigfish24 10y ago
  Adam from Realm, thanks for the feedback. We do plan to offer a pure Java API, but it hasn't been a top priority. However, with the launch of Realm Mobile Platform we have been getting more requests for this because teams want to use Java vs. Node on the server-side to build integrations. My best guess would be Q3 or Q4 of this year, since we still have higher priority features to work on.
netdur 10y ago
guys I think this is new, the title is misleading, this is realm for javascript
cyberferret 10y ago
It would be really cool to see this working in Ionic/Cordova. I've played with Firebase and CouchDB (and also SQLite) in Ionic, so would be good to see how Realm could work in the same context. Sometimes I just want a simple persistent on board storage mechanism for my mobile apps without all the problems that localStorage exhibits. I wonder if Realm could be a lightweight alternative?
- bigfish24 10y ago
  There is unofficial Realm Cordova support being done by the community: https://github.com/airamrguez/cordova-plugin-realm https://github.com/airamrguez/cordova-plugin-realm
laex 10y ago
Shameless plug: If you're looking for a simple key-value store for Swift, please checkout https://github.com/hemantasapkota/SwiftStore https://github.com/hemantasapkota/SwiftStore Underneath it's LevelDB.
theamk 10y ago
Is there any technical information on how the database works? For a supposed Sqlite replacement, there is a surprising lack of the information. The only reference to concurrency mode I have found is this: > All changes to an object (addition, modification and deletion) must be done within a write transaction. > Write transactions incur non-negligible overhead - you should architect your code to minimize the number of write transactions. So is there a single write lock? If you try to start two transactions, will the second one be delayed or retried? Can reads proceed while the transaction is processing? > Results instances are live, auto-updating views into the underlying data So if I am doing x.first, x.second while the object is being updated in a second thread, will I get mixed state? Is database crash proof? If the phone runs out of battery while in write transaction, will the database get damaged? ... so many questions
- brmunk 10y ago
  It's ACID & Serializable. A write will block all other writes, but multiple reads can occur without overhead or blocking due to MVCC. Reads are consistent within the runloop, but updates to your objects occurs when you start a write transaction, so you know you always work on the latest version of the data. Basically, you are in an implicit read transaction all the time, except when you are in a write transaction :-). > So if I am doing x.first, x.second while the object is being updated in a second thread, will I get mixed state? So no you will not get mixed state. > Is database crash proof? If the phone runs out of battery while in write transaction, will the database get damaged? Yes it is crashproof. Fully ACID. Any other questions? This article goes a bit more into the details of threading: https://realm.io/news/threading-deep-dive/ https://realm.io/news/threading-deep-dive/ And this one more into the core design: https://realm.io/news/jp-simard-realm-core-database-engine/ https://realm.io/news/jp-simard-realm-core-database-engine/

54 points by viebel 10y ago

The documentation links on the github page are broken; see http://realm.io http://realm.io to find out what it is. (AFAICT it's basically Firebase.)

citruspi 10y ago
There are two products - Realm Mobile Database[0] and Realm Mobile Platform[1]. The former is, as the HN post suggests, basically a database for mobile applications which you can use in place of SQLite or as a key value store. The latter combines the database software which runs on the mobile devices with the Realm Object Server[2] which runs server side to provide something like Firebase (disclaimer: I've used the Mobile Database but haven't used the Mobile Platform (and Object Server) yet). [0]: https://realm.io/products/realm-mobile-database/ https://realm.io/products/realm-mobile-database/ [1]: https://realm.io/products/realm-mobile-platform/ https://realm.io/products/realm-mobile-platform/ [2]: https://realm.io/docs/realm-object-server https://realm.io/docs/realm-object-server
- lpga510 10y ago
  What are the differences with firebase? There must be something.
  - bigfish24 10y ago
    The biggest would be the Realm Mobile Platform is not a service, but a server-side application, Realm Object Server, that you can run on your own alongside the mobile client SDKs embedded in your apps. Another key difference, is that with our Node.js SDK you can create a level of integration with any existing backend infrastructure that would be harder with Firebase.
kristiandupont 10y ago
We're fixing those, but they should be: https://realm.io/docs/javascript/latest https://realm.io/docs/javascript/latest and https://realm.io/docs/javascript/latest/api https://realm.io/docs/javascript/latest/api
Grustaf 10y ago
No, it's (at least) as easy to use as Firebase, but it's much more powerful. It uses actual objects, not json. It's very close to manipulating normal objects in the language, extremely nice to work with. There are limitations, especially with the syncing, but it's by far the nicest database solution I know of.
HodGreeley 10y ago
I don't think of Realm as being at all like Firebase, except from a very high level. Realm isn't hosted, it's an object database, until recently didn't do sync, and more.

wmblaettler 10y ago

Realm has been discussed on HN quite a bit for the past 3 years: https://hn.algolia.com/?query=realm&sort=byPopularity&prefix&page=0&dateRange=all&type=story https://hn.algolia.com/?query=realm&sort=byPopularity&prefix... It's been really nice to use so far for me, plus the Realm team puts out a lot of helpful deeper dive blog posts: https://realm.io/news/ https://realm.io/news/

danbruc 10y ago

OT: What happened in the database space and why? In 2000 you would have had a hard time naming enough database systems to run out of fingers for counting. But in recent years database system seem to pop out of the ground like JavaScript frameworks. So what changed? Did the idea of a general purpose database system break because the usage scenarios became so diverse that we can no longer pretend that one size fits almost all? Is this just a perception thing because everybody started uploading his weekend hack toy database to GitHub and blogs about it? And what really irritates me the most, database systems are really complex pieces of software requiring years and years of development time, where does this time come from and who pays for it? Especially having more and more different databases system to choose from does not really promise that developing yet another one will be an easy commercial success.

jinjin2 10y ago
Realm has been available for many years and is actually really different from what else is out there by the fact that it is a real object database. It has just been flying under the radar a bit given that it has only been available for mobile devices. I for one think that it is awesome that it is now available for node.js as well. Especially with the new replication functionality that allows you to share live data between node instances running on different machines.
- lobster_johnson 10y ago
  "Real" object databases — "OODBMS" — uses to be a thing, too. They still survive in some niches, like finance and health, though largely as legacy systems. Products like Objectivity, ObjectStore, POET, Caché (which is built on MUMPS), GemStone (Smalltalk) etc. were once big, expensive products that seemed poised to take over from old-hat SQL databases. Those databases failed not necessarily because the technology was wrong, although they certainly made some choices that didn't help; those databases were often tightly coupled with opaque binary object serialization formats (with no way of looking into a database without going the binary route) that were in turn tightly coupled with the app's class hiearchy. For example, several of these actually post-processed your code (e.g. your Java bytecode) to insert serialization code, so that when you followed a relationship (e.g. "library.books[0].author"), it would automatically query under the hood and return the author as an object. They ended up being clunky to work with as a result; for example, in the beginning, very few of them had a query language, so the only way to query the data was by following relationships through your graph of objects. Realm looks interesting as a new approach to the object database design, but at the same time seems a bit limited. It doesn't seem to have schemas or any kind of role-based permissions, and seems to make the same mistake of not including a query language (so you can't build a REPL for it, and every language binding needs its own query builder). The transaction semantics also seem to be entirely undefined (is it read-committed?). From what I can tell, it's also "offline first", in the sense that you have a local database that can sync to/from a server, but you're technically always working on an offline copy of the master data; while nice to have for a mobile app, it does limit what you can use it for. I also don't see any support for fine-grained updates. Every write you do seems to overwrite, which I assume means the last write always wins; how do you deal with conflicts, or implement things like counters, maps or arrays that need to be incrementally updated?

azinman2 10y ago

I've had some nasty bugs happen in production with Realm, as is expected with any new database store. Newer version seem better in this regard, but if you read the changelogs you might be shocked at how fundamental the fixes can be. It also times some time getting use to -- my biggest gripe is that the notification API won't tell you what actually has been modified, just that an object changed _somehow_. That makes a pure reactive design quite hard if those changes have non-trivial consequences, and not just a quick re-rendering of some element on the screen. Looking forward to finding time to go back to SQLite where I can actually do complicated queries, relational joins, triggers, etc on a fast and mature code base.

BoorishBears 10y ago
SQLBrite adds reactivity to SQLite (https://github.com/square/sqlbrite https://github.com/square/sqlbrite) But in general I rely on RxJava to provide fine grained notifications. For example (psudeocode): someUserQueryThatReturnsObservable() .map(user->user.firstName) .distinct() .subscribe(newFirstName->newFirstName) It's messy because you end up having to repeat everything after map for every property, not really an ideal solution
- unsoundInput 10y ago
  I started using sqlbrite (in combination with sqldelight) and stopped treating the database as an object store. Instead I create an interface for the data the customer needs (e.g. an item interface for elements in a listview), write a select query whose result fulfills this interface, and let the rest happen via databinding. This seems to work fine up to a couple of thousand of elements for me.
- deleted 10y ago
  [deleted]
andrewbarba 10y ago
Re: your gripe, this is solved in their latest release with object-level notifications: https://realm.io/news/realm-objc-swift-2.4/ https://realm.io/news/realm-objc-swift-2.4/ I personally think they did an awesome job with the change api

akeruu 10y ago

I have dabbled a bit with it quite a few times and was quite happy about it, but my main issue is that AFAIK there is not "pure" Java API for Realm. It seems there have been attempts at removing Android dependencies but it seems they were never finished, meaning you can't have it running on desktop apps. This is (sadly) a showstopper for me.

HodGreeley 10y ago
Have you looked at Couchbase? NoSQL on-device db, cross-platform for Android, iOS, native Java, and others. (I work for Couchbase.)
bigfish24 10y ago
Adam from Realm, thanks for the feedback. We do plan to offer a pure Java API, but it hasn't been a top priority. However, with the launch of Realm Mobile Platform we have been getting more requests for this because teams want to use Java vs. Node on the server-side to build integrations. My best guess would be Q3 or Q4 of this year, since we still have higher priority features to work on.

netdur 10y ago

guys I think this is new, the title is misleading, this is realm for javascript

cyberferret 10y ago

It would be really cool to see this working in Ionic/Cordova. I've played with Firebase and CouchDB (and also SQLite) in Ionic, so would be good to see how Realm could work in the same context. Sometimes I just want a simple persistent on board storage mechanism for my mobile apps without all the problems that localStorage exhibits. I wonder if Realm could be a lightweight alternative?

bigfish24 10y ago
There is unofficial Realm Cordova support being done by the community: https://github.com/airamrguez/cordova-plugin-realm https://github.com/airamrguez/cordova-plugin-realm

laex 10y ago

Shameless plug: If you're looking for a simple key-value store for Swift, please checkout https://github.com/hemantasapkota/SwiftStore https://github.com/hemantasapkota/SwiftStore Underneath it's LevelDB.

theamk 10y ago

Is there any technical information on how the database works? For a supposed Sqlite replacement, there is a surprising lack of the information. The only reference to concurrency mode I have found is this: > All changes to an object (addition, modification and deletion) must be done within a write transaction. > Write transactions incur non-negligible overhead - you should architect your code to minimize the number of write transactions. So is there a single write lock? If you try to start two transactions, will the second one be delayed or retried? Can reads proceed while the transaction is processing? > Results instances are live, auto-updating views into the underlying data So if I am doing x.first, x.second while the object is being updated in a second thread, will I get mixed state? Is database crash proof? If the phone runs out of battery while in write transaction, will the database get damaged? ... so many questions

brmunk 10y ago
It's ACID & Serializable. A write will block all other writes, but multiple reads can occur without overhead or blocking due to MVCC. Reads are consistent within the runloop, but updates to your objects occurs when you start a write transaction, so you know you always work on the latest version of the data. Basically, you are in an implicit read transaction all the time, except when you are in a write transaction :-). > So if I am doing x.first, x.second while the object is being updated in a second thread, will I get mixed state? So no you will not get mixed state. > Is database crash proof? If the phone runs out of battery while in write transaction, will the database get damaged? Yes it is crashproof. Fully ACID. Any other questions? This article goes a bit more into the details of threading: https://realm.io/news/threading-deep-dive/ https://realm.io/news/threading-deep-dive/ And this one more into the core design: https://realm.io/news/jp-simard-realm-core-database-engine/ https://realm.io/news/jp-simard-realm-core-database-engine/