4 ms·

Former Realm employee here. So glad to see the team finally launch this. I've seen them work their asses for years (literally!) to launch something I know Real

Former Realm employee here. So glad to see the team finally launch this. I've seen them work their asses for years (literally!) to launch something I know Realm’s 100k+ dev community was clamoring for from day one but was really hard to build. I'm biased of course but honestly I think they have soft-key shipped one of the most transformational mobile technologies ever: real-time, conflict-free sync that works as well offline as online. There was a lot of literature on this of course but bringing it all together in one product was something else! From engineering to marketing, product & founders I know they've sweated to get here and I'm just so, so happy for them. I know they will make a lot of developers and customers happy, and for that I say: bravo.

danielrhodes 10y ago
CouchDB and iCloud already have this.
- HaloZero 10y ago
  iCloud's implementation is awful. It doesn't help you understand or resolve the conflicts at all and just assumes last write wins.
  - danielrhodes 10y ago
    Is such a conflict likely to occur with a single user?
marknadal 10y ago
@timanglade, congrats to you and your former team! Always awesome to launch, good job. Full disclosure, I work on a competing product (MIT licensed, https://github.com/amark/gun https://github.com/amark/gun ) and do a lot of testing on distributed systems. You guys say it is offline-first with conflict-resolution. However your documentation ( https://realm.io/docs/realm-object-server/#conflict-resolution https://realm.io/docs/realm-object-server/#conflict-resoluti... ) states the "Last update wins". This is contradictory for two reasons: A) Either you mean last write wins relative to the server. But an offline update is going to get sent late (on reconnect), which would make it "last" and lead to weird problems. B) Or you mean last write wins relative to the time stamp of the client. But this does not account for wall clock drift. What happens if the client's clock is intentionally set 2 years in the future? In a distributed system, everybody has a different "last". The demo video, while very well done, uses a drawing app which does not really demonstrate real conflict (ie, if 2 users draw over the same space, it becomes a z-index rendering issue, which could be solved with conflict resolution but isn't a good example of conflict).
- timanglade 10y ago
  See the full answer from Realm’s founder/CEO here: https://news.ycombinator.com/item?id=12590753 https://news.ycombinator.com/item?id=12590753
  - marknadal 10y ago
    Thanks, did not see that. Not all vector clocks provide conflict resolution though - actually, they can very much lack in this regard. For instance, you are much more likely to get two clients that increment to the same vector while offline than two clients getting the exact same millisecond timestamp. Which would make the problem worse, not resolved. What type of vector are you using?
    - simonask 10y ago
      Simon from Realm here - just wanted to chime in and clarify some details on this. It would be more correct to say that we are using Lamport timestamps, with the addition of a globally unique ID for each client and some tracking of changeset ancestry. No two identical [timestamp, id] tuples will ever be produced in our system, so there is always an unambiguous way to decide which side "came first" in case of a causally unrelated conflict. It's important to understand that most "conflicts" in our system are resolved without resorting to timestamps at all, which is possible due to the way we encode our changesets on the wire and the fact that we track their ancestry. It is only used in cases where we absolutely have to, i.e. when there is no causal relationship between two updates of the same property (in which case, the "later" timestamp wins). Hope that answered your question. :-)
      - marknadal 10y ago
        Awesome. Thanks!
      - jlu 10y ago
        Thanks for the details, just wondering is the implementation mentioned here available to review somewhere? Is it included in the realm core repo?
amelius 10y ago
How do you deal with security/access control? This seems to me the most difficult aspect of a client-side database, but please correct me if I'm wrong.
- bigfish24 10y ago
  Realm supports AES-256 encryption so you can encrypt your data at rest and then SSL/TLS for security in transit. Realm Object Server supports the ability to control which users can access which Realms, supporting for example shared Realms among users to power collaborative experiences like the drawing app in our demo video. The client side APIs to manage these permissions weren't exposed in this release but will be coming soon in the beta.
johnm1019 10y ago
If Firebase releases an on-prem or self-hosted server, what does the Realm Mobile Platform do differently? I'm really excited about Realm, but I honestly can't read this without asking (rhetorically), "have you ever heard about Firebase?" From a technological perspective I think it's unfair to claim Realm is the first to the party here, Firebase engineers have shipped an equally impressive technical feat. I understand the vendor and platform lock-in with Firebase is a _major_ differentiator when compared to Realm - but the underlying technology appears to offer the same benefits.
- bigfish24 10y ago
  (Adam from Realm) First, we respect everything Firebase and team has done. Others might feel differently, but shipping great software is hard, so nothing but respect from us there. Second, and to address your question, we are very different architecturally than Firebase. We built from the ground up an object database that runs on mobile. As a result, this means when using Realm Object Server, the mobile development is still focused around that database. In practice, this means developers only have to think about local APIs because their UI is now bound to the data in their local Realm, with the networking handled automatically. This isn't just a basic cache, though, but a full-featured database that includes: memory-mapped object interface, ACID compliance, querying, and much more. Firebase doesn't offer that capability since it's mobile SDKs aren't built off a mobile database like Realm. Beyond the mobile capability/developer experience, the server-side capabilities of Realm Object Server go beyond whether it is self hosted or not. While not included in the developer edition, we offer server-side SDK support and event handling capabilities so that the server can integrate into existing infrastructure vs attempt to control how you build your backend.

danielrhodes 10y ago

CouchDB and iCloud already have this.

HaloZero 10y ago
iCloud's implementation is awful. It doesn't help you understand or resolve the conflicts at all and just assumes last write wins.
- danielrhodes 10y ago
  Is such a conflict likely to occur with a single user?

marknadal 10y ago

@timanglade, congrats to you and your former team! Always awesome to launch, good job. Full disclosure, I work on a competing product (MIT licensed, https://github.com/amark/gun https://github.com/amark/gun ) and do a lot of testing on distributed systems. You guys say it is offline-first with conflict-resolution. However your documentation ( https://realm.io/docs/realm-object-server/#conflict-resolution https://realm.io/docs/realm-object-server/#conflict-resoluti... ) states the "Last update wins". This is contradictory for two reasons: A) Either you mean last write wins relative to the server. But an offline update is going to get sent late (on reconnect), which would make it "last" and lead to weird problems. B) Or you mean last write wins relative to the time stamp of the client. But this does not account for wall clock drift. What happens if the client's clock is intentionally set 2 years in the future? In a distributed system, everybody has a different "last". The demo video, while very well done, uses a drawing app which does not really demonstrate real conflict (ie, if 2 users draw over the same space, it becomes a z-index rendering issue, which could be solved with conflict resolution but isn't a good example of conflict).

timanglade 10y ago
See the full answer from Realm’s founder/CEO here: https://news.ycombinator.com/item?id=12590753 https://news.ycombinator.com/item?id=12590753
- marknadal 10y ago
  Thanks, did not see that. Not all vector clocks provide conflict resolution though - actually, they can very much lack in this regard. For instance, you are much more likely to get two clients that increment to the same vector while offline than two clients getting the exact same millisecond timestamp. Which would make the problem worse, not resolved. What type of vector are you using?
  - simonask 10y ago
    Simon from Realm here - just wanted to chime in and clarify some details on this. It would be more correct to say that we are using Lamport timestamps, with the addition of a globally unique ID for each client and some tracking of changeset ancestry. No two identical [timestamp, id] tuples will ever be produced in our system, so there is always an unambiguous way to decide which side "came first" in case of a causally unrelated conflict. It's important to understand that most "conflicts" in our system are resolved without resorting to timestamps at all, which is possible due to the way we encode our changesets on the wire and the fact that we track their ancestry. It is only used in cases where we absolutely have to, i.e. when there is no causal relationship between two updates of the same property (in which case, the "later" timestamp wins). Hope that answered your question. :-)
    - marknadal 10y ago
      Awesome. Thanks!
    - jlu 10y ago
      Thanks for the details, just wondering is the implementation mentioned here available to review somewhere? Is it included in the realm core repo?

amelius 10y ago

How do you deal with security/access control? This seems to me the most difficult aspect of a client-side database, but please correct me if I'm wrong.

bigfish24 10y ago
Realm supports AES-256 encryption so you can encrypt your data at rest and then SSL/TLS for security in transit. Realm Object Server supports the ability to control which users can access which Realms, supporting for example shared Realms among users to power collaborative experiences like the drawing app in our demo video. The client side APIs to manage these permissions weren't exposed in this release but will be coming soon in the beta.

johnm1019 10y ago

If Firebase releases an on-prem or self-hosted server, what does the Realm Mobile Platform do differently? I'm really excited about Realm, but I honestly can't read this without asking (rhetorically), "have you ever heard about Firebase?" From a technological perspective I think it's unfair to claim Realm is the first to the party here, Firebase engineers have shipped an equally impressive technical feat. I understand the vendor and platform lock-in with Firebase is a _major_ differentiator when compared to Realm - but the underlying technology appears to offer the same benefits.

bigfish24 10y ago
(Adam from Realm) First, we respect everything Firebase and team has done. Others might feel differently, but shipping great software is hard, so nothing but respect from us there. Second, and to address your question, we are very different architecturally than Firebase. We built from the ground up an object database that runs on mobile. As a result, this means when using Realm Object Server, the mobile development is still focused around that database. In practice, this means developers only have to think about local APIs because their UI is now bound to the data in their local Realm, with the networking handled automatically. This isn't just a basic cache, though, but a full-featured database that includes: memory-mapped object interface, ACID compliance, querying, and much more. Firebase doesn't offer that capability since it's mobile SDKs aren't built off a mobile database like Realm. Beyond the mobile capability/developer experience, the server-side capabilities of Realm Object Server go beyond whether it is self hosted or not. While not included in the developer edition, we offer server-side SDK support and event handling capabilities so that the server can integrate into existing infrastructure vs attempt to control how you build your backend.