4 ms·

Are transactions supported in that solution or in the mentioned Datalog related ecosystem? I have built my own similar reactive inmemory triple store with types

by pezo1919 4y ago

Are transactions supported in that solution or in the mentioned Datalog related ecosystem? I have built my own similar reactive inmemory triple store with typescripts type safety, but then I realized I still need transactions which are a bit of a pain, because transactions are bundling the otherwise separete triplets, so the atomic, independent logic of triplets and related effects breaks a bit. (I am sure it's solvable.)

jitl 4y ago
The example code uses a `transact` function. But it really depends on what you mean by “transaction”. We don’t use a triple store at Notion, but we do use an abstraction that ensures a collection of operations either all succeed or all fail. We don’t support “interactive” transaction, where you can read, modify, write as an atomic group. This just isn’t desirable in a multiplayer or offline system - in cases we need that kind of consistency we use a normal HTTP API which is online-only.
- pezo1919 4y ago
  Cool, thanks! Now a question: how a dev picks up related knowledge? I have a BSc and extra 7 years in development, but this area was totally gray to me. My motivation was only to come up a scalable solution for offline first apps with some kind of automatic persistence support, so at the end of the day my design goals were quite similar. How do you come up with stuff at notion? Are there must have books or just going with gut, experience and existing solutions you are aware about?
stopachka 4y ago
As jitl points out, in multiplayer settings, what you need is some way to commit a series of transactions all-together or not at all. We support this. In the case where you want to read the database inside your transaction, we take inspiration from Datomic. Datomic runs all mutations in one high-memory box. You can provide functions that run in that box. This way, you can guarantee that the reads inside your transaction have the latest value. There's a lot of UX to figure out there, and this would be something to try to avoid in an offline-available setting.
- pezo1919 4y ago
  Yes exactly this UX issue bothers me a lot, I wouldn't even go there. :D What about migrations? Do you support? That's another thing I need in my offline first project, one of my other project has died because the lack of it. (I need something which plays well with Expo.io)
  - stopachka 4y ago
    Migrations are very tough when you go offline-first. Cambria [^1] is an interesting read. For Instant, we are schemaless and think about offline more like a cache. In our case it's less of a problem. [^1]: https://www.inkandswitch.com/cambria/ https://www.inkandswitch.com/cambria/
    - pezo1919 4y ago
      Thank you! Having one of my projects died to it (lack of migrations, offline-first) I had promised myself I'll first solve that problem before writing a meaningful line of business code in a future project. :)
  - tommiegannert 4y ago
    I'm working on building a database in the same space as InstantDB. Currently, it's an "object/graph database using Protobuf". There's a check to ensure updated Protobuf definitions are backwards-compatible. Of course, this still implicitly relies on using Protobufs correctly (i.e. a missing value is the same as zero/empty/null/nil), even though I'm trying to make it safe by default. I'm curious what your needs are. Would you mind elaborating on what kind of migrations your project would have needed to not die?
- pezo1919 4y ago
  Oh another thing: do you guys have twitter or something? :) Would love to follow the project and devs too.
  - stopachka 4y ago

jitl 4y ago

The example code uses a `transact` function. But it really depends on what you mean by “transaction”. We don’t use a triple store at Notion, but we do use an abstraction that ensures a collection of operations either all succeed or all fail. We don’t support “interactive” transaction, where you can read, modify, write as an atomic group. This just isn’t desirable in a multiplayer or offline system - in cases we need that kind of consistency we use a normal HTTP API which is online-only.

pezo1919 4y ago
Cool, thanks! Now a question: how a dev picks up related knowledge? I have a BSc and extra 7 years in development, but this area was totally gray to me. My motivation was only to come up a scalable solution for offline first apps with some kind of automatic persistence support, so at the end of the day my design goals were quite similar. How do you come up with stuff at notion? Are there must have books or just going with gut, experience and existing solutions you are aware about?

stopachka 4y ago

As jitl points out, in multiplayer settings, what you need is some way to commit a series of transactions all-together or not at all. We support this. In the case where you want to read the database inside your transaction, we take inspiration from Datomic. Datomic runs all mutations in one high-memory box. You can provide functions that run in that box. This way, you can guarantee that the reads inside your transaction have the latest value. There's a lot of UX to figure out there, and this would be something to try to avoid in an offline-available setting.

pezo1919 4y ago
Yes exactly this UX issue bothers me a lot, I wouldn't even go there. :D What about migrations? Do you support? That's another thing I need in my offline first project, one of my other project has died because the lack of it. (I need something which plays well with Expo.io)
- stopachka 4y ago
  Migrations are very tough when you go offline-first. Cambria [^1] is an interesting read. For Instant, we are schemaless and think about offline more like a cache. In our case it's less of a problem. [^1]: https://www.inkandswitch.com/cambria/ https://www.inkandswitch.com/cambria/
  - pezo1919 4y ago
    Thank you! Having one of my projects died to it (lack of migrations, offline-first) I had promised myself I'll first solve that problem before writing a meaningful line of business code in a future project. :)
- tommiegannert 4y ago
  I'm working on building a database in the same space as InstantDB. Currently, it's an "object/graph database using Protobuf". There's a check to ensure updated Protobuf definitions are backwards-compatible. Of course, this still implicitly relies on using Protobufs correctly (i.e. a missing value is the same as zero/empty/null/nil), even though I'm trying to make it safe by default. I'm curious what your needs are. Would you mind elaborating on what kind of migrations your project would have needed to not die?
pezo1919 4y ago
Oh another thing: do you guys have twitter or something? :) Would love to follow the project and devs too.
- stopachka 4y ago