10 ms·

Turso SQLite Offline Sync Public Beta

222 points by charlieirish 2y ago

kmetan 2y ago
Looks good, but i am fine with midas.dll
senderista 2y ago
LOL "[conflict] resolution is not yet implemented"
shanth 2y ago
how about gun.js with sqlite3 storage adapter https://github.com/gundb/sqlite https://github.com/gundb/sqlite
benwilber0 2y ago
I just wrote a post about this kind of stuff and why it's (almost always) nonsense. You will spend more time and effort trying to shoe-horn SQLite into a server database than you will ever get any benefits from. https://benwilber.github.io/programming/2025/03/31/sqlite-is-great-for-what-it-is.html https://benwilber.github.io/programming/2025/03/31/sqlite-is...
- crazygringo 2y ago
  Agreed. I've been seeing a lot of these posts lately, about embedding SQLite into web servers. I think a lot of people just don't realize how few resources Postgres or MySQL use, and how fast they are. You can run Apache and MySQL and a scripting language on a tiny little 512 MB memory instance, and serve some decent traffic. It works great. Wanting to use SQLite and deal with replication is a nightmare. I don't get it. (And I love using SQLite in apps and scripts. But not websites!)
- GrumpyCat42 2y ago
  Yup - I just recently learned this lesson the hard way with Turso's LibSQL server. While some of the features (like s3 replication) are cool and interesting, the amount of time working around multiple writers and foreign key shenanigans was not worth it when Postgres would have just gotten the job done.
  - NathanFlurry 2y ago
    Can you elaborate on both of these issues? Vanilla SQLite solved multiple writers a long time ago when they introduced WAL in 2010. Does Turso not support this? Is the issue with foreign keys that they're not enabled by default?
joelkoen 2y ago
> Point-of-Sale Systems — process transactions regardless of internet connectivity Mentioning this as an example use case is simply untrue, right?
- aabhay 2y ago
  I think this might be true in an invoicing based world but I agree that PoS is a misnomer. But I can imagine e.g. a field sales rep in construction taking orders on site in a rural area then syncing up when they return to connectivity.
brikym 2y ago
Does Turso have any plans to support streaming queries for real time applications?
curtisszmania 2y ago
[dead]
progx 2y ago
Does it support type safety? Or exists an add / module on top of turso that support it?
billconan 2y ago
This sounds great, but I have some questions regarding data integrity and security. If I build an offline first app using Turso, will my client directly exchange data with the database, without a layer of backend APIs to guarantee data integrity and security? For example, certain db write is only permitted for certain users, but when the db API is exposed, will that cause problems? A concrete example would be a forum where only moderators can remove users and posts. Say if I build an offline first forum, can a hacker hack the database on the filesystem and utilize the syncing feature to propagate the hacked data to the server?
- ochiba 2y ago
  I am not sure about Turso but I've seen a few different approaches to this with other sync engine architectures: 1. At a database level: Using something like RLS in Postgres 2. At a backend level: The sync engine processes write operations via the backend API, where custom validation and authorization logic can be applied. 3. At a sync engine level: If the sync engine processes the write operations, there can be some kind of authorization layer similar to RLS enforced by the sync engine on the backend.
- aboodman 2y ago
  Yes, this is a central issue in sync. For most applications, sync engines just aren't useful without some solution. Of course you need to validate inputs, support fine-grained permissions, etc., as developers have done with web apps for eons. In Replicache, we addressed this by making your application server responsible for writes: https://doc.replicache.dev/concepts/how-it-works https://doc.replicache.dev/concepts/how-it-works By doing this, your server can implement any validation it wants. It can also interact with external systems, do notifications, etc. Anything you can do with a traditional API. In our new sync engine, Zero (https://zerosync.dev https://zerosync.dev), we're adding this same ability soon (like this week) under the name custom mutators: https://bugs.rocicorp.dev/issue/3045 https://bugs.rocicorp.dev/issue/3045 This has been a hard project, but is really critical to use sync engines for anything serious.
  - isaachinman 2y ago
anovick 2y ago
Is Turso open-source? is libSQL open-source?
- anonzzzies 2y ago
  Seems not really or only partially anyway; I cannot see, with disasters like Fauna, that anyone would trust their core stuff with something not open source. But maybe it's just me. I need to be able to switch and sure, I can switch to the open source fork libSQL (open core I think, so nope) (which they are rewriting to Rust for some bizarre reason; sqlite is one of the most readable, robust and well tested codebases in the business so it looks like burning vc money, but whatever), but once my business depends on their offline / replication / etc, then I cannot switch to anything, so never going to happen. Again, opinion. It's core infra, in my opinion at least that should never depend on others or, if the others inevitably screw you over for a few $, you need to be able to move without possibly bankrupting your company.
- cultofmetatron 2y ago
  turso is a SAAS built on libsql. libsql itself is opensource
  - canadiantim 2y ago
    Mind you they're migrating from libsql to Turso (formerly called Limbo), a rust-based rewrite of sqlite entirely
    - adamrezich 2y ago
      Also important to note that, by all appearances, they seem to intend to Embrace Extend Extinguish sqlite3 with said rewrite.
      - jakelazaroff 2y ago
        That’s a tall order; by SQLite’s own estimates, SQLite is used more than all other databases combined and may even be the most widely deployed software component, ever: https://www.sqlite.org/mostdeployed.html https://www.sqlite.org/mostdeployed.html
DaveMcMartin 2y ago
I would like to test it with a large database to see how it handles a 3–5 GB database sync. In the example, it shows syncing returning a promise. Is there no way to track the progress of the sync?
srameshc 2y ago
I really admire Turso. This Offline Sync is probably like ELectric offering but if it will work as anticipated , lot of read only data can be closed to the frontend and can improve performance and save cost.
klabb3 2y ago
What is the realtime and reactivity support in Turso? It’s starting to look attractive but do updates propagate automatically without polling?
- avinassh 2y ago
  (I work at Turso) libSQL does not support realtime/reactivity features. There have been attempts (by both our team and the community), but we encountered difficulties and ultimately abandoned the idea. I can check with the team for the exact reasons. However, Limbo (our SQLite rewrite in Rust) will include this feature, though there is no ETA yet. It's currently being discussed here: https://github.com/tursodatabase/limbo/discussions/1149 https://github.com/tursodatabase/limbo/discussions/1149. Please share your thoughts and use cases.
  - klabb3 2y ago
    Thanks. My personal use case is a client with business logic written in Go with intermittent connectivity, some data is private to the device, some private to the server, and with some data synced from server to clients. Many events are time critical and need instant updates. To me it’s more general than that though. The expectation of having reactive behavior comes up everywhere from chat apps to collaborative to even games. Polling and refreshing feels very 90s and ”web page centric” but of course in some cases (say long form blogging or similar) it’s not a concern.
ezekg 2y ago
It'd be nice if the post went into how conflict resolution will (?) work because that's the hard part here and the main selling point imo.
- Matthias247 2y ago
  I'm wondering too. In general this seems to work only if there's a single offline client that accepts the writes. With limitations to the data scheme (e.g. have distinct tables per client), it might work with multiple clients. However those would need to be documented and I couldn't see anything in this blog post.
- titaphraz 2y ago
  A lot of offline sync projects just drop data on conflict and pretend it didn't happen. It's the salesman job to divert your questions about it. I found another blogpost from turso where they say they offer 3 options on conflict: drop it, rebase it (and hope for no conflict?) and "handle it yourself". Writing an offline sync isn't hard. Dealing with conflicts is a PITA. https://turso.tech/blog/introducing-offline-writes-for-turso#how-does-conflict-resolution-work- https://turso.tech/blog/introducing-offline-writes-for-turso...
- bob1029 2y ago
  Conflict resolution can't work in a general sense. How you reconcile many copies of the same record could depend on time of action, server location, authority level of the user, causality between certain business events, enabled account features, prior phase of the moon, etc. Whether or not offline sync can even work is very much a domain specific concern. You need to talk to the business about the pros & cons first. For example, they might not like the semantics regarding merchant terminals and offline processing. I can already hear the "what if the terminal never comes back online?" afternoon meeting arising out of that one.
  - ochiba 2y ago
    I would say this is why server-authoritative systems that allow for custom logic in the backend for conflict resolution work well in practice (like Replicache, PowerSync and Zero - custom mutators coming in beta for the latter). Predefined deterministic distributed conflict resolution such as CRDT data structures work well for certain use cases like text editing, but many other use cases require deeper customizability based on various factors like you said.
cheema33 2y ago
What are some of the differences from Electric SQL?
- thruflo 2y ago
  Electric is a read-path sync engine for Postgres that does partial replication [1]. It's agnostic to your choice of client -- you can sync into anything you like. It doesn't handle or mandate a pattern for syncing local writes back to Postgres. Instead, it's designed to allow you to handle concerns like writes and auth using your existing API [2]. Turso Offline Sync is an active-active replication system for distributed SQLite/Turso databases. IIUC, this release adds the capability to sync local writes back to the cloud [3]. [1] https://electric-sql.com https://electric-sql.com [2] https://electric-sql.com/blog/2024/11/21/local-first-with-your-existing-api https://electric-sql.com/blog/2024/11/21/local-first-with-yo... [3] https://turso.tech/blog/turso-offline-sync-public-beta#what-s-available-in-the-public-beta- https://turso.tech/blog/turso-offline-sync-public-beta#what-...
0x6c6f6c 2y ago
Since enough comments seem to focus on consolidating writes and security issues, please see the post. They explicitly state this as something they're working on. > we're working hard on the following features: > > Automatic and manual conflict resolution As such everything within this thread is conjecture unless otherwise informed by their work.
- SahAssar 2y ago
  And it is probably not useful for most applications until there is some sort of security there.

222 points by charlieirish 2y ago

kmetan 2y ago

Looks good, but i am fine with midas.dll

senderista 2y ago

LOL "[conflict] resolution is not yet implemented"

shanth 2y ago

how about gun.js with sqlite3 storage adapter https://github.com/gundb/sqlite https://github.com/gundb/sqlite

benwilber0 2y ago

I just wrote a post about this kind of stuff and why it's (almost always) nonsense. You will spend more time and effort trying to shoe-horn SQLite into a server database than you will ever get any benefits from. https://benwilber.github.io/programming/2025/03/31/sqlite-is-great-for-what-it-is.html https://benwilber.github.io/programming/2025/03/31/sqlite-is...

crazygringo 2y ago
Agreed. I've been seeing a lot of these posts lately, about embedding SQLite into web servers. I think a lot of people just don't realize how few resources Postgres or MySQL use, and how fast they are. You can run Apache and MySQL and a scripting language on a tiny little 512 MB memory instance, and serve some decent traffic. It works great. Wanting to use SQLite and deal with replication is a nightmare. I don't get it. (And I love using SQLite in apps and scripts. But not websites!)
GrumpyCat42 2y ago
Yup - I just recently learned this lesson the hard way with Turso's LibSQL server. While some of the features (like s3 replication) are cool and interesting, the amount of time working around multiple writers and foreign key shenanigans was not worth it when Postgres would have just gotten the job done.
- NathanFlurry 2y ago
  Can you elaborate on both of these issues? Vanilla SQLite solved multiple writers a long time ago when they introduced WAL in 2010. Does Turso not support this? Is the issue with foreign keys that they're not enabled by default?

joelkoen 2y ago

> Point-of-Sale Systems — process transactions regardless of internet connectivity Mentioning this as an example use case is simply untrue, right?

aabhay 2y ago
I think this might be true in an invoicing based world but I agree that PoS is a misnomer. But I can imagine e.g. a field sales rep in construction taking orders on site in a rural area then syncing up when they return to connectivity.

brikym 2y ago

Does Turso have any plans to support streaming queries for real time applications?

curtisszmania 2y ago

[dead]

progx 2y ago

Does it support type safety? Or exists an add / module on top of turso that support it?

billconan 2y ago

This sounds great, but I have some questions regarding data integrity and security. If I build an offline first app using Turso, will my client directly exchange data with the database, without a layer of backend APIs to guarantee data integrity and security? For example, certain db write is only permitted for certain users, but when the db API is exposed, will that cause problems? A concrete example would be a forum where only moderators can remove users and posts. Say if I build an offline first forum, can a hacker hack the database on the filesystem and utilize the syncing feature to propagate the hacked data to the server?

ochiba 2y ago
I am not sure about Turso but I've seen a few different approaches to this with other sync engine architectures: 1. At a database level: Using something like RLS in Postgres 2. At a backend level: The sync engine processes write operations via the backend API, where custom validation and authorization logic can be applied. 3. At a sync engine level: If the sync engine processes the write operations, there can be some kind of authorization layer similar to RLS enforced by the sync engine on the backend.
aboodman 2y ago
Yes, this is a central issue in sync. For most applications, sync engines just aren't useful without some solution. Of course you need to validate inputs, support fine-grained permissions, etc., as developers have done with web apps for eons. In Replicache, we addressed this by making your application server responsible for writes: https://doc.replicache.dev/concepts/how-it-works https://doc.replicache.dev/concepts/how-it-works By doing this, your server can implement any validation it wants. It can also interact with external systems, do notifications, etc. Anything you can do with a traditional API. In our new sync engine, Zero (https://zerosync.dev https://zerosync.dev), we're adding this same ability soon (like this week) under the name custom mutators: https://bugs.rocicorp.dev/issue/3045 https://bugs.rocicorp.dev/issue/3045 This has been a hard project, but is really critical to use sync engines for anything serious.
- isaachinman 2y ago

anovick 2y ago

Is Turso open-source? is libSQL open-source?

anonzzzies 2y ago
Seems not really or only partially anyway; I cannot see, with disasters like Fauna, that anyone would trust their core stuff with something not open source. But maybe it's just me. I need to be able to switch and sure, I can switch to the open source fork libSQL (open core I think, so nope) (which they are rewriting to Rust for some bizarre reason; sqlite is one of the most readable, robust and well tested codebases in the business so it looks like burning vc money, but whatever), but once my business depends on their offline / replication / etc, then I cannot switch to anything, so never going to happen. Again, opinion. It's core infra, in my opinion at least that should never depend on others or, if the others inevitably screw you over for a few $, you need to be able to move without possibly bankrupting your company.
cultofmetatron 2y ago
turso is a SAAS built on libsql. libsql itself is opensource
- canadiantim 2y ago
  Mind you they're migrating from libsql to Turso (formerly called Limbo), a rust-based rewrite of sqlite entirely
  - adamrezich 2y ago
    Also important to note that, by all appearances, they seem to intend to Embrace Extend Extinguish sqlite3 with said rewrite.
    - jakelazaroff 2y ago
      That’s a tall order; by SQLite’s own estimates, SQLite is used more than all other databases combined and may even be the most widely deployed software component, ever: https://www.sqlite.org/mostdeployed.html https://www.sqlite.org/mostdeployed.html

DaveMcMartin 2y ago

I would like to test it with a large database to see how it handles a 3–5 GB database sync. In the example, it shows syncing returning a promise. Is there no way to track the progress of the sync?

srameshc 2y ago

I really admire Turso. This Offline Sync is probably like ELectric offering but if it will work as anticipated , lot of read only data can be closed to the frontend and can improve performance and save cost.

klabb3 2y ago

What is the realtime and reactivity support in Turso? It’s starting to look attractive but do updates propagate automatically without polling?

avinassh 2y ago
(I work at Turso) libSQL does not support realtime/reactivity features. There have been attempts (by both our team and the community), but we encountered difficulties and ultimately abandoned the idea. I can check with the team for the exact reasons. However, Limbo (our SQLite rewrite in Rust) will include this feature, though there is no ETA yet. It's currently being discussed here: https://github.com/tursodatabase/limbo/discussions/1149 https://github.com/tursodatabase/limbo/discussions/1149. Please share your thoughts and use cases.
- klabb3 2y ago
  Thanks. My personal use case is a client with business logic written in Go with intermittent connectivity, some data is private to the device, some private to the server, and with some data synced from server to clients. Many events are time critical and need instant updates. To me it’s more general than that though. The expectation of having reactive behavior comes up everywhere from chat apps to collaborative to even games. Polling and refreshing feels very 90s and ”web page centric” but of course in some cases (say long form blogging or similar) it’s not a concern.

ezekg 2y ago

It'd be nice if the post went into how conflict resolution will (?) work because that's the hard part here and the main selling point imo.

Matthias247 2y ago
I'm wondering too. In general this seems to work only if there's a single offline client that accepts the writes. With limitations to the data scheme (e.g. have distinct tables per client), it might work with multiple clients. However those would need to be documented and I couldn't see anything in this blog post.
titaphraz 2y ago
A lot of offline sync projects just drop data on conflict and pretend it didn't happen. It's the salesman job to divert your questions about it. I found another blogpost from turso where they say they offer 3 options on conflict: drop it, rebase it (and hope for no conflict?) and "handle it yourself". Writing an offline sync isn't hard. Dealing with conflicts is a PITA. https://turso.tech/blog/introducing-offline-writes-for-turso#how-does-conflict-resolution-work- https://turso.tech/blog/introducing-offline-writes-for-turso...
bob1029 2y ago
Conflict resolution can't work in a general sense. How you reconcile many copies of the same record could depend on time of action, server location, authority level of the user, causality between certain business events, enabled account features, prior phase of the moon, etc. Whether or not offline sync can even work is very much a domain specific concern. You need to talk to the business about the pros & cons first. For example, they might not like the semantics regarding merchant terminals and offline processing. I can already hear the "what if the terminal never comes back online?" afternoon meeting arising out of that one.
- ochiba 2y ago
  I would say this is why server-authoritative systems that allow for custom logic in the backend for conflict resolution work well in practice (like Replicache, PowerSync and Zero - custom mutators coming in beta for the latter). Predefined deterministic distributed conflict resolution such as CRDT data structures work well for certain use cases like text editing, but many other use cases require deeper customizability based on various factors like you said.

cheema33 2y ago

What are some of the differences from Electric SQL?

thruflo 2y ago
Electric is a read-path sync engine for Postgres that does partial replication [1]. It's agnostic to your choice of client -- you can sync into anything you like. It doesn't handle or mandate a pattern for syncing local writes back to Postgres. Instead, it's designed to allow you to handle concerns like writes and auth using your existing API [2]. Turso Offline Sync is an active-active replication system for distributed SQLite/Turso databases. IIUC, this release adds the capability to sync local writes back to the cloud [3]. [1] https://electric-sql.com https://electric-sql.com [2] https://electric-sql.com/blog/2024/11/21/local-first-with-your-existing-api https://electric-sql.com/blog/2024/11/21/local-first-with-yo... [3] https://turso.tech/blog/turso-offline-sync-public-beta#what-s-available-in-the-public-beta- https://turso.tech/blog/turso-offline-sync-public-beta#what-...

0x6c6f6c 2y ago

Since enough comments seem to focus on consolidating writes and security issues, please see the post. They explicitly state this as something they're working on. > we're working hard on the following features: > > Automatic and manual conflict resolution As such everything within this thread is conjecture unless otherwise informed by their work.

SahAssar 2y ago
And it is probably not useful for most applications until there is some sort of security there.