18 ms·

Downsides of Offline First

346 points by typingmonkey 5y ago

taneq 5y ago
The most confusing thing to me is a discussion of "offline first" applications which starts with, and maintains, the assumption that your only option is a web app. Back in my day we had a word for software that always worked without an internet connection. We called it "software" and it was installed on the user's computer.
- hengheng 5y ago
  I will still call any web app a "web site", and if that is my age showing, so be it.
  - taneq 5y ago
    We can shout at the cloud together.
  - atatatat 5y ago
    That's fine, if it's on your tablet or PC. If you're looking at a web "site" on a vertical phone screen, one side or the other did something wrong. My point is: sites and services should have two separate experiences, depending on screensize. These are: desktop site, mobile webapp.
  - hoarad 5y ago
    i do so also. and it is because they have a link
- tehbeard 5y ago
  Har de har look at these modern programmers with their web 2.0s and js frameworks of the week... Offline first is not "software" as you classified it. Your "software" you are on about is more accurately described as "offline only", little to none of the functionality requires network access. Offline first refers to how online functionality is needed for the design of that app, but with steps taken to ensure that even without connectivity for periods of time, it still functions.
- jjnoakes 5y ago
  While offline first seems to be discussed in the context of web apps a lot, to me it is more about the data and synchronization than where the executable lives, and most of the ideas also apply to certain kinds of desktop software.
- dspillett 5y ago
  That isn't offline first though: desktop software like that is generally offline only, and the user wraps their own chosen sync method (which could simply be good ol' sneaker-net or frizby-net) around that if they want/need to. Offline first is only used in the context of web applications and sometimes their Android/iOS cousins (which probably share the same backend, where both are available), once the decision had been made that a not-locally-installed and/or remote synced application is desirable where possible, so isn't being suggested (directly) as an alternative to locally installed offline programs.
- jasode 5y ago
  >The most confusing thing to me is a discussion of "offline first" applications which starts with, and maintains, the assumption that your only option is a web app. I didn't downvote your comment but in this author's article, it's deliberate for the starting context for discussion to be a networked collaborative app. Yes, internet connected apps is a subset of all possible software but that's not the point. As an analogy, imagine if someone else submitted an article about C Language memory techniques on an embedded chip. E.g.: https://www.embedded.com/memory-allocation-in-c/ https://www.embedded.com/memory-allocation-in-c/ And then a commenter misunderstands that article complaining, "it's confusing to me because this article maintains that the only option is C in an embedded app but over here, I'm using Python with Cloudflare Workers" In other words, it doesn't seem like you're interested in collaborative apps that require distributed data consistency so this article looks "wrong" to you.
  - phkahler 5y ago
    It's confusing because "offline-first" doesn't even seem to make sense in the context of web-apps, which I thought meant "fancy (functional, able to do stuff) web sites" or similar.
  - josephg 5y ago
    I’ve been working in this problem space for awhile now and I sort of agree with the GP poster. I really like native software; and I want native software which can work simply in a distributed, collaborative context. As an example, I have a note taking app on my laptop. I want to be able to read and edit all my notes on all my other devices. And I want that to work in a way that doesn’t depend on some random startup keeping their servers on the other side of the planet running. Right now every software company which wants to build something like this needs to invent their own data stack, network protocols and storage systems. And the prize at the end is with software that can only talk to itself via closed protocols. It’s infeasible, and inefficient. We have an opportunity right now to do an awful lot better. When we do, I want to service both native and web apps. If we do it right, from the network level the distinction should just boil away anyway.
    - 5y ago
- psychometry 5y ago
  Ok, and if you want your non-web app to be compatible with Windows, MacOS, iOS, and Android and you don't want to write more than one app, your options are...?
  - justinclift 5y ago
    Generally Qt.
  - poetaster 5y ago
    Qt/qml works for. With python sometimes.
- Swenrekcah 5y ago
  A user’s computer? Written with a possessive? What a bizarre idea!
- jitl 5y ago
  It is approximately infinity times easier to distribute and grow the user base of a web app compared to locally installed software. The consumer software economy is moving online for this reason - it’s much better for business. RxDB software comes from this context - it’s a JavaScript library built for this world that attempts to retain the distribution and sharing advantages of the web, while adding back the responsiveness and availability of traditional installed software. I think you’ll find the original “local first” manifesto more aligned with both the user & traditional installed software with less of a web focused bias: https://www.inkandswitch.com/local-first.html https://www.inkandswitch.com/local-first.html > In this article we propose “local-first software”: a set of principles for software that enables both collaboration and ownership for users. Local-first ideals include the ability to work offline and collaborate across multiple devices, while also improving the security, privacy, long-term preservation, and user control of data.
- blacktriangle 5y ago
  Also back in that day 95% of our users were on Windows and we had a direct relationship with them. Now your user base is roughly split 40/40 between iOS and Android with a non-ignorable 20 running some windows tablet thing. Then for extra fun your access to those users is mediated by the giant black boxes staffed by assholes that are the iOS and Play stores. And of course back then nobody had any expectation that your software would easily sync between devices and users because that just wasn't something that was doable easily. The world changes. Yeah it was better for us devs back then, but honestly the new world has some real advantages. For me personally, I'm cautiously optimistic that PWAs are our way out of the hell that is supporting 3 native platforms for all but the rarest cases.
  - lytefm 5y ago
    For me, PWA doesn't quite cut it yet but using a single codebase based on web technology + adding the appropriate wrappers for native functionality (Electron, React Native, Capacitor, whatever...) is fine for now.
  - poetaster 5y ago
    Qt/qml works for me. Write once.
- JamesSwift 5y ago
  The difference is that back in the day it _only_ worked on your computer. There was no cloud component and no collaborative use-case. If you only have a single, local client then a lot of this doesn't apply. But its rare for that to be the case in modern software.
- blondin 5y ago
  i agree with the essence of your comment. the "back in my day" is what is, perhaps, causing resistance. my impression these days is that web engineers have outnumbered other software engineers. that's a problem because context matters as you are alluding to. we shouldn't use terms like "offline first" without an appropriate context. or assume context.
candiddevmike 5y ago
This is a good list. The app I built (https://about.homechart.app https://about.homechart.app) is "read only" offline first, it's a compromise I chose over having to solve queuing writes and consistency checks during the (assuming rare) occurrences users find themselves offline. I'd add write support, but no one has asked for it. Don't go into offline first thinking you need to solve for writes, it can be added later if necessary.
- JamesSwift 5y ago
  The write queue is messy but not hard as long as you are serializing all the operations. The real difficult part is conflict resolution. If you have an easy resolution model (e.g. last one wins) then I think its not too much extra work for you.
megamix 5y ago
Humans first. How about that for an idea.
- lovemenot 5y ago
  Not great to be honest. I cannot imagine how to use this idea to support or reject a decision. Maybe as a slogan or sales pitch... Reminds me of Fujitsu's Vision Statement: Human-centric Computing. I asked a dozen employees of that company what it meant, and not one could give a coherent answer
- olah_1 5y ago
  Idk why you're being voted down. It's a genuine design principle. Some ideas that come to mind that fit with "human first": - offline-first - human curation instead of algorithms (or at least transparent algorithms that are customizable) - the user is not the account number. the account is just the mech that the user climbs into. user can have multiple mechs. - leverage existing social fabric to provide better user experience. account recovery, etc.
ltearno 5y ago
I once made an app and a talk on this matter. Slides about problems mentionned in the article were addressed from slide 15 onwards. Mainly I remember having used Lamport clocks to track rows' causal history. And negative primary keys for inserting data in offline mode (these days I'd use UUID). Since IndexedDB was not a thing yet on every browser, I used asm.js (ancestor to WebAssembly) to compile SQLite for the browser. The database file was stored in the LocalStorage and I used zlib (compiled with emscripten) to make most of the little space LocalStorage gave to us. It has been a learning experience, and worked at the end. I wonder how I would do differently today... By the way, we were using GWT to code for the browser, but that's just an anecdote and not important for that matter... Here is the link : https://fr.slideshare.net/ltearno/easing-offline-web-application-development-with-gwt https://fr.slideshare.net/ltearno/easing-offline-web-applica...
- leetrout 5y ago
  Thanks for sharing. I do not miss GWT but I think it was inspirational.
- JamesSwift 5y ago
  Ive also gone the negative key route, and would move to a "client dictates the key via UUID" like you mention if I redid it today.
  - mamcx 5y ago
    Why? I have done this before and found negative much easier to use and understand. UUID have the nasty property that deny easy debugging, logging or any kind of HUMAN understanding...
    - JamesSwift 5y ago
      The primary benefit is theres no post-hoc reconciliation you need to do where you re-write all the foreign keys and object ids. It greatly streamlines the entire process, and lets you eliminate a lot of code / potential bugs.
jcun4128 5y ago
I made a PWA that used IndexedDB and base64 photos. I ran into a funky max-length issue that was an error from Chromium kind of interesting. But yeah the main problem I had was the base64 images would get too big (if you had too many to load) and then you would see a slower render vs. an image pulled by url. Still pretty cool since I'm not a native developer and RN is something I've dabbled in but don't use daily.
- jitl 5y ago
  These days IndexedDB supports storing binary blobs as File objects or even ArrayBuffers; the relevant bug on the Chrome issue tracker was marked fixed in 2014: https://bugs.chromium.org/p/chromium/issues/detail?id=108012 https://bugs.chromium.org/p/chromium/issues/detail?id=108012
  - jcun4128 5y ago
    Interesting I think was straight up using just text or whatever is the default. Though I was using the Dexie wrapper which is nice. To be clear I don't know if the bug was from IndexedDB it was something about length exceeded. I did try to use small images too eg. some 150px by 150px but going to base64 usually multiplies the size by 1.3 For anyone interested [1] not a phenomenal app but one I poured idk 2-300+hrs into (was working on an RN version too) and went nowhere sucks. I was contributing to one of those codefor# deals. [1] https://github.com/codeforkansascity/tagging-tracker-alt-apps/blob/master/src/App.js#L143 https://github.com/codeforkansascity/tagging-tracker-alt-app...
- ngokevin 5y ago
  Perhaps Service Workers is better than IDB for that?
  - jcun4128 5y ago
    I'll keep it in mind. I used creat-react-app and the built in way to do PWA, although they did update that so I had to do some code updates... some Box thing I forget. (to get cached files to work) Still it's cool to store everything as text/blobs locally.
psychometry 5y ago
It's infuriating to me that WebSQL was killed. 99% of real-world data is relational and yet the powers that be decided that we should all be forced to use IndexedDB and hacky layers like PouchDB built atop it. I'm excited about https://github.com/jlongster/absurd-sql https://github.com/jlongster/absurd-sql though.
- dragonwriter 5y ago
  WebSQL wasn't killed by opposition to having a relational API, it was killed because the spec was tied to, and only implemented by embedding, a specific, identified version of SQLite.
  - EvanAnderson 5y ago
    I think that a developer distaste for relational databases was a major driver. Digging back into correspondence on this a few months ago (when this came up on HN) I found clear statements that Mozilla opposed anything relational. The SQLite version is a convenient excuse for some developers who, at the time, we're enamoured with "NoSQL". Discussion here: https://news.ycombinator.com/item?id=28156831 https://news.ycombinator.com/item?id=28156831
    - WorldMaker 5y ago
      Mozilla for a long time backed their IndexedDB with SQLite, they wouldn't have done that if they were that antagonistic to relational databases. I trust Mozilla's surface reasons here: they inherited the mess that was NPAPI from Netscape, then decades of experience with XUL binary components, were among the many dealing with Flash bugs and zero-day fallout well after Flash's "heyday", and have combined multiple decades of experience in what happens if the web depends on specific binaries to do its job. From that standpoint of they were already knee deep in trying to sandbox/reign in NPAPI, remove XUL, and remove Flash I absolutely understand why "you want the web to depend on the bugs and zero days of SQLite directly with no abstraction layer between?" was a complete non-starter.
  - ec109685 5y ago
hdjjhhvvhga 5y ago
An alternative: a native app.
- endisneigh 5y ago
  native to what?
  - j1elo 5y ago
    The word "native" has diluted in a sea of abstractions and supporting technologies, but in this context I'd read "native app" as removing the browser engine layer (so, not even really native by far, but much closer to the OS and hardware than when writing code on top of the mentioned layer)
    - hdjjhhvvhga 5y ago
      Yes. Putting an app in a browser has solved tens of problems and introduced a hundred new ones. At some point you start to wonder if a plain old native app wouldn't work much better that an uncontrollably growing stack of technologies, some ow which designed for something completely different than what we're trying to accomplish.
      - hunterb123 5y ago
        Okay. You still need a client side database with syncing. Which is what this is about. The pure native approach only solves (kind of) the limited storage issues, nothing else. If you nuke your site and only develop about native apps then you'll lose users to competitors because people can try their service without downloading an app. Yeah maybe we shouldn't have shoved an app run-time into a document viewer, but here we are and things run decently well (thanks to v8 and other tech) if you do things right.
        hdjjhhvvhga 5y ago
        I think you underestimate the number of issues a (purely) native app solves. The fact that many developers don't care about some of them it does not mean they don't exist. One of them is shorter delay, very important in interactive applications. Syncing can be implemented in billion ways, from dedicated solutions to custom ones. Paradoxically, I think Microsoft got it (somewhat) right with Office. You can use a native C++ app, and this is what most people use because of speed and control. However, if you like working online, don't mind saving your documents unencrypted on Microsoft's systems, don't worry about being offline, you need collaborate with others online, or just like the convenience of having your documents always online, you can use the 365 version with inferior but probably acceptable performance - in this way everybody is happy.
josephg 5y ago
I’m a “true believer” in CRDTs, which I have some experience in. You can implement a useful CRDT for simple applications in under 100 lines if all you care about are standard database objects - like maps, sets and values. List CRDTs are where they get complicated, but most applications aren’t collaborative text editors. The promise of CRDTs is that unlike most conflict resolution systems, you can layer over a crdt library and basically ignore all the implantation details. Applications should (can) mostly ignore how a CRDT works when building up the stack. The biggest roadblock to their use is that they’re poorly understood. Well, that and implementation maturity. Automerge-rs merged a PR the other day which brought a 5 minute benchmark run down to 2 seconds. But by bit we’re getting there.
- a_conservative 5y ago
  Can you point to a good introduction to Conflict-free replicated data types (CRDT)? It's crossed my (short attention span) radar a couple times and seems interesting. I really love the idea about being able to use it, but also "forget" about it while developing. Does "leaky abstraction" apply here? If you constrain things enough, can a dev really use it and forget about the details?
- jitl 5y ago
  I work on a collaborative text editing system (https://www.notion.so https://www.notion.so) and read a lot about CRDTs, thank you for your work in this area, particularly https://josephg.com/blog/crdts-go-brrr/ https://josephg.com/blog/crdts-go-brrr/ I agree with your assessments here - CRDT is the way forward for most applications; no user wants to fiddle with a merge UI or picking versions like with iCloud. I think RxDB’s position here is from their CouchDB lineage. > The biggest roadblock to their use is that they’re poorly understood. Well, that and implementation maturity. I certainly have more understanding to do. My biggest open question is how to design my centralized server side storage system for CRDT data. To service writes from old clients I need to retain a total history of a document, but I don’t want to force new clients to download the entire history nor do I want these big histories in my cache, so I end up wanting a hot/cold system; and building that kind of thing and dealing with the edge cases seems like more than 100 lines of code. It seems like the Yjs authors also recognize that CRDT storage on the server is an area to address, there was some work on a custom database in 2018, although my thinking is more about how to retrofit text CRDTs into my existing very conservative production cloud software stack than about writing to block storage.
kevincox 5y ago
I do find it disappointing that there is no reliable local storage system for the web. I get the resistance to trackers but there should be a way to request permission to store data that isn't deleted except for explicit action by users. It means that you effectively have to store the data in "the Cloud" which means 1. I have to pay for it 2. If I shut down the service you are screwed and 3. I have at least some access to it (encryption aside). I would also like to see a synced version since most browsers these days support syncing settings and passwords. But creating a generic syncing solution that is actually useful is hard.
- richardwhiuk 5y ago
  Can you allow the user to load and save files?
  - kevincox 5y ago
    I mean sure, you can get the user to download/upload files, but this is very awkward and not suited for storing every change. (But is good for migrating and backing up the data). Having to get the user to manually break the data out of the browser is not a good UX for day-to-day work. I know that Chrome is pushing for a filesystem API but I don't know if that will be exempt from the usual ephemerality. IIRC it is just a private storage space with a filesystem-like API.
    - easrng 5y ago
      Chrome has an API that allows you to save to files without a new download every time. They made a pretty nice library that wraps that API on Chrome and it gracefully degrades on other browsers. https://web.dev/browser-fs-access/ https://web.dev/browser-fs-access/
      - jcims 5y ago
        i noticed this behavior in a drawing app called excalidraw. first save opens a file dialog, subsequent saves just update the file, basically like a standard local text editor. i keep doing 'save as' to create new files because i don't trust it lol
deleted 5y ago
[deleted]
LeanderK 5y ago
I am a strong believer in PWAs. I think most of the apps I use could be PWAs without a problem. I really don't get why Apple isn't developing them for MacOS, since I've never really used the app-store on MacOS and in comparison to iOS a lot of the apps on my Mac are productivity apps that are basically electron-apps. I think some basic PWA functionality on the desktop would be more interesting for me than more advanced PWAs for ios.
- jdavis703 5y ago
  I want to believe in PWAs. But in the real world they are just way too clunky. I’ve observed this on both software I’ve written, and on PWAs from teams that should objectively know what they’re doing like Google Calendar.
- atatatat 5y ago
  PWAs lead to the walls of Apple's walled garden being torn down. They'll fight it with subtlety until the end, and I hope they burn in hell for it.
  - blauditore 5y ago
    Exactly. They introduced auto-deletion of localStorage under the hood of "privacy", when it was really about driving people to building standalone apps (and use their store).
    - breakfastduck 5y ago
      Good. PWAs are absolutely awful compared to native apps.
      - catlifeonmars 5y ago
        What don’t you like about PWAs?
- breakfastduck 5y ago
  Because native apps are basically a huge chunk of the appeal to using macOS and PWAs are the absolute antithesis of that. Encourage developers to make apps that all look/feel completely different in terms of style/UX - no thanks.
yanis_t 5y ago
PWAs are the future. Along with other benefits comes the fact that you can "install" apps on devices bypassing the app stores (which means not paying commission fees), which I guess is the primarily reason why Apples is so reluctant of giving it a proper support in Safari.
JamesSwift 5y ago
Its a good article but it should be noted that this very much is a web-centric view of Offline First and its challenges. When I say web-centric, I mean as opposed to offline-first on a mobile app. Native apps dont deal with the same issues around storage, and are actually _much_ more performant overall. If you haven't done an offline-first app, I highly recommend it. The experience is magical. You can fly around your app at the speed of the users touch. Content is magically loaded as soon as its clicked on. Its amazing as a user. As for similarities, conflict resolution is a universal problem and is either the most difficult or second most difficult problem [1] to solve. What makes this more difficult is that there is no one-size-fits-all for this. You need to have a deep, nuanced understanding of your system and what makes sense for your use case in terms of resolution strategies. Then you need to implement them, which is not easy especially if your backend is bog-standard REST on a classic SQL datastore. I've enjoyed reading the past 2 rxdb articles on this (the one mentioned here as well as the one from a couple days ago [2]). Its great to have more content on this publicly available, when I was getting into offline-first I only had a couple options. [1] - At the end of the day, offline-first is a whole bunch of caching, and so you end up needing to deep dive into cache invalidation strategies which we all know is a hairy problem. [2] - https://news.ycombinator.com/item?id=28690427 https://news.ycombinator.com/item?id=28690427
- Trufa 5y ago
  Yeah,I'm trying to implement an electron offline first app that syncs, there seems to readymade solution. Stuff like https://github.com/aerogear/offix https://github.com/aerogear/offix seem to be in the right direction of what I'm looking for but not nearly mature enough. I don't want to pu to much effort on the app so I would like something more or less ready made, preferably with graphql apis. Any suggestions welcome.
  - JamesSwift 5y ago
    I would hesitate to use any client-server setup that isnt also your primary data store. So, if you have an "offline aware, multi-tenant" store like CouchDB but you still need to sync to the primary store which is SQL, then you lose a lot of context and awareness around conflict resolution. If you are going to eventually sync to the other store, I would say only use Couch on the frontend, and do the syncing/resolution from the perspective of the client, since the client knows what it was trying to do and how best to notify on conflict. The "better" option is to have an offline-aware client-server component (e.g. Realm) which is the primary store as well. This eliminates the sync and so all conflict resolution stays in the same system with well defined semantics.
endisneigh 5y ago
In practice I believe last write wins or compare last two writes is sufficient. Any thoughts on this in practice?
- sroussey 5y ago
  Last write wins is a simple CRDT merge type. It’s just that their are others depending on the data type. LWW might be good for an avatar photo, but not great for a counter.
twobitshifter 5y ago
Good article. For structured data, I’ve had good luck using UUIDs for keys rather than needing an approach that relies on an atomic clock.
ItsMonkk 5y ago
The way I've learned to use git is to 0. Sync with remote 1. Edit files until I'm ready to check-in 2. Stash changes 3. Sync up with latest from remote 4. Pop changes 5. If there are any conflicts, deal with them here, locally. Possibly delete all changes and redo. Redoing is equivalent to someone checking in something at step 0. If this takes some time, move back to Step 2. 6. Push changes If done this way, the only benefit of CRDT's is during step 5. One of the lessons I've learned and truly believe is that if something sucks, and it gets worse as the problem gets bigger, you need to do it more often. Git merges are a great example of this concept. And this is where CRDT's are in trouble. The best way to make step 5 easier is by making step 1 smaller. CRDT's viewpoint is that we are offline, and therefore should allow any number of edits, and when we reconnect the system should be able to work it out. It flies in the face of smaller commits. The more changes you need to merge, the harder it is. We live in a world that's connected 99% of the time, and CRDT's simply aren't needed. On the other hand, the "Offline First" model is great! A user having access to all of their data is wonderful. As the blog notes, it's not preferable for a user to have the entire Wikipedia or Google indexes on their device. So you need to have a use-case for Offline, and we need to do better about this type of stuff, but we don't need to wait on CRDT research for this. Smart caching and materialized views are where I think the real progress is going to come from, making things like Offline Wikipedia possible.
austincheney 5y ago
> When you create a web based offline first app, you cannot store data directly on the users filesystem. In fact there are many layers between your JavaScript code and the filesystem of the operation system. Solved: File system in the browser plus network distribution - https://github.com/prettydiff/share-file-systems https://github.com/prettydiff/share-file-systems
thayne 5y ago
> in the end the user itself could be the one that deletes the browsers local data This is especially true, because often customer support for many websites have customers begin troubleshooting with clearing the browser's "cache and cookies". In other words deleting all of the local data for all sites. There are ways to delete the local data for just one site, but they are pretty hidden, and involve multiple steps. I wish browsers had a simple "delete all data for just this site" button.
- ranger207 5y ago
  Firefox does! You can "clear cookies and site data" by clicking on the padlock. I use it often to get around Reuters' new paywall that kicks in after a few articles
jadafaa 5y ago
Enjoyed reading this
lambda_dn 5y ago
It's not worth it, have a internet connection is more ubiquitous every day with Wifi, 4/5G and coming soon Low orbit satellite grids. Trying to engineer your app to work offline causes complexity in the design and implementation for a issue that might never be an issue for most customers. Assuming your app is pretty crippled when it can't access the cloud. What's next making your app still work if there is no display by screen reading?
- lytefm 5y ago
  This statement very much depends on the kind of app you're developing, as mentioned in the article. Is the main use case communicating with another user? Do you need a third party API for your app's basic functionality? Sure, don't even consider offline first. But what if the app is only about storing and displaying data entered by the user and you'd definitely also want to be able to use it on an airplane, e.g. a todo/notetaking/journaling app? Then offline first can make sense. I'd even say that the complexity of developing an offline-first app can be lower than that of a classical client/server app + caching logic. Sure, you need to figure out how to do schema migrations and you probably want a kill switch to lock out older apps at some point. But the same applies in a classical setting when API-endpoints should be changed or removed. Basically, your document model is now your API. And a very simplistic conflict resolution strategy like „just take the most recent version“ is often good enough. Once that + basics like auth and account creation are set up, it's very productive and low overhead to work with PouchDB/CouchDB and offline first: - no need to coordinate with a backend team because nothing else than auth happens there - simpler state management and error handling because the state in the local DB can always be assumed to be correct and the DB is always there - no need for schema migrations as long as you only add new docs or extend existing ones - it's great for quickly hacking a prototype or for beginners who just know some HTML/CSS/JS
- jamil7 5y ago
  Ideally yes, you’d make some effort to make your app accessible by a screen reader.
- mmmmmbop 5y ago
  I disagree. For example, I've found Google Docs' offline functionality extremely useful, since it allows me to continue working on documents while traveling on a plane or a train with no data connection.
begueradj 5y ago
Just before yesterday, I read an article here praising the "Offline First" principle. And as for everything related to software, I am reading again what was good is bad, and what was bad is good. All you have to do for a successful clickbait is to wait until someone shares his opinion so that you can finally write something against it... (most of the time, just for the purpose of saying you're against)
- esrch 5y ago
  The other article you mention comes from the same website: https://rxdb.info/offline-first.html https://rxdb.info/offline-first.html. It's in an opinion section, giving the arguments from both sides.
  - typingmonkey 5y ago
    Yes. I have written both of them, mostly to make sure I trigger 100% of people, no matter if they like offline first or not. I mean, read all these comments on both articles. People say offline first does not work, then they say that every software is offline first by default and it is nothing new. It was totally worth it spending two 3 days on it :)
    - somenewaccount1 5y ago
      And you know what, I AM triggered! I feel like you missed the most important point of "offline first" is that your data belongs to you and does not need to be shared to the cloud in order to have tremendous value. The code/logic to enhance your data should be shipped to you, rather than the other way around.
      - typingmonkey 5y ago
        That gave me a smile :) Yes, I totally missed this side of the dice. Having your data stored locally, being in control, being able to reset the state to it's origin, export the local state as json and import it somewhere else, knowing what is stored and how long.
akkartik 5y ago
I remember the good old days when we had offline first by default. We called them just "computer programs". Offline first as a principle is more important than web apps. If today's browsers have trouble with offline first, consider that a downside of today's browsers.
- srcreigh 5y ago
  The problem with browsers is Apple. A decent iOS browser would handicap their App Store ecosystem. If the browser was as powerful as iOS, developers would build for iOS and Android using JS, and Apple wouldn't get to reject apps or earn their cut of all payments. Because of Apple, there has to be a severe handicap somewhere, so that nobody can build real cross-platform apps in the browser. That's why iOS browsers can't use APNS/push notifications or reliable local storage, but iOS apps can. Yes, every other browser supports these things, even macOS web browsers. [1] [1]: https://developer.apple.com/notifications/safari-push-notifications/ https://developer.apple.com/notifications/safari-push-notifi...
lytefm 5y ago
I've been working on offline-first apps (CouchDB/PouchDB + Cordova/Capacitor and published via App/Play Store) in the last years and can definitely relate. But some points to add: - The 7 days IDB limitation does not apply for apps that are published through the stores - Conflicts can happen, but depending on your design they might not matter in practise. „Implement a proper conflict resulution strategy“ has been on my „todo: maybe“ list for over 3 years now but was never important enough. - Data migration is not needed as long as schema changes are additive (new doc fields, new doc types). Design carefully early on, keep track of „abandoned“ properties and you'll rarely need a difficult migration. - Depending on the performance of your customers' phones and the amount of data your app is processing, it (JS -> ... -> IDB and back) might not be fast enough. I had to add caching layers for some use cases. But at some point, you probably want a proper state management library anyways which should include caching nearly for free. - You can (and should!) still consider most of your data relational. There is even a relational-pouch plugin. But I'm strongly missing foreign key constraints and better DB-level data validation than CouchDB's design docs provide.
ngrilly 5y ago
Many of the problems mentioned in the article are solved by doing offline first with a native app instead of a PWA.

346 points by typingmonkey 5y ago

The most confusing thing to me is a discussion of "offline first" applications which starts with, and maintains, the assumption that your only option is a web app. Back in my day we had a word for software that always worked without an internet connection. We called it "software" and it was installed on the user's computer.

hengheng 5y ago
I will still call any web app a "web site", and if that is my age showing, so be it.
- taneq 5y ago
  We can shout at the cloud together.
- atatatat 5y ago
  That's fine, if it's on your tablet or PC. If you're looking at a web "site" on a vertical phone screen, one side or the other did something wrong. My point is: sites and services should have two separate experiences, depending on screensize. These are: desktop site, mobile webapp.
- hoarad 5y ago
  i do so also. and it is because they have a link
tehbeard 5y ago
Har de har look at these modern programmers with their web 2.0s and js frameworks of the week... Offline first is not "software" as you classified it. Your "software" you are on about is more accurately described as "offline only", little to none of the functionality requires network access. Offline first refers to how online functionality is needed for the design of that app, but with steps taken to ensure that even without connectivity for periods of time, it still functions.
jjnoakes 5y ago
While offline first seems to be discussed in the context of web apps a lot, to me it is more about the data and synchronization than where the executable lives, and most of the ideas also apply to certain kinds of desktop software.
dspillett 5y ago
That isn't offline first though: desktop software like that is generally offline only, and the user wraps their own chosen sync method (which could simply be good ol' sneaker-net or frizby-net) around that if they want/need to. Offline first is only used in the context of web applications and sometimes their Android/iOS cousins (which probably share the same backend, where both are available), once the decision had been made that a not-locally-installed and/or remote synced application is desirable where possible, so isn't being suggested (directly) as an alternative to locally installed offline programs.
jasode 5y ago
>The most confusing thing to me is a discussion of "offline first" applications which starts with, and maintains, the assumption that your only option is a web app. I didn't downvote your comment but in this author's article, it's deliberate for the starting context for discussion to be a networked collaborative app. Yes, internet connected apps is a subset of all possible software but that's not the point. As an analogy, imagine if someone else submitted an article about C Language memory techniques on an embedded chip. E.g.: https://www.embedded.com/memory-allocation-in-c/ https://www.embedded.com/memory-allocation-in-c/ And then a commenter misunderstands that article complaining, "it's confusing to me because this article maintains that the only option is C in an embedded app but over here, I'm using Python with Cloudflare Workers" In other words, it doesn't seem like you're interested in collaborative apps that require distributed data consistency so this article looks "wrong" to you.
- phkahler 5y ago
  It's confusing because "offline-first" doesn't even seem to make sense in the context of web-apps, which I thought meant "fancy (functional, able to do stuff) web sites" or similar.
- josephg 5y ago
  I’ve been working in this problem space for awhile now and I sort of agree with the GP poster. I really like native software; and I want native software which can work simply in a distributed, collaborative context. As an example, I have a note taking app on my laptop. I want to be able to read and edit all my notes on all my other devices. And I want that to work in a way that doesn’t depend on some random startup keeping their servers on the other side of the planet running. Right now every software company which wants to build something like this needs to invent their own data stack, network protocols and storage systems. And the prize at the end is with software that can only talk to itself via closed protocols. It’s infeasible, and inefficient. We have an opportunity right now to do an awful lot better. When we do, I want to service both native and web apps. If we do it right, from the network level the distinction should just boil away anyway.
  - 5y ago
psychometry 5y ago
Ok, and if you want your non-web app to be compatible with Windows, MacOS, iOS, and Android and you don't want to write more than one app, your options are...?
- justinclift 5y ago
  Generally Qt.
- poetaster 5y ago
  Qt/qml works for. With python sometimes.
Swenrekcah 5y ago
A user’s computer? Written with a possessive? What a bizarre idea!
jitl 5y ago
It is approximately infinity times easier to distribute and grow the user base of a web app compared to locally installed software. The consumer software economy is moving online for this reason - it’s much better for business. RxDB software comes from this context - it’s a JavaScript library built for this world that attempts to retain the distribution and sharing advantages of the web, while adding back the responsiveness and availability of traditional installed software. I think you’ll find the original “local first” manifesto more aligned with both the user & traditional installed software with less of a web focused bias: https://www.inkandswitch.com/local-first.html https://www.inkandswitch.com/local-first.html > In this article we propose “local-first software”: a set of principles for software that enables both collaboration and ownership for users. Local-first ideals include the ability to work offline and collaborate across multiple devices, while also improving the security, privacy, long-term preservation, and user control of data.
blacktriangle 5y ago
Also back in that day 95% of our users were on Windows and we had a direct relationship with them. Now your user base is roughly split 40/40 between iOS and Android with a non-ignorable 20 running some windows tablet thing. Then for extra fun your access to those users is mediated by the giant black boxes staffed by assholes that are the iOS and Play stores. And of course back then nobody had any expectation that your software would easily sync between devices and users because that just wasn't something that was doable easily. The world changes. Yeah it was better for us devs back then, but honestly the new world has some real advantages. For me personally, I'm cautiously optimistic that PWAs are our way out of the hell that is supporting 3 native platforms for all but the rarest cases.
- lytefm 5y ago
  For me, PWA doesn't quite cut it yet but using a single codebase based on web technology + adding the appropriate wrappers for native functionality (Electron, React Native, Capacitor, whatever...) is fine for now.
- poetaster 5y ago
  Qt/qml works for me. Write once.
JamesSwift 5y ago
The difference is that back in the day it _only_ worked on your computer. There was no cloud component and no collaborative use-case. If you only have a single, local client then a lot of this doesn't apply. But its rare for that to be the case in modern software.
blondin 5y ago
i agree with the essence of your comment. the "back in my day" is what is, perhaps, causing resistance. my impression these days is that web engineers have outnumbered other software engineers. that's a problem because context matters as you are alluding to. we shouldn't use terms like "offline first" without an appropriate context. or assume context.

candiddevmike 5y ago

This is a good list. The app I built (https://about.homechart.app https://about.homechart.app) is "read only" offline first, it's a compromise I chose over having to solve queuing writes and consistency checks during the (assuming rare) occurrences users find themselves offline. I'd add write support, but no one has asked for it. Don't go into offline first thinking you need to solve for writes, it can be added later if necessary.

JamesSwift 5y ago
The write queue is messy but not hard as long as you are serializing all the operations. The real difficult part is conflict resolution. If you have an easy resolution model (e.g. last one wins) then I think its not too much extra work for you.

megamix 5y ago

Humans first. How about that for an idea.

lovemenot 5y ago
Not great to be honest. I cannot imagine how to use this idea to support or reject a decision. Maybe as a slogan or sales pitch... Reminds me of Fujitsu's Vision Statement: Human-centric Computing. I asked a dozen employees of that company what it meant, and not one could give a coherent answer
olah_1 5y ago
Idk why you're being voted down. It's a genuine design principle. Some ideas that come to mind that fit with "human first": - offline-first - human curation instead of algorithms (or at least transparent algorithms that are customizable) - the user is not the account number. the account is just the mech that the user climbs into. user can have multiple mechs. - leverage existing social fabric to provide better user experience. account recovery, etc.

ltearno 5y ago

I once made an app and a talk on this matter. Slides about problems mentionned in the article were addressed from slide 15 onwards. Mainly I remember having used Lamport clocks to track rows' causal history. And negative primary keys for inserting data in offline mode (these days I'd use UUID). Since IndexedDB was not a thing yet on every browser, I used asm.js (ancestor to WebAssembly) to compile SQLite for the browser. The database file was stored in the LocalStorage and I used zlib (compiled with emscripten) to make most of the little space LocalStorage gave to us. It has been a learning experience, and worked at the end. I wonder how I would do differently today... By the way, we were using GWT to code for the browser, but that's just an anecdote and not important for that matter... Here is the link : https://fr.slideshare.net/ltearno/easing-offline-web-application-development-with-gwt https://fr.slideshare.net/ltearno/easing-offline-web-applica...

leetrout 5y ago
Thanks for sharing. I do not miss GWT but I think it was inspirational.
JamesSwift 5y ago
Ive also gone the negative key route, and would move to a "client dictates the key via UUID" like you mention if I redid it today.
- mamcx 5y ago
  Why? I have done this before and found negative much easier to use and understand. UUID have the nasty property that deny easy debugging, logging or any kind of HUMAN understanding...
  - JamesSwift 5y ago
    The primary benefit is theres no post-hoc reconciliation you need to do where you re-write all the foreign keys and object ids. It greatly streamlines the entire process, and lets you eliminate a lot of code / potential bugs.

jcun4128 5y ago

I made a PWA that used IndexedDB and base64 photos. I ran into a funky max-length issue that was an error from Chromium kind of interesting. But yeah the main problem I had was the base64 images would get too big (if you had too many to load) and then you would see a slower render vs. an image pulled by url. Still pretty cool since I'm not a native developer and RN is something I've dabbled in but don't use daily.

jitl 5y ago
These days IndexedDB supports storing binary blobs as File objects or even ArrayBuffers; the relevant bug on the Chrome issue tracker was marked fixed in 2014: https://bugs.chromium.org/p/chromium/issues/detail?id=108012 https://bugs.chromium.org/p/chromium/issues/detail?id=108012
- jcun4128 5y ago
  Interesting I think was straight up using just text or whatever is the default. Though I was using the Dexie wrapper which is nice. To be clear I don't know if the bug was from IndexedDB it was something about length exceeded. I did try to use small images too eg. some 150px by 150px but going to base64 usually multiplies the size by 1.3 For anyone interested [1] not a phenomenal app but one I poured idk 2-300+hrs into (was working on an RN version too) and went nowhere sucks. I was contributing to one of those codefor# deals. [1] https://github.com/codeforkansascity/tagging-tracker-alt-apps/blob/master/src/App.js#L143 https://github.com/codeforkansascity/tagging-tracker-alt-app...
ngokevin 5y ago
Perhaps Service Workers is better than IDB for that?
- jcun4128 5y ago
  I'll keep it in mind. I used creat-react-app and the built in way to do PWA, although they did update that so I had to do some code updates... some Box thing I forget. (to get cached files to work) Still it's cool to store everything as text/blobs locally.

psychometry 5y ago

It's infuriating to me that WebSQL was killed. 99% of real-world data is relational and yet the powers that be decided that we should all be forced to use IndexedDB and hacky layers like PouchDB built atop it. I'm excited about https://github.com/jlongster/absurd-sql https://github.com/jlongster/absurd-sql though.

dragonwriter 5y ago
WebSQL wasn't killed by opposition to having a relational API, it was killed because the spec was tied to, and only implemented by embedding, a specific, identified version of SQLite.
- EvanAnderson 5y ago
  I think that a developer distaste for relational databases was a major driver. Digging back into correspondence on this a few months ago (when this came up on HN) I found clear statements that Mozilla opposed anything relational. The SQLite version is a convenient excuse for some developers who, at the time, we're enamoured with "NoSQL". Discussion here: https://news.ycombinator.com/item?id=28156831 https://news.ycombinator.com/item?id=28156831
  - WorldMaker 5y ago
    Mozilla for a long time backed their IndexedDB with SQLite, they wouldn't have done that if they were that antagonistic to relational databases. I trust Mozilla's surface reasons here: they inherited the mess that was NPAPI from Netscape, then decades of experience with XUL binary components, were among the many dealing with Flash bugs and zero-day fallout well after Flash's "heyday", and have combined multiple decades of experience in what happens if the web depends on specific binaries to do its job. From that standpoint of they were already knee deep in trying to sandbox/reign in NPAPI, remove XUL, and remove Flash I absolutely understand why "you want the web to depend on the bugs and zero days of SQLite directly with no abstraction layer between?" was a complete non-starter.
- ec109685 5y ago

hdjjhhvvhga 5y ago

An alternative: a native app.

endisneigh 5y ago
native to what?
- j1elo 5y ago
  The word "native" has diluted in a sea of abstractions and supporting technologies, but in this context I'd read "native app" as removing the browser engine layer (so, not even really native by far, but much closer to the OS and hardware than when writing code on top of the mentioned layer)
  - hdjjhhvvhga 5y ago
    Yes. Putting an app in a browser has solved tens of problems and introduced a hundred new ones. At some point you start to wonder if a plain old native app wouldn't work much better that an uncontrollably growing stack of technologies, some ow which designed for something completely different than what we're trying to accomplish.
    - hunterb123 5y ago
      Okay. You still need a client side database with syncing. Which is what this is about. The pure native approach only solves (kind of) the limited storage issues, nothing else. If you nuke your site and only develop about native apps then you'll lose users to competitors because people can try their service without downloading an app. Yeah maybe we shouldn't have shoved an app run-time into a document viewer, but here we are and things run decently well (thanks to v8 and other tech) if you do things right.
      - hdjjhhvvhga 5y ago
        I think you underestimate the number of issues a (purely) native app solves. The fact that many developers don't care about some of them it does not mean they don't exist. One of them is shorter delay, very important in interactive applications. Syncing can be implemented in billion ways, from dedicated solutions to custom ones. Paradoxically, I think Microsoft got it (somewhat) right with Office. You can use a native C++ app, and this is what most people use because of speed and control. However, if you like working online, don't mind saving your documents unencrypted on Microsoft's systems, don't worry about being offline, you need collaborate with others online, or just like the convenience of having your documents always online, you can use the 365 version with inferior but probably acceptable performance - in this way everybody is happy.

josephg 5y ago

I’m a “true believer” in CRDTs, which I have some experience in. You can implement a useful CRDT for simple applications in under 100 lines if all you care about are standard database objects - like maps, sets and values. List CRDTs are where they get complicated, but most applications aren’t collaborative text editors. The promise of CRDTs is that unlike most conflict resolution systems, you can layer over a crdt library and basically ignore all the implantation details. Applications should (can) mostly ignore how a CRDT works when building up the stack. The biggest roadblock to their use is that they’re poorly understood. Well, that and implementation maturity. Automerge-rs merged a PR the other day which brought a 5 minute benchmark run down to 2 seconds. But by bit we’re getting there.

a_conservative 5y ago
Can you point to a good introduction to Conflict-free replicated data types (CRDT)? It's crossed my (short attention span) radar a couple times and seems interesting. I really love the idea about being able to use it, but also "forget" about it while developing. Does "leaky abstraction" apply here? If you constrain things enough, can a dev really use it and forget about the details?
jitl 5y ago
I work on a collaborative text editing system (https://www.notion.so https://www.notion.so) and read a lot about CRDTs, thank you for your work in this area, particularly https://josephg.com/blog/crdts-go-brrr/ https://josephg.com/blog/crdts-go-brrr/ I agree with your assessments here - CRDT is the way forward for most applications; no user wants to fiddle with a merge UI or picking versions like with iCloud. I think RxDB’s position here is from their CouchDB lineage. > The biggest roadblock to their use is that they’re poorly understood. Well, that and implementation maturity. I certainly have more understanding to do. My biggest open question is how to design my centralized server side storage system for CRDT data. To service writes from old clients I need to retain a total history of a document, but I don’t want to force new clients to download the entire history nor do I want these big histories in my cache, so I end up wanting a hot/cold system; and building that kind of thing and dealing with the edge cases seems like more than 100 lines of code. It seems like the Yjs authors also recognize that CRDT storage on the server is an area to address, there was some work on a custom database in 2018, although my thinking is more about how to retrofit text CRDTs into my existing very conservative production cloud software stack than about writing to block storage.

kevincox 5y ago

I do find it disappointing that there is no reliable local storage system for the web. I get the resistance to trackers but there should be a way to request permission to store data that isn't deleted except for explicit action by users. It means that you effectively have to store the data in "the Cloud" which means 1. I have to pay for it 2. If I shut down the service you are screwed and 3. I have at least some access to it (encryption aside). I would also like to see a synced version since most browsers these days support syncing settings and passwords. But creating a generic syncing solution that is actually useful is hard.

richardwhiuk 5y ago
Can you allow the user to load and save files?
- kevincox 5y ago
  I mean sure, you can get the user to download/upload files, but this is very awkward and not suited for storing every change. (But is good for migrating and backing up the data). Having to get the user to manually break the data out of the browser is not a good UX for day-to-day work. I know that Chrome is pushing for a filesystem API but I don't know if that will be exempt from the usual ephemerality. IIRC it is just a private storage space with a filesystem-like API.
  - easrng 5y ago
    Chrome has an API that allows you to save to files without a new download every time. They made a pretty nice library that wraps that API on Chrome and it gracefully degrades on other browsers. https://web.dev/browser-fs-access/ https://web.dev/browser-fs-access/
    - jcims 5y ago
      i noticed this behavior in a drawing app called excalidraw. first save opens a file dialog, subsequent saves just update the file, basically like a standard local text editor. i keep doing 'save as' to create new files because i don't trust it lol

deleted 5y ago

[deleted]

LeanderK 5y ago

I am a strong believer in PWAs. I think most of the apps I use could be PWAs without a problem. I really don't get why Apple isn't developing them for MacOS, since I've never really used the app-store on MacOS and in comparison to iOS a lot of the apps on my Mac are productivity apps that are basically electron-apps. I think some basic PWA functionality on the desktop would be more interesting for me than more advanced PWAs for ios.

jdavis703 5y ago
I want to believe in PWAs. But in the real world they are just way too clunky. I’ve observed this on both software I’ve written, and on PWAs from teams that should objectively know what they’re doing like Google Calendar.
atatatat 5y ago
PWAs lead to the walls of Apple's walled garden being torn down. They'll fight it with subtlety until the end, and I hope they burn in hell for it.
- blauditore 5y ago
  Exactly. They introduced auto-deletion of localStorage under the hood of "privacy", when it was really about driving people to building standalone apps (and use their store).
  - breakfastduck 5y ago
    Good. PWAs are absolutely awful compared to native apps.
    - catlifeonmars 5y ago
      What don’t you like about PWAs?
breakfastduck 5y ago
Because native apps are basically a huge chunk of the appeal to using macOS and PWAs are the absolute antithesis of that. Encourage developers to make apps that all look/feel completely different in terms of style/UX - no thanks.

yanis_t 5y ago

PWAs are the future. Along with other benefits comes the fact that you can "install" apps on devices bypassing the app stores (which means not paying commission fees), which I guess is the primarily reason why Apples is so reluctant of giving it a proper support in Safari.

JamesSwift 5y ago

Its a good article but it should be noted that this very much is a web-centric view of Offline First and its challenges. When I say web-centric, I mean as opposed to offline-first on a mobile app. Native apps dont deal with the same issues around storage, and are actually _much_ more performant overall. If you haven't done an offline-first app, I highly recommend it. The experience is magical. You can fly around your app at the speed of the users touch. Content is magically loaded as soon as its clicked on. Its amazing as a user. As for similarities, conflict resolution is a universal problem and is either the most difficult or second most difficult problem [1] to solve. What makes this more difficult is that there is no one-size-fits-all for this. You need to have a deep, nuanced understanding of your system and what makes sense for your use case in terms of resolution strategies. Then you need to implement them, which is not easy especially if your backend is bog-standard REST on a classic SQL datastore. I've enjoyed reading the past 2 rxdb articles on this (the one mentioned here as well as the one from a couple days ago [2]). Its great to have more content on this publicly available, when I was getting into offline-first I only had a couple options. [1] - At the end of the day, offline-first is a whole bunch of caching, and so you end up needing to deep dive into cache invalidation strategies which we all know is a hairy problem. [2] - https://news.ycombinator.com/item?id=28690427 https://news.ycombinator.com/item?id=28690427

Trufa 5y ago
Yeah,I'm trying to implement an electron offline first app that syncs, there seems to readymade solution. Stuff like https://github.com/aerogear/offix https://github.com/aerogear/offix seem to be in the right direction of what I'm looking for but not nearly mature enough. I don't want to pu to much effort on the app so I would like something more or less ready made, preferably with graphql apis. Any suggestions welcome.
- JamesSwift 5y ago
  I would hesitate to use any client-server setup that isnt also your primary data store. So, if you have an "offline aware, multi-tenant" store like CouchDB but you still need to sync to the primary store which is SQL, then you lose a lot of context and awareness around conflict resolution. If you are going to eventually sync to the other store, I would say only use Couch on the frontend, and do the syncing/resolution from the perspective of the client, since the client knows what it was trying to do and how best to notify on conflict. The "better" option is to have an offline-aware client-server component (e.g. Realm) which is the primary store as well. This eliminates the sync and so all conflict resolution stays in the same system with well defined semantics.

endisneigh 5y ago

In practice I believe last write wins or compare last two writes is sufficient. Any thoughts on this in practice?

sroussey 5y ago
Last write wins is a simple CRDT merge type. It’s just that their are others depending on the data type. LWW might be good for an avatar photo, but not great for a counter.

twobitshifter 5y ago

Good article. For structured data, I’ve had good luck using UUIDs for keys rather than needing an approach that relies on an atomic clock.

ItsMonkk 5y ago

The way I've learned to use git is to 0. Sync with remote 1. Edit files until I'm ready to check-in 2. Stash changes 3. Sync up with latest from remote 4. Pop changes 5. If there are any conflicts, deal with them here, locally. Possibly delete all changes and redo. Redoing is equivalent to someone checking in something at step 0. If this takes some time, move back to Step 2. 6. Push changes If done this way, the only benefit of CRDT's is during step 5. One of the lessons I've learned and truly believe is that if something sucks, and it gets worse as the problem gets bigger, you need to do it more often. Git merges are a great example of this concept. And this is where CRDT's are in trouble. The best way to make step 5 easier is by making step 1 smaller. CRDT's viewpoint is that we are offline, and therefore should allow any number of edits, and when we reconnect the system should be able to work it out. It flies in the face of smaller commits. The more changes you need to merge, the harder it is. We live in a world that's connected 99% of the time, and CRDT's simply aren't needed. On the other hand, the "Offline First" model is great! A user having access to all of their data is wonderful. As the blog notes, it's not preferable for a user to have the entire Wikipedia or Google indexes on their device. So you need to have a use-case for Offline, and we need to do better about this type of stuff, but we don't need to wait on CRDT research for this. Smart caching and materialized views are where I think the real progress is going to come from, making things like Offline Wikipedia possible.

austincheney 5y ago

> When you create a web based offline first app, you cannot store data directly on the users filesystem. In fact there are many layers between your JavaScript code and the filesystem of the operation system. Solved: File system in the browser plus network distribution - https://github.com/prettydiff/share-file-systems https://github.com/prettydiff/share-file-systems

thayne 5y ago

> in the end the user itself could be the one that deletes the browsers local data This is especially true, because often customer support for many websites have customers begin troubleshooting with clearing the browser's "cache and cookies". In other words deleting all of the local data for all sites. There are ways to delete the local data for just one site, but they are pretty hidden, and involve multiple steps. I wish browsers had a simple "delete all data for just this site" button.

ranger207 5y ago
Firefox does! You can "clear cookies and site data" by clicking on the padlock. I use it often to get around Reuters' new paywall that kicks in after a few articles

jadafaa 5y ago

Enjoyed reading this

lambda_dn 5y ago

It's not worth it, have a internet connection is more ubiquitous every day with Wifi, 4/5G and coming soon Low orbit satellite grids. Trying to engineer your app to work offline causes complexity in the design and implementation for a issue that might never be an issue for most customers. Assuming your app is pretty crippled when it can't access the cloud. What's next making your app still work if there is no display by screen reading?

lytefm 5y ago
This statement very much depends on the kind of app you're developing, as mentioned in the article. Is the main use case communicating with another user? Do you need a third party API for your app's basic functionality? Sure, don't even consider offline first. But what if the app is only about storing and displaying data entered by the user and you'd definitely also want to be able to use it on an airplane, e.g. a todo/notetaking/journaling app? Then offline first can make sense. I'd even say that the complexity of developing an offline-first app can be lower than that of a classical client/server app + caching logic. Sure, you need to figure out how to do schema migrations and you probably want a kill switch to lock out older apps at some point. But the same applies in a classical setting when API-endpoints should be changed or removed. Basically, your document model is now your API. And a very simplistic conflict resolution strategy like „just take the most recent version“ is often good enough. Once that + basics like auth and account creation are set up, it's very productive and low overhead to work with PouchDB/CouchDB and offline first: - no need to coordinate with a backend team because nothing else than auth happens there - simpler state management and error handling because the state in the local DB can always be assumed to be correct and the DB is always there - no need for schema migrations as long as you only add new docs or extend existing ones - it's great for quickly hacking a prototype or for beginners who just know some HTML/CSS/JS
jamil7 5y ago
Ideally yes, you’d make some effort to make your app accessible by a screen reader.
mmmmmbop 5y ago
I disagree. For example, I've found Google Docs' offline functionality extremely useful, since it allows me to continue working on documents while traveling on a plane or a train with no data connection.

begueradj 5y ago

Just before yesterday, I read an article here praising the "Offline First" principle. And as for everything related to software, I am reading again what was good is bad, and what was bad is good. All you have to do for a successful clickbait is to wait until someone shares his opinion so that you can finally write something against it... (most of the time, just for the purpose of saying you're against)

esrch 5y ago
The other article you mention comes from the same website: https://rxdb.info/offline-first.html https://rxdb.info/offline-first.html. It's in an opinion section, giving the arguments from both sides.
- typingmonkey 5y ago
  Yes. I have written both of them, mostly to make sure I trigger 100% of people, no matter if they like offline first or not. I mean, read all these comments on both articles. People say offline first does not work, then they say that every software is offline first by default and it is nothing new. It was totally worth it spending two 3 days on it :)
  - somenewaccount1 5y ago
    And you know what, I AM triggered! I feel like you missed the most important point of "offline first" is that your data belongs to you and does not need to be shared to the cloud in order to have tremendous value. The code/logic to enhance your data should be shipped to you, rather than the other way around.
    - typingmonkey 5y ago
      That gave me a smile :) Yes, I totally missed this side of the dice. Having your data stored locally, being in control, being able to reset the state to it's origin, export the local state as json and import it somewhere else, knowing what is stored and how long.

akkartik 5y ago

I remember the good old days when we had offline first by default. We called them just "computer programs". Offline first as a principle is more important than web apps. If today's browsers have trouble with offline first, consider that a downside of today's browsers.

srcreigh 5y ago
The problem with browsers is Apple. A decent iOS browser would handicap their App Store ecosystem. If the browser was as powerful as iOS, developers would build for iOS and Android using JS, and Apple wouldn't get to reject apps or earn their cut of all payments. Because of Apple, there has to be a severe handicap somewhere, so that nobody can build real cross-platform apps in the browser. That's why iOS browsers can't use APNS/push notifications or reliable local storage, but iOS apps can. Yes, every other browser supports these things, even macOS web browsers. [1] [1]: https://developer.apple.com/notifications/safari-push-notifications/ https://developer.apple.com/notifications/safari-push-notifi...

lytefm 5y ago

I've been working on offline-first apps (CouchDB/PouchDB + Cordova/Capacitor and published via App/Play Store) in the last years and can definitely relate. But some points to add: - The 7 days IDB limitation does not apply for apps that are published through the stores - Conflicts can happen, but depending on your design they might not matter in practise. „Implement a proper conflict resulution strategy“ has been on my „todo: maybe“ list for over 3 years now but was never important enough. - Data migration is not needed as long as schema changes are additive (new doc fields, new doc types). Design carefully early on, keep track of „abandoned“ properties and you'll rarely need a difficult migration. - Depending on the performance of your customers' phones and the amount of data your app is processing, it (JS -> ... -> IDB and back) might not be fast enough. I had to add caching layers for some use cases. But at some point, you probably want a proper state management library anyways which should include caching nearly for free. - You can (and should!) still consider most of your data relational. There is even a relational-pouch plugin. But I'm strongly missing foreign key constraints and better DB-level data validation than CouchDB's design docs provide.

ngrilly 5y ago

Many of the problems mentioned in the article are solved by doing offline first with a native app instead of a PWA.