5 ms·

What has been your experience writing the "off line first" style PWA? There's an appeal to it, basically using the web purely as distribution, and the browser

What has been your experience writing the "off line first" style PWA? There's an appeal to it, basically using the web purely as distribution, and the browser as the runtime. But I'm concerned with the fiddly-ness of local storage and such like that. The "out of site, out of mind" nature of it. The data not being in a "~/.app/app.dat" file, etc. The idea of it potentially just up and vanishing with a browser update. And, heck, just the complexity of dealing with the schema versioning in the native web model. As well as the portability of data (say when you copy over to a new computer). The idea of simple web distribution is compelling. Now you need nothing more than a github account, and off you go. No server, no nothing. But I still feel (perhaps ignorantly) that the data situation is still on shaky ground. (Discounting the whole lack of something like SQLite, etc.) And, the idea of bundling something like SQLite as a webassembly blob just makes me itch. Just curious how that's worked out.

winrid 2y ago
This was my concern with building a PWA, which is why SidewaysData is a native desktop app. The downside of course is updates, but that's manageable.
- cyanydeez 2y ago
  If you have a.desktop app, zetup a webrtc connection and you can create a teathered.webapp in PWA.
  - winrid 2y ago
    SidewaysData needs to be able to work offline. It also needs access to serial/USB ports. Also, I prefer Java to TS etc anyway.
tithe 2y ago
It seems it's using `better-sqlite3`[0] and saving the database to a file called "private/unforget.db"[1]. (Although, I don't have experience with SQLite, so that file might be as volatile as `localStorage`.) [0] https://www.npmjs.com/package/better-sqlite3 https://www.npmjs.com/package/better-sqlite3 [1] https://github.com/computing-den/unforget/blob/master/src/server/db.ts https://github.com/computing-den/unforget/blob/master/src/se...
- seansh 2y ago
  Yes that's the database on the server for storing encrypted notes and syncing. My experience with SQLite has been very positive. It really is rock solid. Also, wanted to mention that Unforget doesn't use browser's localStorage. LocalStorage has a low limit and the browser can nuke it when it wants. Instead we use IndexedDB which has much higher limits and persistent.
seansh 2y ago
There's definitely a lot of fiddlyness when it comes to the web stack. Having said that, indexeddb has been very reliable. There are some annoying limitations though. For example, if you want to create an index on a store using multiple keys, you can't specify which key should be ascending or descending. If you add the PWA to your home page, i.e. install it, then the browser must not clear indexeddb on an update or something. And as far as I can tell, browsers do respect that. My overall experience building PWA has been mixed. I think it really depends on what app you're making. For a note taking app, where the data can be backed up to the cloud encrypted and synced, I really think a PWA is the best option. But if you want tighter integration with the platform e.g. accessing the file system, then I would stay away from PWA. Another issue is ios safari which is missing a lot of features for PWAs and I think unlikely to support them in the future. E.g. background sync, offline push notifications, etc. Are you considering PWA for any particular app?
- whartung 2y ago
  I'm just writing little stuff in Java, and use that as my x-platform environment, but even it has its challenges. So, I was considering a PWA. I don't need tight local integration per se. I do have a conceptual problem not being able to save a file or read a file, even if it's just chosen by the user. I THINK I can do that. I THINK you can pop a file chooser to read a file, but not necessarily just open any file you want. If you can read/write a file as directed by a user, that would be fine. I can see the rest being stored in the IndexDB. And that gives the option of an export/backup if it becomes necessary for peace of mind. Then there's the whole x-browser game, I'm not really looking at mobile, just desktop, but I'd like to support FF/Chrome/Safari/Edge. It's not rocket science stuff by any means. But I don't want a user to call up and talking about how their data suddenly vanished because they went to get the latest, and I did something innocent/ignorant (or not), or THEY did something (deleted their PWA icon, copied it, moved it, etc.), or the browser did something. I have no experience with it, so for now, I'm not comfortable because I don't have my "hands" on the data. If I can export it, then the comfort level goes way up. Inform the user to back up "just in case", especially early on when we're all still bumping in the dark. To be clear, I don't want a server side, I have no interest in standing up a service component. I have less interest in maintaining a service component. I'd like the code to be local and "standalone" as practical. But I also want something a touch more than "git clone app; cd app; ./build".
reitanuki 2y ago
> The idea of it potentially just up and vanishing with a browser update. Anyone writing applications in the browser should be very cautious to note that many browsers will jettison some types of their website storage if disk space is low. This is frustrating at the best of times, but if there's no server-side backup then frustration would be an understatement. I am not sure if this caveat applies to localStorage but from experience it does to IndexedDB — seemingly even in Electron applications (??).
- seansh 2y ago
  > from experience it does to IndexedDB — seemingly even in Electron applications (??). oh fun! I haven't run into that. But if you're using electron or any platform's local webkit wrapper, then it makes a lot more sense to forgo the browser's storage and directly use sqlite + server-side backup if possible.
balajics 2y ago
!The data not being in a "~/.app/app.dat" file, etc. It is possible to do this now in chromium based browsers. Refer https://developer.chrome.com/docs/capabilities/web-apis/file-system-access https://developer.chrome.com/docs/capabilities/web-apis/file...
lxgr 2y ago
I’ve been using Omnivore (a “read it later” app), which I otherwise love, but even though it’s an iOS app, offline support is one of its biggest weaknesses. As far as I know, it’s using some caching library at the HTTP level (as opposed to handling its own persistence at the application layer), and in falls apart every time I have a poor connection, remaining broken (even after fully going offline) until I’m back online.
dugmartin 2y ago
I think the safest offline storage is to use the browser file system (https://developer.mozilla.org/en-US/docs/Web/API/File_System_API https://developer.mozilla.org/en-US/docs/Web/API/File_System...) so the data is out of the browser's hands. It works pretty well with the only downside being the user needs to re-select the file/directory to use each time they use the app and there is no way to force the file picker to start in a specific directory other than the "well known directories".
- mayne 2y ago
  Now, you only need to select the directory once, and the browser will retain permissions https://developer.chrome.com/blog/persistent-permissions-for-the-file-system-access-api https://developer.chrome.com/blog/persistent-permissions-for...
  - dugmartin 2y ago
    Thanks for the heads up!

winrid 2y ago

This was my concern with building a PWA, which is why SidewaysData is a native desktop app. The downside of course is updates, but that's manageable.

cyanydeez 2y ago
If you have a.desktop app, zetup a webrtc connection and you can create a teathered.webapp in PWA.
- winrid 2y ago
  SidewaysData needs to be able to work offline. It also needs access to serial/USB ports. Also, I prefer Java to TS etc anyway.

tithe 2y ago

It seems it's using `better-sqlite3`[0] and saving the database to a file called "private/unforget.db"[1]. (Although, I don't have experience with SQLite, so that file might be as volatile as `localStorage`.) [0] https://www.npmjs.com/package/better-sqlite3 https://www.npmjs.com/package/better-sqlite3 [1] https://github.com/computing-den/unforget/blob/master/src/server/db.ts https://github.com/computing-den/unforget/blob/master/src/se...

seansh 2y ago
Yes that's the database on the server for storing encrypted notes and syncing. My experience with SQLite has been very positive. It really is rock solid. Also, wanted to mention that Unforget doesn't use browser's localStorage. LocalStorage has a low limit and the browser can nuke it when it wants. Instead we use IndexedDB which has much higher limits and persistent.

seansh 2y ago

There's definitely a lot of fiddlyness when it comes to the web stack. Having said that, indexeddb has been very reliable. There are some annoying limitations though. For example, if you want to create an index on a store using multiple keys, you can't specify which key should be ascending or descending. If you add the PWA to your home page, i.e. install it, then the browser must not clear indexeddb on an update or something. And as far as I can tell, browsers do respect that. My overall experience building PWA has been mixed. I think it really depends on what app you're making. For a note taking app, where the data can be backed up to the cloud encrypted and synced, I really think a PWA is the best option. But if you want tighter integration with the platform e.g. accessing the file system, then I would stay away from PWA. Another issue is ios safari which is missing a lot of features for PWAs and I think unlikely to support them in the future. E.g. background sync, offline push notifications, etc. Are you considering PWA for any particular app?

whartung 2y ago
I'm just writing little stuff in Java, and use that as my x-platform environment, but even it has its challenges. So, I was considering a PWA. I don't need tight local integration per se. I do have a conceptual problem not being able to save a file or read a file, even if it's just chosen by the user. I THINK I can do that. I THINK you can pop a file chooser to read a file, but not necessarily just open any file you want. If you can read/write a file as directed by a user, that would be fine. I can see the rest being stored in the IndexDB. And that gives the option of an export/backup if it becomes necessary for peace of mind. Then there's the whole x-browser game, I'm not really looking at mobile, just desktop, but I'd like to support FF/Chrome/Safari/Edge. It's not rocket science stuff by any means. But I don't want a user to call up and talking about how their data suddenly vanished because they went to get the latest, and I did something innocent/ignorant (or not), or THEY did something (deleted their PWA icon, copied it, moved it, etc.), or the browser did something. I have no experience with it, so for now, I'm not comfortable because I don't have my "hands" on the data. If I can export it, then the comfort level goes way up. Inform the user to back up "just in case", especially early on when we're all still bumping in the dark. To be clear, I don't want a server side, I have no interest in standing up a service component. I have less interest in maintaining a service component. I'd like the code to be local and "standalone" as practical. But I also want something a touch more than "git clone app; cd app; ./build".

reitanuki 2y ago

> The idea of it potentially just up and vanishing with a browser update. Anyone writing applications in the browser should be very cautious to note that many browsers will jettison some types of their website storage if disk space is low. This is frustrating at the best of times, but if there's no server-side backup then frustration would be an understatement. I am not sure if this caveat applies to localStorage but from experience it does to IndexedDB — seemingly even in Electron applications (??).

seansh 2y ago
> from experience it does to IndexedDB — seemingly even in Electron applications (??). oh fun! I haven't run into that. But if you're using electron or any platform's local webkit wrapper, then it makes a lot more sense to forgo the browser's storage and directly use sqlite + server-side backup if possible.

balajics 2y ago

!The data not being in a "~/.app/app.dat" file, etc. It is possible to do this now in chromium based browsers. Refer https://developer.chrome.com/docs/capabilities/web-apis/file-system-access https://developer.chrome.com/docs/capabilities/web-apis/file...

lxgr 2y ago

I’ve been using Omnivore (a “read it later” app), which I otherwise love, but even though it’s an iOS app, offline support is one of its biggest weaknesses. As far as I know, it’s using some caching library at the HTTP level (as opposed to handling its own persistence at the application layer), and in falls apart every time I have a poor connection, remaining broken (even after fully going offline) until I’m back online.

dugmartin 2y ago

I think the safest offline storage is to use the browser file system (https://developer.mozilla.org/en-US/docs/Web/API/File_System_API https://developer.mozilla.org/en-US/docs/Web/API/File_System...) so the data is out of the browser's hands. It works pretty well with the only downside being the user needs to re-select the file/directory to use each time they use the app and there is no way to force the file picker to start in a specific directory other than the "well known directories".

mayne 2y ago
Now, you only need to select the directory once, and the browser will retain permissions https://developer.chrome.com/blog/persistent-permissions-for-the-file-system-access-api https://developer.chrome.com/blog/persistent-permissions-for...
- dugmartin 2y ago
  Thanks for the heads up!