6 ms·

I don't think "Offline Is Just Online with Extreme Latency" is a useful concept because it doesn't encapsulate the key difference: pessimistic or optimistic UI.

I don't think "Offline Is Just Online with Extreme Latency" is a useful concept because it doesn't encapsulate the key difference: pessimistic or optimistic UI. For example, say you have a form. If you built it thinking online first you'll probably have some pessimistic UI which shows a spinner and waits for the server to respond with ok/error. You can't simply think, okay since we're offline, more latency -> show spinner for longer. You have to re-architect things so that the UI is optimistic, commits to a local database and that local database is synced up to the server when you come online. In my experience optimistic UI is way more complex to build. Many times the complexity is worth it though.

Karellen 3y ago
> You have to re-architect things so that the UI is optimistic, commits to a local database and that local database is synced up to the server when you come online. Isn't that exactly what the article argues for though? > This kind of idea would move you away from a product full of API calls to one based on data synchronization.
klabb3 3y ago
Those cases should both be solved with optimistic UI, so there’s no difference. You can have a little checkmark to indicate that it’s synced, like many chat apps do. > In my experience optimistic UI is way more complex to build. Many times the complexity is worth it though. Yeah, can attest to this. I’m working on an app[1] that strikes the trifecta: p2p, real-time and offline first. All of those things combined makes the amount of tooling, design patterns and resources available shrink to a tiny sliver compared to a typical web-based tech stack. I have researched probably 100 projects that sound promising but almost all have been a poor fit for one reason or another. I opted to build almost everything from scratch. Kudos to the JS ecosystem. They are way ahead in this space, with rxdb, watermelon, yjs, automerge etc. Unfortunately I couldn’t use any of them because I use a split-language stack. [1]: https://payload.app/ https://payload.app/
- dustingetz 3y ago
  to add to this - rotate CRUD mindset to CQRS, now you have a start event and an end event, this is the extent of the pending sync state which can be afforded to the user at any granularity - document level, message level, form or field level, etc. The cost is that other view queries don't reflect the pending event, only the view that issued the command has an association to the event. Which is usually the UX you want. Consider a master/detail form app where you insert a new record into a collection, but the view is sorted/filtered/paginated and your new record does not match the criteria and therefore vanishes. That is never the right UX. A less surprising alternative UX is to limit record creation to a specific form for the business rules of creation, and that form exists in the context of a specific collection view with business rules around newly created entities of that kind, with an invariant that newly created entities will always appear at the top or bottom of that collection (e.g. new messages are always added to the bottom of the chat history view). Now the optimistic update bypasses the database and simply writes through to the view optimistically and then when the ack is eventually received it seamlessly changes state without any UI jank. Now you can send many new messages rapidly and if the nth message fails and the rest of the messages went through you get a properly located error. With a CRUD mindset, probably the form is simply disabled until each individual insert succeeds - which means your chat app cannot keep up with you typing!
- silvestrov 3y ago
  When you need a single source of truth you cannot use optimistic UI. E.g. if the user is a realtor, then she can't tell the customer "you have now bought the house" if there is no online connection and you're waiting for a sync. You can fill out and upload the form async, but commitment must be online. You cannot tell the customer "you might have (or might not have) bought the house, we will only know later".
  - BrentOzar 3y ago
    > You cannot tell the customer "you might have (or might not have) bought the house, we will only know later". Sure you can - in fact, that's what realtors call offers. You place an offer, and you only find out later if you bought the house, often days later.
friendzis 3y ago
I agree with the sentiment, but for entirely different reasons. An offline capable application that sometimes tries to sync can be called "Online with Extreme Latency", but that is not what true ~~Scotsman~~ offline is. Offline/online is first and foremost about data locality and data ownership. In an online application (regardless of latency!) the source of truth is "the server". Offline application itself is the source of truth and "the server" is a slave. The OP seems to be talking about thin vs fat clients. Fat client is still a client - it fetches data. Offline application is the data source for "the server", it's the server that has to adapt to local changes not the other way around. Naturally, this is problematic since now you have multiple sources of truth. However, shifting source of truth to "the server" does create online application with extreme latency - fat client.
- inferense 3y ago
  > Offline/online is first and foremost about data locality and data ownership. not necessarily if the data is stored in a proprietary format.
  - Sakos 3y ago
    It doesn't matter if it's a proprietary format. Proprietary formats can be reverse engineered and modified and converted. There is a world of difference between a proprietary file that I have online in a form that I can't access directly (and I have no idea how it's being accessed, changed or stored by the service itself or third parties) and a file that I have on my harddrive that I have complete control over.
  - deleted 3y ago
    [deleted]
  - tnel77 3y ago
    Yes, but I think most people care about availability of data over true ownership of data. If I’m running a business and my internet goes out, there’s a huge selling point to a local DB that allows my operations to keep chugging along until Comcast figures out their mess. Internet returns, my DB syncs with a server offsite, and now I have that backup.
homero 3y ago
You'd have to make sure the user knew whether it finally worked even weeks later or never sent. They'll forget.
Lio 3y ago
I think you've got it spot on. I've also worked on systems like that where a mobile user could loose connection at any time and they are indeed very complex to get right. I'm not sure there's a perfect way to do it but we ended having have certain functions that had to be done online and others where the user built a "request" for service that was handled optimistically with failures sent to the user's inbox later.
hyperhopper 3y ago
Abstracting this away from the user is what meteor was built to solve, and did it very well for most use cases.
danenania 3y ago
The really hard part is conflict resolution. You updated something 5 times while offline, but another user deleted that something between updates 2 and 3, or made a divergent update of their own. There are so many potential scenarios like this in a multi-user app. It's a huge rabbit hole, and you can end up in situations where it's impossible to sync in a user-friendly way. Essentially you are taking on all the challenges of distributed databases. Maybe it's worth it, but you should know what you're getting into.
- giantrobot 3y ago
  Online-only collaboration systems already have to do this. The local view has to synchronize state between other local views and the canonical version on the server. Offline changes are online edits with high latency.
  - danenania 3y ago
    Yes, but the problem gets much harder to resolve the longer you allow client states to diverge.
simplotek 3y ago
> You have to re-architect things so that the UI is optimistic, commits to a local database and that local database is synced up to the server when you come online. I'm not sure you got the point. The design guideline that "Offline Is Just Online with Extreme Latency" already reflects very specific architectural requirements, and state transitions in the application life cycle. We're talking event-driven architectures, batching events, flushing events in offline/online transitions or even when minimizing windows, pulling events when going online, etc etc etc. I'd go even further and claim that this whole "pessimistic vs optimistic UI" thing is just "adequate vs broken UI design",regardless whether the app is even expected to go online.
saltcured 3y ago
A related concern would be surfacing the "online" part as explicit resources and actions for the user instead of implicit, background magic. If you are sufficiently offline in your lifestyle, you need to plan your communication phases. I recently got into using a GPS sports watch. It is the kind of thing I would want to use in an offline fashion, i.e. go somewhere off the grid and track my hikes or bike rides. These devices are designed to function offline for a stretch of time, but they have a requirement to eventually sync to an online system. They will eventually fill up with recorded sensor data and you want to offload that somewhere else before clearing local device storage. More importantly, satellite positioning receivers need a cached "ephemeris" file that helps them predict which satellites will be overhead at a given time to operate efficiently, accurately, and quickly. Unfortunately, the manufacturers have been infected with smartwatch expectations. They started designing it as if it is always online, and the syncing functions are implicit background behaviors. When doing sporadic syncs and going back offline, it is hard to influence it to "get latest ephemeris" when you known you have internet connectivity and will be going offline again. Worse, these results are localized and it implicitly gets ephemeris for its current location. The UI doesn't allow you to indicate a destination and pre-load the right data to allow fully offline function on arrival.
- deceptionatd 3y ago
  > More importantly, satellite positioning receivers need a cached "ephemeris" file that helps them predict which satellites will be overhead at a given time to operate efficiently, accurately, and quickly. Interesting, I've never heard of that particular optimization for GNSS before. I know GPS transmits ephemeris information in each frame, since that's the input data for the positioning calculations. I've got a number of Garmin watches, and they've always been able to get a position fix even after being disconnected for weeks. I find GNSS implementations very interesting, which manufacturer is making their watches like this?
  - saltcured 3y ago
    Garmin watches do! Go into the System -> About menu and there is a page showing ephemeris status. Their documentation states that this may expire after approximately 30 days or if you travel more than 200 miles, while it will update during syncing. I've seen it expire in less than two weeks with daily use of GPS but phone syncing disabled. It will still get position, but it can be the difference between an almost immediate fix after opening an activity menu or a delay for tens of seconds to minutes. Distance and pace measurements also seem to be lower quality when operating without a current ephemeris file.
thomastjeffery 3y ago
Sounds like it is a useful concept: you just used it to introduce your own. I think that optimistic UI isn't more complex to build: it's just less familiar. In a broader sense, UI/UX design culture places way too much focus and value on familiarity. The best UI/UX designs I have ever experienced are when software design diverges from the norm and tries a new approach.
- stepbeek 3y ago
  > I think that optimistic UI isn't more complex to build: it's just less familiar. Having multiple records of state that have to be resolved when systems come online is complex. Anything requiring consensus is difficult to do.
  - thomastjeffery 3y ago
    The same complexity is present in both approaches. The only difference is cadence.
    - ahzhou 3y ago
      Not exactly - there’s a bunch of non-trivial stuff you need to worry about when both local and remote states represent sources of truth. Things like CRDTs and event sourcing make it easier, but there’s still more complexity than dealing with only one source of truth. In an online-first experience, any local state being held is (for the most part) an optimization or is intentionally ephemeral. You can just toss it away at a performance penalty if it ever gets too hairy. In an offline-first experience, you need to be very careful that you treat everything like a source of truth. You also need to deal with schema migrations and business logic migrations, since you need those to partially live on the client.
treeman79 3y ago
Was required to use a MS sql server database for a project that would go offline for 5-10 minutes every hour. (Getting admins to fix it was a no go). Now of course I was judged on uptime of my app that relied on it. Finally just cloned the data to a local MySQL and refreshed data frequently. Database team totally clueless that app servers were hosting their own databases.

Karellen 3y ago

> You have to re-architect things so that the UI is optimistic, commits to a local database and that local database is synced up to the server when you come online. Isn't that exactly what the article argues for though? > This kind of idea would move you away from a product full of API calls to one based on data synchronization.

klabb3 3y ago

Those cases should both be solved with optimistic UI, so there’s no difference. You can have a little checkmark to indicate that it’s synced, like many chat apps do. > In my experience optimistic UI is way more complex to build. Many times the complexity is worth it though. Yeah, can attest to this. I’m working on an app[1] that strikes the trifecta: p2p, real-time and offline first. All of those things combined makes the amount of tooling, design patterns and resources available shrink to a tiny sliver compared to a typical web-based tech stack. I have researched probably 100 projects that sound promising but almost all have been a poor fit for one reason or another. I opted to build almost everything from scratch. Kudos to the JS ecosystem. They are way ahead in this space, with rxdb, watermelon, yjs, automerge etc. Unfortunately I couldn’t use any of them because I use a split-language stack. [1]: https://payload.app/ https://payload.app/

dustingetz 3y ago
to add to this - rotate CRUD mindset to CQRS, now you have a start event and an end event, this is the extent of the pending sync state which can be afforded to the user at any granularity - document level, message level, form or field level, etc. The cost is that other view queries don't reflect the pending event, only the view that issued the command has an association to the event. Which is usually the UX you want. Consider a master/detail form app where you insert a new record into a collection, but the view is sorted/filtered/paginated and your new record does not match the criteria and therefore vanishes. That is never the right UX. A less surprising alternative UX is to limit record creation to a specific form for the business rules of creation, and that form exists in the context of a specific collection view with business rules around newly created entities of that kind, with an invariant that newly created entities will always appear at the top or bottom of that collection (e.g. new messages are always added to the bottom of the chat history view). Now the optimistic update bypasses the database and simply writes through to the view optimistically and then when the ack is eventually received it seamlessly changes state without any UI jank. Now you can send many new messages rapidly and if the nth message fails and the rest of the messages went through you get a properly located error. With a CRUD mindset, probably the form is simply disabled until each individual insert succeeds - which means your chat app cannot keep up with you typing!
silvestrov 3y ago
When you need a single source of truth you cannot use optimistic UI. E.g. if the user is a realtor, then she can't tell the customer "you have now bought the house" if there is no online connection and you're waiting for a sync. You can fill out and upload the form async, but commitment must be online. You cannot tell the customer "you might have (or might not have) bought the house, we will only know later".
- BrentOzar 3y ago
  > You cannot tell the customer "you might have (or might not have) bought the house, we will only know later". Sure you can - in fact, that's what realtors call offers. You place an offer, and you only find out later if you bought the house, often days later.

friendzis 3y ago

I agree with the sentiment, but for entirely different reasons. An offline capable application that sometimes tries to sync can be called "Online with Extreme Latency", but that is not what true ~~Scotsman~~ offline is. Offline/online is first and foremost about data locality and data ownership. In an online application (regardless of latency!) the source of truth is "the server". Offline application itself is the source of truth and "the server" is a slave. The OP seems to be talking about thin vs fat clients. Fat client is still a client - it fetches data. Offline application is the data source for "the server", it's the server that has to adapt to local changes not the other way around. Naturally, this is problematic since now you have multiple sources of truth. However, shifting source of truth to "the server" does create online application with extreme latency - fat client.

inferense 3y ago
> Offline/online is first and foremost about data locality and data ownership. not necessarily if the data is stored in a proprietary format.
- Sakos 3y ago
  It doesn't matter if it's a proprietary format. Proprietary formats can be reverse engineered and modified and converted. There is a world of difference between a proprietary file that I have online in a form that I can't access directly (and I have no idea how it's being accessed, changed or stored by the service itself or third parties) and a file that I have on my harddrive that I have complete control over.
- deleted 3y ago
  [deleted]
- tnel77 3y ago
  Yes, but I think most people care about availability of data over true ownership of data. If I’m running a business and my internet goes out, there’s a huge selling point to a local DB that allows my operations to keep chugging along until Comcast figures out their mess. Internet returns, my DB syncs with a server offsite, and now I have that backup.

homero 3y ago

You'd have to make sure the user knew whether it finally worked even weeks later or never sent. They'll forget.

Lio 3y ago

I think you've got it spot on. I've also worked on systems like that where a mobile user could loose connection at any time and they are indeed very complex to get right. I'm not sure there's a perfect way to do it but we ended having have certain functions that had to be done online and others where the user built a "request" for service that was handled optimistically with failures sent to the user's inbox later.

hyperhopper 3y ago

Abstracting this away from the user is what meteor was built to solve, and did it very well for most use cases.

danenania 3y ago

The really hard part is conflict resolution. You updated something 5 times while offline, but another user deleted that something between updates 2 and 3, or made a divergent update of their own. There are so many potential scenarios like this in a multi-user app. It's a huge rabbit hole, and you can end up in situations where it's impossible to sync in a user-friendly way. Essentially you are taking on all the challenges of distributed databases. Maybe it's worth it, but you should know what you're getting into.

giantrobot 3y ago
Online-only collaboration systems already have to do this. The local view has to synchronize state between other local views and the canonical version on the server. Offline changes are online edits with high latency.
- danenania 3y ago
  Yes, but the problem gets much harder to resolve the longer you allow client states to diverge.

simplotek 3y ago

> You have to re-architect things so that the UI is optimistic, commits to a local database and that local database is synced up to the server when you come online. I'm not sure you got the point. The design guideline that "Offline Is Just Online with Extreme Latency" already reflects very specific architectural requirements, and state transitions in the application life cycle. We're talking event-driven architectures, batching events, flushing events in offline/online transitions or even when minimizing windows, pulling events when going online, etc etc etc. I'd go even further and claim that this whole "pessimistic vs optimistic UI" thing is just "adequate vs broken UI design",regardless whether the app is even expected to go online.

saltcured 3y ago

A related concern would be surfacing the "online" part as explicit resources and actions for the user instead of implicit, background magic. If you are sufficiently offline in your lifestyle, you need to plan your communication phases. I recently got into using a GPS sports watch. It is the kind of thing I would want to use in an offline fashion, i.e. go somewhere off the grid and track my hikes or bike rides. These devices are designed to function offline for a stretch of time, but they have a requirement to eventually sync to an online system. They will eventually fill up with recorded sensor data and you want to offload that somewhere else before clearing local device storage. More importantly, satellite positioning receivers need a cached "ephemeris" file that helps them predict which satellites will be overhead at a given time to operate efficiently, accurately, and quickly. Unfortunately, the manufacturers have been infected with smartwatch expectations. They started designing it as if it is always online, and the syncing functions are implicit background behaviors. When doing sporadic syncs and going back offline, it is hard to influence it to "get latest ephemeris" when you known you have internet connectivity and will be going offline again. Worse, these results are localized and it implicitly gets ephemeris for its current location. The UI doesn't allow you to indicate a destination and pre-load the right data to allow fully offline function on arrival.

deceptionatd 3y ago
> More importantly, satellite positioning receivers need a cached "ephemeris" file that helps them predict which satellites will be overhead at a given time to operate efficiently, accurately, and quickly. Interesting, I've never heard of that particular optimization for GNSS before. I know GPS transmits ephemeris information in each frame, since that's the input data for the positioning calculations. I've got a number of Garmin watches, and they've always been able to get a position fix even after being disconnected for weeks. I find GNSS implementations very interesting, which manufacturer is making their watches like this?
- saltcured 3y ago
  Garmin watches do! Go into the System -> About menu and there is a page showing ephemeris status. Their documentation states that this may expire after approximately 30 days or if you travel more than 200 miles, while it will update during syncing. I've seen it expire in less than two weeks with daily use of GPS but phone syncing disabled. It will still get position, but it can be the difference between an almost immediate fix after opening an activity menu or a delay for tens of seconds to minutes. Distance and pace measurements also seem to be lower quality when operating without a current ephemeris file.

thomastjeffery 3y ago

Sounds like it is a useful concept: you just used it to introduce your own. I think that optimistic UI isn't more complex to build: it's just less familiar. In a broader sense, UI/UX design culture places way too much focus and value on familiarity. The best UI/UX designs I have ever experienced are when software design diverges from the norm and tries a new approach.

stepbeek 3y ago
> I think that optimistic UI isn't more complex to build: it's just less familiar. Having multiple records of state that have to be resolved when systems come online is complex. Anything requiring consensus is difficult to do.
- thomastjeffery 3y ago
  The same complexity is present in both approaches. The only difference is cadence.
  - ahzhou 3y ago
    Not exactly - there’s a bunch of non-trivial stuff you need to worry about when both local and remote states represent sources of truth. Things like CRDTs and event sourcing make it easier, but there’s still more complexity than dealing with only one source of truth. In an online-first experience, any local state being held is (for the most part) an optimization or is intentionally ephemeral. You can just toss it away at a performance penalty if it ever gets too hairy. In an offline-first experience, you need to be very careful that you treat everything like a source of truth. You also need to deal with schema migrations and business logic migrations, since you need those to partially live on the client.

treeman79 3y ago

Was required to use a MS sql server database for a project that would go offline for 5-10 minutes every hour. (Getting admins to fix it was a no go). Now of course I was judged on uptime of my app that relied on it. Finally just cloned the data to a local MySQL and refreshed data frequently. Database team totally clueless that app servers were hosting their own databases.