4 ms·

I know another government contractor whose idea of client responsiveness is to download the entire server database to the client browser on initial load

couchand 5y ago
That's completely the right strategy for the scale most web apps operate at.
- foota 5y ago
  This works great until you have something that doesn't fit, and it can block new features as a result of that. I worked on an application a number of years ago where it was trying to load all the comments and details about an internal bug tracker into memory. It must have worked fine at first, but after time it was a POS.
  - altfredd 5y ago
    There is no "one size fits all" solution. If the database fits onto client hard drive and the modifications are rare, preloading everything is almost always better. If you have a dynamically changing system such as bug tracker, it is still possible to go fully local, but that would require considerable cooperation from server side. When the back-end does not have a fast, efficient API for sending diffs, you may get stuck waiting for it to be implemented. But that's a purely organizational problem. Of course, all of above applies to actually saving data to permanent storage. Storing everything in memory is a sin by itself.
- thakoppno 5y ago
  How much more expensive is 100KB of JSON compared to last year?
- owl57 5y ago
  Moreover, most mobile apps, web or native, could suck A LOT less if they did just that, load all the data at start. I have taken that to the extreme of serving all the code and data in one file. Definitely will do that again if I ever make another app with similar and fairly common constraints. Never waiting for unstable network and zero bugs with some part of HTML/CSS/JS/data missing or out of sync is pure joy.
- Oddskar 5y ago
  Until someone doesn't think twice and adds some column to the DB that contains PII.
tgsovlerkhgsel 5y ago
For a sufficiently small database (that doesn't require more granular access control), that seems to be the right choice. A 1 MB initial load is cheap nowadays.
- unclebucknasty 5y ago
  Are we using the term "database" loosely here to mean some limited subset of data? Because I think of the term as referring to an app's primary datastore, and I can't recall a single meaningful modern app I've seen for which the database is anywhere close to that small.
  - tgsovlerkhgsel 5y ago
    Think of small, in-house applications managing an inventory of a couple hundred to a couple thousand items. "Hard discount" stores like Aldi are supposed to have <1500 SKUs, for example.
    - unclebucknasty 5y ago
      Got it. Thanks for clarifying. Thought there might be some kind of clever sharding or tooling I was missing.
      - tgsovlerkhgsel 5y ago
        No really clever tooling. But even where the database gets too big, you can still stay in a similar operating mode. For example, when the user requests the first page of items or performs a search (that you have to do server side due to the size of the whole data set) - why not send the first 100 actual items instead of just the visible data from the 10 rows forming the first page? Or send the first 10 immediately followed by prefetching the next 90? Now the user will immediately get to see the full item and will be able to page through the results much more quickly. I've definitely had cases where I had to process the data before sending it to the client, but I've also sent absurd amounts of data and rendered it client side. In fact, I think sending data embedded in HTML to the client is rarely a good idea, and once you've adopted that mindset, apps can look very different.

couchand 5y ago

That's completely the right strategy for the scale most web apps operate at.

foota 5y ago
This works great until you have something that doesn't fit, and it can block new features as a result of that. I worked on an application a number of years ago where it was trying to load all the comments and details about an internal bug tracker into memory. It must have worked fine at first, but after time it was a POS.
- altfredd 5y ago
  There is no "one size fits all" solution. If the database fits onto client hard drive and the modifications are rare, preloading everything is almost always better. If you have a dynamically changing system such as bug tracker, it is still possible to go fully local, but that would require considerable cooperation from server side. When the back-end does not have a fast, efficient API for sending diffs, you may get stuck waiting for it to be implemented. But that's a purely organizational problem. Of course, all of above applies to actually saving data to permanent storage. Storing everything in memory is a sin by itself.
thakoppno 5y ago
How much more expensive is 100KB of JSON compared to last year?
owl57 5y ago
Moreover, most mobile apps, web or native, could suck A LOT less if they did just that, load all the data at start. I have taken that to the extreme of serving all the code and data in one file. Definitely will do that again if I ever make another app with similar and fairly common constraints. Never waiting for unstable network and zero bugs with some part of HTML/CSS/JS/data missing or out of sync is pure joy.
Oddskar 5y ago
Until someone doesn't think twice and adds some column to the DB that contains PII.

tgsovlerkhgsel 5y ago

For a sufficiently small database (that doesn't require more granular access control), that seems to be the right choice. A 1 MB initial load is cheap nowadays.

unclebucknasty 5y ago
Are we using the term "database" loosely here to mean some limited subset of data? Because I think of the term as referring to an app's primary datastore, and I can't recall a single meaningful modern app I've seen for which the database is anywhere close to that small.
- tgsovlerkhgsel 5y ago
  Think of small, in-house applications managing an inventory of a couple hundred to a couple thousand items. "Hard discount" stores like Aldi are supposed to have <1500 SKUs, for example.
  - unclebucknasty 5y ago
    Got it. Thanks for clarifying. Thought there might be some kind of clever sharding or tooling I was missing.
    - tgsovlerkhgsel 5y ago
      No really clever tooling. But even where the database gets too big, you can still stay in a similar operating mode. For example, when the user requests the first page of items or performs a search (that you have to do server side due to the size of the whole data set) - why not send the first 100 actual items instead of just the visible data from the 10 rows forming the first page? Or send the first 10 immediately followed by prefetching the next 90? Now the user will immediately get to see the full item and will be able to page through the results much more quickly. I've definitely had cases where I had to process the data before sending it to the client, but I've also sent absurd amounts of data and rendered it client side. In fact, I think sending data embedded in HTML to the client is rarely a good idea, and once you've adopted that mindset, apps can look very different.