6 ms·

> Someone needs to have the working set of your database in memory, why shouldn't it be your app server and business logic? This one confused me. The obvious r

> Someone needs to have the working set of your database in memory, why shouldn't it be your app server and business logic? This one confused me. The obvious reason why you don't want the whole working set of your database in the app server's memory is because you have lots of app servers, whereas you only have one database[1]. This suggests that you put the working set of the database in the database, so that you still only need the one copy, not in the app servers where you'd need N copies of it. The rest of your post makes sense to me but the thing about keeping the database's working set in your app server's memory does not. That's something we specifically work to avoid. [1] Still talking about "non-webscale" office usage here, that's the world I live in as well. One big central database server, lots of apps and app servers strewn about.

NovemberWhiskey 3y ago
I think the point is that treating your database as an arms-length, RPC component that's independent from your "application" isn't necessarily the only pattern.
- foobiekr 3y ago
  Strong agree. there are vast, massive cost savings and performance advantages to be had if the model is that a shard of the dataset is in memory and the data persistence problem is the part that's made external. The only reason we are where we are today is that doing that well is hard.
  - robertlagrant 3y ago
    Is this not the case already? Database drivers (or just your application code) are allowed to cache results if they like. The problem is cache invalidation.
    - foobiekr 3y ago
      Caches aren't the same. In the shard-in-memory case, the shard is the thing serving the queries, meaning it's not a cache it _is_ the live data.
      - robertlagrant 3y ago
        Understood. For a single-node or read-only system it sounds fine, but then there are a variety of ways to solve that (e.g. a preloaded in-memory sqlite).
  - bombolo 3y ago
    If it's in memory you must live with the fact that it might be gone at any moment though.
    - foobiekr 3y ago
      Hence "the data persistence part."
      - robertlagrant 3y ago
        This sounds like a write-back / write-through cache with extra terminology. What's the difference?
        foobiekr 3y ago
        The difference is the 10,000x slower performance if you move the actual DB transaction leader for the shard to over-the-wire access. Anything can be anywhere if we ignore latency and throughput.
        robertlagrant 3y ago
        I'm not saying that. I'm asking if this is just new terminology for a previous technology idea, or whether it's a new concept.
augustl 3y ago
It's definitely a trade-off! If you have 10s or 100s of app servers that has the exact same working set in memory, it's probably not worth it. But if you have a handful of app servers, it's much more reasonable. The relatively low scale back-office systems I tend to work with typically has 2, max 3. Also, spinning up an extra instance that does some data crunching does not affect the performance of the app servers, as they don't have to coordinate. There's also the performance and practicality benefits you get from not having to do round-trips in order to query. You can now actually do 100 queries in a loop, instead of having to formulate everything as a single query. And if you have many different apps that operates on the same DB, it becomes a benefit as well. The app server will only have the _actual_ working set it queries on in memory, not the sum of the working set across all of the app servers. If this becomes a problem, you can always architecture your way around it as well, by having two beefy API app servers that your 10s or 100s of other app servers talks to.
- electroly 3y ago
  SQLite provides a similar benefit with tremendous results using its in-process database engine, although the benefit there is more muted by default because of the very small default cache size. We do have one app where we do this. There's no database server, the app server uses SQLite to talk directly to S3 and the app server itself caches its working set in memory. I can definitely see the benefit of some situations, but for us it was a pretty unusual situation that we might not ever encounter again. All that said... can't Datomic also do traditional query execution on the server? I thought it had support for scale-out compute for that. AIUI, you have the option to run as either a full peer or just an RPC client on a case-by-case basis? I thought you wouldn't need to resort to writing your own API intermediary, you could just connect to Datomic via RPC, right?
  - carry_bit 3y ago
    AIUI, the full peer is Datomic; the RPC server is just a full peer that exposes the API over http and is mainly intended to be used with clients that don't run on the JVM (and so can't run Datomic itself in-process).
- 3y ago
dimitar 3y ago
Consider this use case - in addition to your web app, you have a reporting service that makes heavy duty reports; if you run one at a bad time, bad things might happen like users not being able to log in or do any other important work, because the database is busy with the reports. So in a traditional DB you might have a DBA set up a reporting database so the operational one is not affected. Using Datomic the reporting service gets a datomic peer that has a copy of the DB in database without any extra DBA work and without affecting any web services. This also works nicely with batch jobs or in any situation where you don't want to have different services affect each others performance. Its true that a lot more memory gets used, but it is relatively cheap - usually the biggest cost when hosting in the cloud being the vCPUs. But usually in Clojure/Datomic web application you don't need to put various cache services like Redis in front of your DB. Thea assumption here is that the usual bottleneck for most information systems and business applications is reading and querying data.
- electroly 3y ago
  I appreciated this insight into other people's use cases, thank you for that! This architecture brings RethinkDB to mind, which also had some ability to run your client as a cluster node that you alone get to query. (Although there it was more about receiving the live feed than about caching a local working set.)
  - thewataccount 3y ago
    Is RethinkDB still around? They actually have recent commits, and a release last year.
    - electroly 3y ago
      The company is gone, but the open source project lives on. We still use it in production.
      - thewataccount 3y ago
        How's it been for production? Would you recommend using it, or would it be better to go with a safer option?
xmlblog 3y ago
Having the working set present on app servers means they don't put load on a precious centralized resource which becomes a bottleneck for reads. The peer model allows app servers to service reads directly, avoiding the cost of contention and an additional network hop, allowing for massive read scale.
bruiseralmighty 3y ago
This is true, but the tradeoff is that now your central DB is a bottleneck that is difficult to scale. Having the applications keep a cached version of the db means that when one of them runs a complex or resource intensive query, it's not affecting everyone else.

NovemberWhiskey 3y ago

I think the point is that treating your database as an arms-length, RPC component that's independent from your "application" isn't necessarily the only pattern.

foobiekr 3y ago
Strong agree. there are vast, massive cost savings and performance advantages to be had if the model is that a shard of the dataset is in memory and the data persistence problem is the part that's made external. The only reason we are where we are today is that doing that well is hard.
- robertlagrant 3y ago
  Is this not the case already? Database drivers (or just your application code) are allowed to cache results if they like. The problem is cache invalidation.
  - foobiekr 3y ago
    Caches aren't the same. In the shard-in-memory case, the shard is the thing serving the queries, meaning it's not a cache it _is_ the live data.
    - robertlagrant 3y ago
      Understood. For a single-node or read-only system it sounds fine, but then there are a variety of ways to solve that (e.g. a preloaded in-memory sqlite).
- bombolo 3y ago
  If it's in memory you must live with the fact that it might be gone at any moment though.
  - foobiekr 3y ago
    Hence "the data persistence part."
    - robertlagrant 3y ago
      This sounds like a write-back / write-through cache with extra terminology. What's the difference?
      - foobiekr 3y ago
        The difference is the 10,000x slower performance if you move the actual DB transaction leader for the shard to over-the-wire access. Anything can be anywhere if we ignore latency and throughput.
        robertlagrant 3y ago
        I'm not saying that. I'm asking if this is just new terminology for a previous technology idea, or whether it's a new concept.

augustl 3y ago

It's definitely a trade-off! If you have 10s or 100s of app servers that has the exact same working set in memory, it's probably not worth it. But if you have a handful of app servers, it's much more reasonable. The relatively low scale back-office systems I tend to work with typically has 2, max 3. Also, spinning up an extra instance that does some data crunching does not affect the performance of the app servers, as they don't have to coordinate. There's also the performance and practicality benefits you get from not having to do round-trips in order to query. You can now actually do 100 queries in a loop, instead of having to formulate everything as a single query. And if you have many different apps that operates on the same DB, it becomes a benefit as well. The app server will only have the _actual_ working set it queries on in memory, not the sum of the working set across all of the app servers. If this becomes a problem, you can always architecture your way around it as well, by having two beefy API app servers that your 10s or 100s of other app servers talks to.

electroly 3y ago
SQLite provides a similar benefit with tremendous results using its in-process database engine, although the benefit there is more muted by default because of the very small default cache size. We do have one app where we do this. There's no database server, the app server uses SQLite to talk directly to S3 and the app server itself caches its working set in memory. I can definitely see the benefit of some situations, but for us it was a pretty unusual situation that we might not ever encounter again. All that said... can't Datomic also do traditional query execution on the server? I thought it had support for scale-out compute for that. AIUI, you have the option to run as either a full peer or just an RPC client on a case-by-case basis? I thought you wouldn't need to resort to writing your own API intermediary, you could just connect to Datomic via RPC, right?
- carry_bit 3y ago
  AIUI, the full peer is Datomic; the RPC server is just a full peer that exposes the API over http and is mainly intended to be used with clients that don't run on the JVM (and so can't run Datomic itself in-process).
3y ago

dimitar 3y ago

Consider this use case - in addition to your web app, you have a reporting service that makes heavy duty reports; if you run one at a bad time, bad things might happen like users not being able to log in or do any other important work, because the database is busy with the reports. So in a traditional DB you might have a DBA set up a reporting database so the operational one is not affected. Using Datomic the reporting service gets a datomic peer that has a copy of the DB in database without any extra DBA work and without affecting any web services. This also works nicely with batch jobs or in any situation where you don't want to have different services affect each others performance. Its true that a lot more memory gets used, but it is relatively cheap - usually the biggest cost when hosting in the cloud being the vCPUs. But usually in Clojure/Datomic web application you don't need to put various cache services like Redis in front of your DB. Thea assumption here is that the usual bottleneck for most information systems and business applications is reading and querying data.

electroly 3y ago
I appreciated this insight into other people's use cases, thank you for that! This architecture brings RethinkDB to mind, which also had some ability to run your client as a cluster node that you alone get to query. (Although there it was more about receiving the live feed than about caching a local working set.)
- thewataccount 3y ago
  Is RethinkDB still around? They actually have recent commits, and a release last year.
  - electroly 3y ago
    The company is gone, but the open source project lives on. We still use it in production.
    - thewataccount 3y ago
      How's it been for production? Would you recommend using it, or would it be better to go with a safer option?

xmlblog 3y ago

Having the working set present on app servers means they don't put load on a precious centralized resource which becomes a bottleneck for reads. The peer model allows app servers to service reads directly, avoiding the cost of contention and an additional network hop, allowing for massive read scale.

bruiseralmighty 3y ago

This is true, but the tradeoff is that now your central DB is a bottleneck that is difficult to scale. Having the applications keep a cached version of the db means that when one of them runs a complex or resource intensive query, it's not affecting everyone else.