3 ms·

Sounds interesting. What's the DB? What framework/type of API are you exposing on the C++ size?

by winrid 4y ago

FpUser 4y ago
Postgresql. No framework (well there is rather low level one of my own but it is fluid). Using 4 libs for JSON, postgres, HTTP, logging. I expose basically JSON based RPC. Peers / Clients post JSON commands and receive JSON replies. Upon startup all data (except couple of giant tables) is sucked from DB into RAM into highly efficient data structures hence most reading requests are handled in microseconds. Those 2 giant tables are loaded into the same structs but partially. This is usually enough as RAM keeps last few years worth of data and requests that need something older come just couple of times per month. Only writes touch the database and immediately reflected in RAM as well. This is just an example. I write game like servers as well and those use different approach. Generally I treat each project / product individually.
- winrid 4y ago
  Nice! A lot of game servers are similar to this too, which you of course know. In my case I usually just "snapshot" the in memory structures to disk... BTW for others considering this approach, the DB will do similar things that FpUser is doing in the app layer - keeping hot/recent stuff in memory. But when tail latency and cost really matters this is a good technique.
  - FpUser 4y ago
    >"DB will do similar things" Not really as the storage data layout and application data layout are different. If I were to use database only relying on it's caching abilities it would lead to complex queries that are definitely way less performant than getting data from in memory structures optimized for this particular application / usage patterns. On top of that some requests require some relatively complex calculations that again are executed against optimized in memory layout and would suffer greatly if done against the database. I did some experiments and the difference was very significant.
    - winrid 4y ago
      Sure, I definitely don't know what you're doing. Usually I would reach for preaggregation here, computing the values on write, but I don't know anything about the use case so... :)
- mbrodersen 4y ago
  Yep that’s the way to do it. One improvement IMHO would be to have an event log as the truth and everything else (in-memory/databases/…) derived from that event log. It is an architecture that scales from tiny to Google scale. You don’t even need something like Kafka. You can literally start with a single disk file as the event long and read it into memory at server startup. And then “scale up” by keeping the event log in a single table in a DB (SQLite/Postgres’s etc.)
  - FpUser 4y ago
    For business it is better to have "properly" designed database with the clean structure looking at which it is easy to understand how everything is organized and works. It can be used / interrogated by other software (writes are of course prohibited). Sure it will not do Google scale and I do not loose my sleep over it. For most "normal" businesses what I have is way more than enough. Besides, as I've already said I treat each product individually so my architecture reflects the actual needs and changes accordingly.
    - mbrodersen 4y ago
      Not my experience at all. The problem with a “clean” “properly” designed database schema is handling migration and schema changes. Also, in an Enterprise environment, you end up having external systems relying on your schema, making it nearly impossible to upgrade the schema without breaking systems you don’t even know about until somebody shouts at you.

Sounds interesting. What's the DB? What framework/type of API are you exposing on the C++ size?

by winrid 4y ago

Sounds interesting. What's the DB? What framework/type of API are you exposing on the C++ size?

Postgresql. No framework (well there is rather low level one of my own but it is fluid). Using 4 libs for JSON, postgres, HTTP, logging. I expose basically JSON based RPC. Peers / Clients post JSON commands and receive JSON replies. Upon startup all data (except couple of giant tables) is sucked from DB into RAM into highly efficient data structures hence most reading requests are handled in microseconds. Those 2 giant tables are loaded into the same structs but partially. This is usually enough as RAM keeps last few years worth of data and requests that need something older come just couple of times per month. Only writes touch the database and immediately reflected in RAM as well. This is just an example. I write game like servers as well and those use different approach. Generally I treat each project / product individually.

winrid 4y ago
Nice! A lot of game servers are similar to this too, which you of course know. In my case I usually just "snapshot" the in memory structures to disk... BTW for others considering this approach, the DB will do similar things that FpUser is doing in the app layer - keeping hot/recent stuff in memory. But when tail latency and cost really matters this is a good technique.
- FpUser 4y ago
  >"DB will do similar things" Not really as the storage data layout and application data layout are different. If I were to use database only relying on it's caching abilities it would lead to complex queries that are definitely way less performant than getting data from in memory structures optimized for this particular application / usage patterns. On top of that some requests require some relatively complex calculations that again are executed against optimized in memory layout and would suffer greatly if done against the database. I did some experiments and the difference was very significant.
  - winrid 4y ago
    Sure, I definitely don't know what you're doing. Usually I would reach for preaggregation here, computing the values on write, but I don't know anything about the use case so... :)
mbrodersen 4y ago
Yep that’s the way to do it. One improvement IMHO would be to have an event log as the truth and everything else (in-memory/databases/…) derived from that event log. It is an architecture that scales from tiny to Google scale. You don’t even need something like Kafka. You can literally start with a single disk file as the event long and read it into memory at server startup. And then “scale up” by keeping the event log in a single table in a DB (SQLite/Postgres’s etc.)
- FpUser 4y ago
  For business it is better to have "properly" designed database with the clean structure looking at which it is easy to understand how everything is organized and works. It can be used / interrogated by other software (writes are of course prohibited). Sure it will not do Google scale and I do not loose my sleep over it. For most "normal" businesses what I have is way more than enough. Besides, as I've already said I treat each product individually so my architecture reflects the actual needs and changes accordingly.
  - mbrodersen 4y ago
    Not my experience at all. The problem with a “clean” “properly” designed database schema is handling migration and schema changes. Also, in an Enterprise environment, you end up having external systems relying on your schema, making it nearly impossible to upgrade the schema without breaking systems you don’t even know about until somebody shouts at you.