3 ms·

Wow, that's a pretty neat approach. I have a bunch of questions, if you don't mind.. Do you do filesystem replication or something else for HA? Is this running

Wow, that's a pretty neat approach. I have a bunch of questions, if you don't mind.. Do you do filesystem replication or something else for HA? Is this running in one of the cloud cloud IaaS providers or something else? Why did you go with this approach? Eg: Was it a deliberate decision after evaluating other options, a "this is what we know" type situation, or did it grow organically from something smaller? How long has it been running? Whats the rough size of data you have (GB/TB, number of servers, number of DB instances)? Any reliability problems? If you were building this service from scratch, would you do it this way again?

lbruder 9y ago
Don't mind at all. ATM we're using a simple VMWare server backing up the virtual HDD every night, with a dirty hack thrown in (no full backups on weekends as that's when the bigger jobs run, so I dump the current day to a CSV on a smaller virtual HDD that's still saved daily). We're running on our own systems as our customers are pretty paranoid about where their data is sent. The history of this system is pretty organic, lots of experimentation. Started with Postgres, but that was too slow if a select statement returns tens of thousands of rows. MariaDB was way faster but became erratic once we passed about 2 billions of records. Now with SQLite we are at about 180 GB, about three years of data so far, so not really "big data" territory yet. Constantly looking for better alternatives. InfluxDB looks promising, but maybe we'll switch to a "hot/cold" approach instead, keeping whole "cold" days as JSON objects in JSONB fields in Postgres again while storing "hot" data in SQLite. We'll see... So far the system has been rock solid. In the last few weeks we went live for real, now storing about 2k records a second. Not a problem so far.
- kgc 9y ago
  Wait, so to be clear, you moved from PG > Maria > SQLite?

lbruder 9y ago

Don't mind at all. ATM we're using a simple VMWare server backing up the virtual HDD every night, with a dirty hack thrown in (no full backups on weekends as that's when the bigger jobs run, so I dump the current day to a CSV on a smaller virtual HDD that's still saved daily). We're running on our own systems as our customers are pretty paranoid about where their data is sent. The history of this system is pretty organic, lots of experimentation. Started with Postgres, but that was too slow if a select statement returns tens of thousands of rows. MariaDB was way faster but became erratic once we passed about 2 billions of records. Now with SQLite we are at about 180 GB, about three years of data so far, so not really "big data" territory yet. Constantly looking for better alternatives. InfluxDB looks promising, but maybe we'll switch to a "hot/cold" approach instead, keeping whole "cold" days as JSON objects in JSONB fields in Postgres again while storing "hot" data in SQLite. We'll see... So far the system has been rock solid. In the last few weeks we went live for real, now storing about 2k records a second. Not a problem so far.

kgc 9y ago
Wait, so to be clear, you moved from PG > Maria > SQLite?