3 ms·

Thanks for sharing your setup here. Go and SQLite are great! I have a few questions :-) Is the Go service directly accessible from Internet, or is it hosted be

Thanks for sharing your setup here. Go and SQLite are great! I have a few questions :-) Is the Go service directly accessible from Internet, or is it hosted behind a reverse proxy? When you deploy a new binary, is there a small downtime between stopping the old binary and starting the new one? How do you supervise the Go process? You use something like systemd? When SQLite backup is ongoing, does it block writes to the database? When you backup to S3, if an attacker gets control of your EC2 instance, is he able to erase your S3 backups, or is it configured in some kind of append-only mode? Where do you store your logs and do you "read" them?

Lord_Zero 8y ago
These are fantastic questions and id love to hear the answers.
- ngrilly 8y ago
  I will answer my own questions, in the context of the apps I develop and maintain: - The Go service is hosted behind a reverse proxy (nginx or haproxy) to enable zero downtime deployments, by 1) starting the new process, 2) directing new requests to the new process, and 3) gracefully stopping the old process. - Since we've started to use Docker, we let the Docker daemon supervise and restart our services. Before Docker, we used systemd. Before systemd was available on our system, we used supervisord. - We thought about using SQLite for some apps. But SQLite can only have a single writer at a time, which goes against the zero downtime deployment described above (two processes can be processing requests at the same time). Thus we use PostgreSQL (and MySQL for legacy reasons) which provides online backups. Must be noted that online backups are possible with SQLite, provided the application implement it using SQLite Online Backup API [1]. Another solution, which doesn't require application cooperation, is to snapshot your disk, if your system supports this. - We backup to rsync.net, which provides an append-only mode, through their snapshot feature [2]. An attacked cannot override or erase the snapshots of your previous backups. I think it's possible to do something similar with S3, albeit in a bit more cumbersome way, using S3 versioning and MFA deletion. - About logs, we're still not satisfied by what we use currently. I'd be curious to read about what others do :-) [1] https://www.sqlite.org/backup.html https://www.sqlite.org/backup.html [2] https://www.rsync.net/resources/howto/snapshots.html https://www.rsync.net/resources/howto/snapshots.html

Lord_Zero 8y ago

These are fantastic questions and id love to hear the answers.

ngrilly 8y ago
I will answer my own questions, in the context of the apps I develop and maintain: - The Go service is hosted behind a reverse proxy (nginx or haproxy) to enable zero downtime deployments, by 1) starting the new process, 2) directing new requests to the new process, and 3) gracefully stopping the old process. - Since we've started to use Docker, we let the Docker daemon supervise and restart our services. Before Docker, we used systemd. Before systemd was available on our system, we used supervisord. - We thought about using SQLite for some apps. But SQLite can only have a single writer at a time, which goes against the zero downtime deployment described above (two processes can be processing requests at the same time). Thus we use PostgreSQL (and MySQL for legacy reasons) which provides online backups. Must be noted that online backups are possible with SQLite, provided the application implement it using SQLite Online Backup API [1]. Another solution, which doesn't require application cooperation, is to snapshot your disk, if your system supports this. - We backup to rsync.net, which provides an append-only mode, through their snapshot feature [2]. An attacked cannot override or erase the snapshots of your previous backups. I think it's possible to do something similar with S3, albeit in a bit more cumbersome way, using S3 versioning and MFA deletion. - About logs, we're still not satisfied by what we use currently. I'd be curious to read about what others do :-) [1] https://www.sqlite.org/backup.html https://www.sqlite.org/backup.html [2] https://www.rsync.net/resources/howto/snapshots.html https://www.rsync.net/resources/howto/snapshots.html