4 ms·
Maybe but unfortunately the database doesn't support "backup store" that we can use directly S3, it just writes the data to specified directory in a filesystem.
by buremba 10y ago
Maybe but unfortunately the database doesn't support "backup store" that we can use directly S3, it just writes the data to specified directory in a filesystem. We should either fork the database or implement "S3 backed" filesystem and we picked the latter one. Unfortunately the performance FUSE is not that good for read and writing data to S3 is somehow expensive.
- ak4g 10y agoIf you don't mind me asking, what DB is this?
- buremba 10y agoWe're implementing Clickhouse (http://clickhouse.yandex http://clickhouse.yandex) to our open-source analytics platform Rakam. (https://github.com/rakam-io/rakam https://github.com/rakam-io/rakam)
- mej10 10y agoWhy not have a service watch for filesystem changes and then upload to S3?
- buremba 10y agoAFAIK, there is no synchronous way to do it and if I do it asynchronously, partial data loss is possible.