6 ms·

WAL-G is not yet production ready, but it has been used in a staging environment for the past few weeks without any issues. Once fdr adds parallel WAL support,

by kafkes 9y ago

WAL-G is not yet production ready, but it has been used in a staging environment for the past few weeks without any issues. Once fdr adds parallel WAL support, he plans to take it into production.

brianwawok 9y ago
Cool cool. is google cloud storage on the roadmap?
- fdr 9y ago
  I answered this question in a couple of other places, but: unknown because I don't have use for that yet. https://news.ycombinator.com/item?id=15049527 https://news.ycombinator.com/item?id=15049527
- justinclift 9y ago
  WAL-G looks like it should be able to talk to Minio (minio.io, S3 compatible thing, also written in go) as a backend instead of S3 itself. Minio has an interesting feature where it can be a "gateway" to other cloud storage. Google Cloud Storage is one of their specific examples: https://docs.minio.io/docs/minio-gateway-for-gcs https://docs.minio.io/docs/minio-gateway-for-gcs So WAL-G would talk to Minio, and Minio would transparently proxy that to GCS.
  - brianwawok 9y ago
    Neat. My concern out of the gate is what would be the perf hit. I assume I am switching from WAL-E to WAL-G for more perf. But WAL-E speaks GCS. If WAL-G needs an extra hop to do so, may lose some of the point of it..
    - justinclift 9y ago
      Yeah, no idea personally. Haven't used the gateway functionality in Minio at all. That being said, the Minio team seem pretty good with writing performance optimised code. Frank Wessels (on Minio team), has been writing articles about Go assembler and other Go optimisation things recently. eg: • https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-in-go-assembly-33ef16c8a56b https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-... • https://blog.minio.io/golang-internals-part-2-nice-benefits-of-named-return-values-1e95305c8687 https://blog.minio.io/golang-internals-part-2-nice-benefits-... So the performance impact might not be such a problem. :)
- boulos 9y ago
  There was some mention about resumable uploads in the blogpost which sadly each provider handles differently (that is the GCS layer that supports the S3 API does not accept resumable uploads). Disclosure: I work on Google Cloud (so I'd love to see this tool point at GCS).

by kafkes 9y ago

WAL-G is not yet production ready, but it has been used in a staging environment for the past few weeks without any issues. Once fdr adds parallel WAL support, he plans to take it into production.

brianwawok 9y ago

Cool cool. is google cloud storage on the roadmap?

fdr 9y ago
I answered this question in a couple of other places, but: unknown because I don't have use for that yet. https://news.ycombinator.com/item?id=15049527 https://news.ycombinator.com/item?id=15049527
justinclift 9y ago
WAL-G looks like it should be able to talk to Minio (minio.io, S3 compatible thing, also written in go) as a backend instead of S3 itself. Minio has an interesting feature where it can be a "gateway" to other cloud storage. Google Cloud Storage is one of their specific examples: https://docs.minio.io/docs/minio-gateway-for-gcs https://docs.minio.io/docs/minio-gateway-for-gcs So WAL-G would talk to Minio, and Minio would transparently proxy that to GCS.
- brianwawok 9y ago
  Neat. My concern out of the gate is what would be the perf hit. I assume I am switching from WAL-E to WAL-G for more perf. But WAL-E speaks GCS. If WAL-G needs an extra hop to do so, may lose some of the point of it..
  - justinclift 9y ago
    Yeah, no idea personally. Haven't used the gateway functionality in Minio at all. That being said, the Minio team seem pretty good with writing performance optimised code. Frank Wessels (on Minio team), has been writing articles about Go assembler and other Go optimisation things recently. eg: • https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-in-go-assembly-33ef16c8a56b https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-... • https://blog.minio.io/golang-internals-part-2-nice-benefits-of-named-return-values-1e95305c8687 https://blog.minio.io/golang-internals-part-2-nice-benefits-... So the performance impact might not be such a problem. :)
boulos 9y ago
There was some mention about resumable uploads in the blogpost which sadly each provider handles differently (that is the GCS layer that supports the S3 API does not accept resumable uploads). Disclosure: I work on Google Cloud (so I'd love to see this tool point at GCS).