4 ms·

would love to do this, but how does one spin up a redshift cluster inside of a docker container?

• nicoburns 4y ago
  And this is why you don’t rely on a closed source stack if you have any alternative.
• contravariant 4y ago
  Well the first step is to get Amazon to part with their lucrative closed source software.
• ntr-- 4y ago
  try not to cry, cry a lot, and then resolve not to vendor lock yourself to a black box data store next time. jokes aside, redshift is based on pg^1, you can try an older version to get some semblance of it running locally. 1. https://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and-postgres-sql.html https://docs.aws.amazon.com/redshift/latest/dg/c_redshift-an...
• danmcs 4y ago
  Redshift speaks the postgres protocol so you might be able to use postgres. There are a few purpose-built docker images (googleable) that may replicate Redshift slightly better than just `docker run -p 127.0.0.1:5439:5432 postgres:latest`, but if you're at the point of having a test suite for your data warehouse code, you're likely using Redshift-specific features in your code and postgres won't suffice. I have seen teams give each developer a personal schema on a dev cluster, to ensure their Redshift SQL actually works. The downside is that now your tests are non-local, so it's a real tradeoff. In CI you probably connect to a real test cluster.
• atwebb 4y ago
  Would redshift serverless help at all? I realize that it may not have parity with the existing functionality, just a thought.