6 ms·

If YOUR life is good but you want to continue to make your analysts' life a living hell => Don't change anything Otherwise, switch to Postgres. At the very lea

If YOUR life is good but you want to continue to make your analysts' life a living hell => Don't change anything Otherwise, switch to Postgres. At the very least, for the love of god, make sure your MySQL is >=v8.0

sslalready 7y ago
Care to expand on why MySQL < 8 will be problematic for analysts? Anything else than the lack of window function?
- paulryanrogers 7y ago
  Possibly JSON operators
- jimktrains2 7y ago
  Window functions.
- oarabbus_ 7y ago
  IMO lack of window functions alone are a death knell. CTEs/WITH clause is the other massive one - almost as important as window functions. Additionally, EXCEPT/INTERSECT support, better EXPLAIN (it's still shit though), much improved REGEXP and JSON support.
grogenaut 7y ago
Or you know just export your data to a data lake like redshift spectrum and let the analysts pay for their usage by querying from s3 and take them out of the equation on how you run the website. There is no need for a one size fits all solution here nor a centralized approach.
- oarabbus_ 7y ago
  Except postgres is a terrific backend for a web app, so not sure where you're going with that. And using Spectrum or Athena and reading directly from the data lake has it's own pros and cons and overhead costs. I wouldn't agree with your approach in most cases. Much better would be to ETL from postgres into a vanilla redshift DW or just set up a replica postgres to query from. If you're using Redshift Spectrum you should probably take a look at Snowflake as an alternative.
  - grogenaut 7y ago
    This is in comparison to rewriting your web application from MySQL to pgsql so your analysts can work. And I'm using spectrum as a data lake solution generically. If you want to run snowflake or some other thing is dependent on if you are in aws, gke, bare metal, etc and if you want to host or not. I prefer not to have to manage hosting and I run in AWS. The important part is you don't need to tie your web app database to all of your needs. Separation of concerns is the important part and independent scaling for independent needs.
- krageon 7y ago
  Exporting the data analysts want to a jurisdiction that has very bad privacy safeguards is unethical. Even if you're not exporting it between jurisdictions, exporting it to a separate company should not be a step that is easy to take for you.

sslalready 7y ago

Care to expand on why MySQL < 8 will be problematic for analysts? Anything else than the lack of window function?

paulryanrogers 7y ago
Possibly JSON operators
jimktrains2 7y ago
Window functions.
oarabbus_ 7y ago
IMO lack of window functions alone are a death knell. CTEs/WITH clause is the other massive one - almost as important as window functions. Additionally, EXCEPT/INTERSECT support, better EXPLAIN (it's still shit though), much improved REGEXP and JSON support.

grogenaut 7y ago

Or you know just export your data to a data lake like redshift spectrum and let the analysts pay for their usage by querying from s3 and take them out of the equation on how you run the website. There is no need for a one size fits all solution here nor a centralized approach.

oarabbus_ 7y ago
Except postgres is a terrific backend for a web app, so not sure where you're going with that. And using Spectrum or Athena and reading directly from the data lake has it's own pros and cons and overhead costs. I wouldn't agree with your approach in most cases. Much better would be to ETL from postgres into a vanilla redshift DW or just set up a replica postgres to query from. If you're using Redshift Spectrum you should probably take a look at Snowflake as an alternative.
- grogenaut 7y ago
  This is in comparison to rewriting your web application from MySQL to pgsql so your analysts can work. And I'm using spectrum as a data lake solution generically. If you want to run snowflake or some other thing is dependent on if you are in aws, gke, bare metal, etc and if you want to host or not. I prefer not to have to manage hosting and I run in AWS. The important part is you don't need to tie your web app database to all of your needs. Separation of concerns is the important part and independent scaling for independent needs.
krageon 7y ago
Exporting the data analysts want to a jurisdiction that has very bad privacy safeguards is unethical. Even if you're not exporting it between jurisdictions, exporting it to a separate company should not be a step that is easy to take for you.