Indeed. X might be the hyped up flavour of the day, but if you've got large codebases and/or workflows based on Y, then changing to X is a) non-trivial and b) X

Indeed. X might be the hyped up flavour of the day, but if you've got large codebases and/or workflows based on Y, then changing to X is a) non-trivial and b) X may not solve the problems as well as Y. As an example from my experience, when Apache Spark became popular (around v1.8), there were a fair few people in certain circles sneering at Hive, clearly Spark was superior because you could cache intermediate results to RAM. Except Spark at the time (I haven't used it in anger since v2.2) was rather terrible at the very large datasets Hive could grind through with no issue. Sure, Hive was slower, but then it had the distinct advantage on very large datasets of not dying with an OOM error 2 hours into a job.