In my experience, architecture stays. A nice modular codebase (where you can switch-out the various models and have control on the minutia of the pipeline from

In my experience, architecture stays. A nice modular codebase (where you can switch-out the various models and have control on the minutia of the pipeline from data to model) brings a lot of value and can fairly easily be kept up-to-date with best practices.