The unspoken assumption in this article (and many of the replies in this thread) is that all statistical or machine learning problems are online problems, where

The unspoken assumption in this article (and many of the replies in this thread) is that all statistical or machine learning problems are online problems, where data must be ingested, processed and analyzed continuously and indefinitely. For something like a SaaS app that's a no brainer because you have all of that data anyway, but the overwhelming amount of business analysis / optimization / modeling / prediction still revolves around one-off analyses and specific experiments with specific goals, for which all of this data engineering is overkill. I've worked in teams that didn't get anything done because the data was a mess or there wasn't any, but I've also had the experience of spending so much time on the engineering side of things that we stopped asking what kind of things would actually be useful to know or learn or test. Computer scientists and engineers in general often have this attitude of "okay, let's take things one step at a time, first we focus on the data and get that 100% right" with a good amount of smugness about these data scientists and their messy, unorganized code, but consider that maybe they know something you don't.