> That’s the whole point of ML, to reason about new data based on data seen in past. I think we like to think this is true. In reality, I have seen a lot of

> That’s the whole point of ML, to reason about new data based on data seen in past. I think we like to think this is true. In reality, I have seen a lot of real world ML models. I wouldn't trust ANY of them to do extrapolation. There are just tons of real world problems, and extrapolation is HARD. I have to put extremely tight boundaries on ML models for deployment scenarios, and ALWAYS have a backup rule engine in case the ML model comes up with an answer that has a low confidence score. > How do you identify the outlier? You need to write some rules that could look at it. But that’s a lot of rules. What if you could use computers to do that? > You need to write some rules that could look at it. Pretty much. Any time ML is involved, you will need TONS of lines of code. In short, tightly define the target classes your ML model deals with. Any variable that falls outside your tightly bound list of target classes, you have to deal with using a rules engine. THEN you need to spend a lot of time doing work to minimize false positive classification in your target classes. And make sure that "false positive, high confidence" classifications don't do racist things/lose the business a lot of money things. ML projects are just a ton of work. You essentially make the entire ML workflow, and you NEED a backup "not-ML" workflow. In my experience, 50-80% of normal software engineering projects fail. 90% of ML projects fail. Square the fraction of normal software projects. ML is complex AND it's a ton of work. Really, really hard.