I tend to find the following few questions a quick way to evaluate an ML/AI pitch in an elevator: Ask yourself: could a human given the inputs reasonably produ

I tend to find the following few questions a quick way to evaluate an ML/AI pitch in an elevator: Ask yourself: could a human given the inputs reasonably produce the outputs you are looking for? — this helps avoid/identify the pure magic pitches. Then ask the person pitching: 1. What’s your training data and how is it collected? 2. What’s your validation data and how is that constructed, and how does your system perform on that set? 3. What are the blind spots and biases in your model and how are you mitigating them? If they don’t have succinct and competent answers or major red flags like no validation or claiming no biases then run away.