3 ms·

Confounding factors. For example, crashes can be more likely after sunset or during inclement weather so seasonality can play a role. Or maybe the drivers who h

• celticninja 10y ago
  Could this be accounted for if the sample period was large enough, ie over 12 months to cover seasonality and a few months for time of day.
  • tehlike 10y ago
    this particular one, maybe, but there might be others. like the user base pre-ota and post ota (number-wise) could be different. I think there is a good SaaS business with A/B testing robot/car software OTA. I would imagine some metrics would be: 1. car speed 2. battery consumption per mile. 3. user comfort metrics (if it's possible to measure, maybe like hearthrate, body temperature etc - i can imagine if robots were driving bad, it'd be visible in heart rate or something like that).4 The list probably goes on. It would be interesting to see how much battery consumption changed after OTA, for example.
  • pps43 10y ago
    I suspect OTA update was applied simultaneously rather than staggered over the course of the year, so probably not. You can control for seasonality using some other data as a proxy, but the more adjustments you make, the more potential there is for introducing errors. Randomized test/control split is almost always better than before/after comparison.
• deleted 10y ago
  [deleted]