4 ms·
Do you have as many events in both samples? If not, even if you measure the same thing, the variance may differ - you can start applying the law of large numbe
by 1996 6y ago
Do you have as many events in both samples?
If not, even if you measure the same thing, the variance may differ - you can start applying the law of large numbers since you mentionned "millions" and just consider a normal distribution, but for rare events you need something different (Poisson)
Are the series static? If not (ex: temperature) you will have to model on time. As the link mentions people crossing intersection, I assume more people will at 9am than at 2am (7 hours ago). Likewise if one series is a weekday and the other a weekend, you may have surprises!
You have to consider other things (ex: day of the week).
I would recommend avoiding the problem altogether: fit something (KDE, ARIMA...) on your timeseries, and plot the fit with the 95% CI from X=0 to X=24
Do the same for the other series, from X=0 to X=7 and see if they are at least in the same CI
Then compute the correlation of the 2 fits (just to have a reference number) and use that as a metric.
- dabreegster 6y agoTo be more specific, I'm measuring throughput along road segments in a deterministic traffic simulation. Differences would occur when the user modifies the road network. The modifications often have no effect on most roads, but some wind up with more or less traffic than usual at a certain time. I'm interested in showing differences like "during morning rush hour, there was more traffic here, but it was about the same the rest of the day", so I'm not sure a single correlation as a metric would be best. Ideally I'd plot two line plots and call out differences over time.
- 1996 6y agoThere is going to be complex behavior (adapting to traffic by routing around it) that a simulation will poorly predict Also, you will have correlation between adjacent segments of the network- and the idea of extracting "rush hours" brings its own set of problems. Consult with a statistician familiar with geographical models. I'm sorry I can't help much more than that. It's very different from finance.
- dabreegster 6y agoSurprises from emergent behavior are some of the most satisfying things about working on this. I'm not trying to extract rush hour patterns or anything quantitatively, just show two timeseries plots to the user and have them figure out what's going on. Thanks for the ideas! Lots of new things to research. Stats isn't my background at all.