3 ms·
As it is you still have the problem that you're not comparing like with like, you can't compare the total for an hour with the total for part of an hour. So you
by contravariant 6y ago
As it is you still have the problem that you're not comparing like with like, you can't compare the total for an hour with the total for part of an hour. So you'd basically need to restrict your visualization to only show the image at the moments when you do have all the data. I'd strongly suggest biting the bullet and just storing the data in 5 minute buckets if at all possible, then you can have both sides in the same format and compare them easily. 5 minute buckets is only 12 times the amount of data you currently need, which might be a reasonable trade-off.
If you can't or don't want to do that then you'll need to pull some tricks. In essence you want to compare two 'rates' against each other right? Now technically the 5-minute bucket idea is just a crude approximation, what you actually want is the average rate at any point in time a histogram is just a crude method to determine this rate. This means you might get away with it if you try to calculate this rate using 2 different methods on both sides.
For the presampled data you could:
- Interpolate the hourly data (keep in mind that the hourly totals run 30 minutes behind)
- Fit a function to the data rather than just sampling it (i.e. if its periodic you could just use the low frequency part of its Fourier series)
Now I presume the data coming in is coming in online (i.e. you can't just wait for a full sample and compare it afterwards, otherwise just do the same thing as above). This means you'll need to do something like:
- Use a sliding window (again keep in mind that this delays the signal)
- Use an online low-pass filter.
- Fit a function to the partial data (tricky and might be unstable)
Hope this helps.