4 ms·
great work! When it comes to batched computations, why not leverage intermediate state much like streaming jobs. For example, if we need to calculate past 30 da
by evolutionblues 2y ago
great work! When it comes to batched computations, why not leverage intermediate state much like streaming jobs. For example, if we need to calculate past 30 day sum for a value daily - it seems like this would compute so from scratch daily. Would it not make sense to model this as a sliding window that's updated daily?
- nikhilsimha 2y agoWe do this for training data generation already. We have plans to implement this behavior for computing the batch arm of feature serving.
- evolutionblues 2y agocan you point to documentation/ examples which talk about it - I could not find this while exploring the pages (or maybe I missed it altogether).