There is a lot of literature, but it may not be easy to find out, and different domains use different names for it. One keyword to look for is 'stochastic appr

There is a lot of literature, but it may not be easy to find out, and different domains use different names for it. One keyword to look for is 'stochastic approximation', with a long (> 50 years) history in engineering and stats. Another one is 'stochastic gradient', which got a resurgence following the whole deep learning craze. There is also 'particle filtering' in signal processing. One generic article I may recommend is http://leon.bottou.org/publications/pdf/online-1998.pdf http://leon.bottou.org/publications/pdf/online-1998.pdf, it does not require much math knowledge, but that may be less practical than what you want. The 'onepass' keyword mentioned in the original article is another keyword to look for (see e.g. interesting work to do Monte-Carlo based linear algebra operations for very large matrices, this used to be a reference, not sure what's the state of the art is: http://cs-www.cs.yale.edu/homes/mmahoney/pubs/matrix1_SICOMP.pdf http://cs-www.cs.yale.edu/homes/mmahoney/pubs/matrix1_SICOMP...)