You are right. The degree of dependence matters though. Mutual information is one way to measure that. Thankfully, some theorems remain valid even when indepe

You are right. The degree of dependence matters though. Mutual information is one way to measure that. Thankfully, some theorems remain valid even when independence is violated. The next stop after independence is martingale criteria. Martingale difference sequences can be quite strongly dependent yet allow some of the usual theorems to go through but with worse convergence rates.