This is my understanding as well. Section 3 sub "A decentralized algorithm" of their paper discusses the computation approach: "In practice we can assign each

This is my understanding as well. Section 3 sub "A decentralized algorithm" of their paper discusses the computation approach: "In practice we can assign each eigenvector update to its own device (e.g. a GPU or TPU). Systems with fast interconnects may facilitate tens, hundreds or thousands of accelerators to be used. In such settings, the overhead of broadcast(vˆi) is minimal. We can also specify that the data stream is co-located with the update so vˆi updates with respect to its own Xi,t. This is a standard paradigm for e.g. data-parallel distributed neural network training. We provide further details in Section [6]." So a general methodology that permits mapping analysis to "embarrassingly parallel" computational models. Tangent: Given the provenance of PCA (Karl Pearson) this is all a bit ironic ..