Roughly they use a model of the face, render it, compare the source image and the rendered image, estimate changes of position, orientation and deformation for

Roughly they use a model of the face, render it, compare the source image and the rendered image, estimate changes of position, orientation and deformation for the model that will yield a better match and then just repeat this until the result is good enough. While you probably can exploit temporal coherence between frames the process is inherently pretty expensive due to its iterative nature. But it may also be relatively easy to parallelize bbecause of this.