The linked repository has demos you can run (though you have to install bazel), and some of the smaller models run inference in about a second, while the larger

The linked repository has demos you can run (though you have to install bazel), and some of the smaller models run inference in about a second, while the larger ones take minutes. That said, there is a lot of ongoing work on GPU acceleration. Cf. the recent FHE-based CIFAR demo that runs in 200ms: https://sofar.belfortlabs.cloud/ https://sofar.belfortlabs.cloud/ Still maybe 1000x slower than cleartext, but progress!