2 ms·
> amortizing across a batch size of 1000 This put the problem outside of L2 and thus is not a fair comparison, especially if your problem is small enough to fi
by jxy 4y ago
> amortizing across a batch size of 1000
This put the problem outside of L2 and thus is not a fair comparison, especially if your problem is small enough to fit in L1.
In addition, XLA does a poor job in optimizing for CPU cache reuse, so I'm not surprised that directly transcoded C++ code would be much faster than XLA.
However, TensorFlow code can directly run on GPU. SymForce cannot.
- haykmartiros 4y agoTo be clear, we tested a large set of batch sizes from 1 up to ~1e8. The smaller the batch size, the less competitive JAX was. The crossover point for JAX to be faster was amortizing across a batch size in the tens of thousands I believe. The ideal batch size for some of our applications is one to five. Please see symforce/benchmarks/notebooks for some of our timing. SymForce is not trying to compute on large data-parallelism, but it could generate an inner kernel for that. In that way it complements TensorFlow nicely.
- crubier 4y ago> TensorFlow code can directly run on GPU. SymForce cannot. I think this is just a matter of tweaking the code generator, and not a hard limitation of Symforce.