4 ms·
OpenXLA is an optimizing compiler... It's main purpose is to optimize stuff... So why does there seem to be no published metrics showing performance of vari
by londons_explore 4y ago
OpenXLA is an optimizing compiler... It's main purpose is to optimize stuff...
So why does there seem to be no published metrics showing performance of various common ML models on common hardware with OpenXLA vs other frameworks/compilers?
- londons_explore 4y agoI kinda get the feeling everyone wants the optimizing compiler framework to be out there so that it gets widespread use... But people like Google offering ML compute as a service want to keep the actual optimization modules secret and closed source. That way they can make their hardware perform superspeed while competitors hardware looks slow.
- VHRanger 4y agoThat's the Cuda lock-in solution which sadly works well.
- VHRanger 4y agoBecause there's a bunch of layers between the high level code and the output? For simple stuff, we can compare JAX to PyTorch on a 4090, and JAX seems faster by 10-50%. It's way way way faster on TPU. That said, JAX is apparently a pain to work with in comparison to PyTorch (disclaimer I haven't used JAX enough to opine yet). I'd also want a comparison between JAX and Taichi-lang for parallel stuff, even though the problem domains aren't exactly aligned.
- chillee 4y ago> For simple stuff, we can compare JAX to PyTorch on a 4090, and JAX seems faster by 10-50%. It's way way way faster on TPU. What benchmarks are you looking at here?
- VHRanger 4y agoIndividual user benchmarks on github and 2-3 blogs. Not super scientific, Im sorry to say, but methodologically benchmarking this is really hard.
- Joky 4y agoAll of Google TPU is powered by the XLA compiler, so any MLPerf benchmark result from Google comes powered by XLA. Anything JAX is also built on top of XLA, so you can take JAX performance as a point of comparison as well if you'd like.