We tried Inferentia & though they claim their compiler should "just work" on your existing models it was producing a bunch of NaNs for us & was opaque enough th

We tried Inferentia & though they claim their compiler should "just work" on your existing models it was producing a bunch of NaNs for us & was opaque enough that we didn't really have a path forward so we gave up on it. If we were operating at 100x or 1000x the scale it might make sense to spend the extra engineering time, but we just pay for the NVIDIA chips (especially because a big chunk of our customers run at the edge where NVIDIA compatibility is even more important so we'd be doing double the engineering on an ongoing basis).