I would start with this submission by my colleague Nishad Gothoskar to NeurIPS: https://proceedings.neurips.cc/paper/2021/hash/4fc66104f8ada6257fa55f29a2a567c7-

I would start with this submission by my colleague Nishad Gothoskar to NeurIPS: https://proceedings.neurips.cc/paper/2021/hash/4fc66104f8ada6257fa55f29a2a567c7-Abstract.html https://proceedings.neurips.cc/paper/2021/hash/4fc66104f8ada... as an excellent starting point to what good inference abstraction can enable