Inference should be fast, so an async view (which we're excited finally happened!) should be fine. It's similar to waiting on a db. Read consistency means using

Inference should be fast, so an async view (which we're excited finally happened!) should be fine. It's similar to waiting on a db. Read consistency means using similar memory. Likewise, you want it to be warm, not reloaded between calls. In our case, we do on-the-fly GPU calls, with weird spiking behavior, so been looking for a good serverless GPU solution for this kind of stuff.