Cool work! It feels like the improvement is a little overstated due to how you're measuring - your measurements include import/setup time so you get big gains b

Cool work! It feels like the improvement is a little overstated due to how you're measuring - your measurements include import/setup time so you get big gains by improving imports. But in reality, you won't be creating a new client for each request and client import/setup time is unrelated to TF serving performance. TF serving performance is really about the time elapsed between request received and response returned.