I could see how Rust could be a good option for writing the prediction server. In my use case, DL models are trained offline. It does not matter if it’s faster

I could see how Rust could be a good option for writing the prediction server. In my use case, DL models are trained offline. It does not matter if it’s faster by 30 mins in training (maybe when I R&D-ing). On the other hand, you typically have to reach for something like c++ for a low latency, high thruput environment. It can be a bear to write a server. Anything more ergonomic would be welcome