4 ms·
Curious about: does it have fault tolerance if one of the machines goes down mid-inference? Can it dynamically reroute, or does it just retry?
by DerivativeBS 3mo ago
Curious about: does it have fault tolerance if one of the machines goes down mid-inference? Can it dynamically reroute, or does it just retry?
- i386 3mo agoIt can dynamically route. If a machine drops out of split, the topology is recalculated and the request is automatically retried.