3 ms·
If you have a good model router, you can route to older, cheaper models that run on older hardware, for simpler tasks. That helps labs extend the economic life
by try-working 4mo ago
If you have a good model router, you can route to older, cheaper models that run on older hardware, for simpler tasks. That helps labs extend the economic life of their hardware investments. They will likely fight it at first though as they see it as reducing ASP.
This is why I'm building role-model, a routing protocol and a router runtime: https://role-model.dev/ https://role-model.dev/
- jurgenburgen 4mo agoRunning cheaper models on newer hardware is always going to beat running them on older hardware.