3 ms·

So, using the smallest model for the task would help, as expected. A very small model could run on device to automatically switch and choose the right model ba

• potatolicious 1y ago
  > ” A very small model could run on device to automatically switch and choose the right model based on the request.” This is harder than it looks. A “router” model often has to be quite large to maintain routing accuracy, especially if you’re trying to understand regular user requests. Small on-device models gating more powerful models most likely just leads to mis-routes.