3 ms·
in my opinion there is room for small and fast and large and slow but much smarter models. Use cases like phone keyboard autocomplete and next few words suggest
by msoad 2y ago
in my opinion there is room for small and fast and large and slow but much smarter models. Use cases like phone keyboard autocomplete and next few words suggestion in coding or writing need very fast models that should by definition small. Very large models that are much smarter are also useful, for instance debugging issues or proofreading long letters.
Cursor really aced this. The Cursor model is very fast to suggest useful inline completions and then leaves big problems to big models.
- mycall 2y agoCould chaining models together via tool calls based on benchmarking that would redirect to the best model allow for smaller models to perform as well as big[er] models for memory constrained/local environments?
- isoprophlex 2y agoYes, indeed, see for example https://arxiv.org/abs/2310.03094 https://arxiv.org/abs/2310.03094