3 ms·
Ok I think I understand better now. In my mind there are two separate discussion points here: 1. How necessary is it to engineer a prompt for a specific model
by wbakst 2y ago
Ok I think I understand better now. In my mind there are two separate discussion points here:
1. How necessary is it to engineer a prompt for a specific model vs. build with the expectation that models will get better and better and require less and less engineering.
If the former is the most important, then there's nothing really to do here. The tool you're using should make it easy to use any provider you want with minimal effort / change, but the model/provider-specific prompt engineering will still be up to you. This would be best supported through a second layer of abstraction that sits on top of something like Mirascope. For example, a tool for automatically re-engineering a prompt for a specific model. Or perhaps versioning tooling to make it easy to take a snapshot of a prompt that's been engineered for a specific model and compare it to another snapshot for another prompt/model.
If the latter is true, then the most important aspect here is to make using and swapping between any and all providers extremely easy. I think we've done a good job of making this possible with Mirascope without taking away the control necessary for the former.
2. Static vs. Dynamic constructs.
We spent a lot of time supporting both static and dynamic flows. For dynamic flows (i.e. dynamic configuration), the configuration object is tied to the model on purpose. For example, if you want to write your own custom messages array (perhaps to take advantage of a model-specific feature), you can do this at the expense of losing provider-agnostic support.
I think these things are really important to think about and keep in mind as we continue to develop, but I think they are issues to be handled separately from (or in addition to) purely improving the ergonomics of calling LLM APIs.