3 ms·

So I can use this with any supported models? The reason I'm asking is because I can only run 1b-3b models reliably on my hardware.

• parthsareen 2y ago
  Hey! Author of the blog post here. Yes you should be able to use any model. Your mileage may vary with the smaller models but asking them to “return x in json” tends to help with accuracy (anecdotally).
  • pamelafox 2y ago
    Do you happen to know if got-4o would be negatively affected by the addition of “return x in json”? I’m debating whether I could use the same prompt across all models, hosted and ollama.
  • dcreater 2y ago
    Why do smaller models fail to follow? Isn't the objective of constraints that it always provides the right output type?
    • parthsareen 2y ago
      The constraints will always be met. It’s the data inside that might be inaccurate. YMMV with smaller models in that sense.