3 ms·

Using this feature will obviously "lock you in" to OpenAI, specifically to this model too, at least until other companies catch on. While text prompts can more

Using this feature will obviously "lock you in" to OpenAI, specifically to this model too, at least until other companies catch on. While text prompts can more easily be moved to other LLMs, this feature cannot currently be ported as such. I would use it only if a text prompt is insufficient despite retries.

BoorishBears 2y ago
It's already supported by multiple other providers. Fireworks, Together, probably more.
dtquad 2y ago
OpenAI-style JSON mode and function calling rapidly became the industry standard way of doing it. It will probably also happen for this feature.
- toomuchtodo 2y ago
  “S3 compatible”
PufPufPuf 2y ago
This feature has existed for quite some time in several inference libraries, like Outlines, under the names "constrained decoding" or "guided decoding". Some even include it in their OpenAI-compatible API in a very similar form (allowing to pass in a JSON Schema). All this required doing your own inference, though -- so the announcement really just brings this popular feature "to the masses".
moralestapia 2y ago
OTOH, not using it could "lock you out" of building a cool product for your users, so ...
- OutOfHere 2y ago
  It depends. Quite often the intended schema is simple enough that parsing of a plaintext lines using a regex is sufficient. I do however have to use a verbose prompt to get it to comply with the expected format. Combine this with retries in case of rare failures, and the job is done. I think this new feature is more relevant for intricate schemas and dynamic schemas when a text prompt cannot do the job.
faizshah 2y ago
The converse API in AWS bedrock lets you use function calling across a number of different providers (doesn’t support OpenAI): https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html https://docs.aws.amazon.com/bedrock/latest/userguide/convers... I have been using it so that my agents aren’t specific to a particular model or api. Like others have said many other providers already have function calling and json schema for structure outputs.

BoorishBears 2y ago

It's already supported by multiple other providers. Fireworks, Together, probably more.

dtquad 2y ago

OpenAI-style JSON mode and function calling rapidly became the industry standard way of doing it. It will probably also happen for this feature.

toomuchtodo 2y ago
“S3 compatible”

PufPufPuf 2y ago

This feature has existed for quite some time in several inference libraries, like Outlines, under the names "constrained decoding" or "guided decoding". Some even include it in their OpenAI-compatible API in a very similar form (allowing to pass in a JSON Schema). All this required doing your own inference, though -- so the announcement really just brings this popular feature "to the masses".

moralestapia 2y ago

OTOH, not using it could "lock you out" of building a cool product for your users, so ...

OutOfHere 2y ago
It depends. Quite often the intended schema is simple enough that parsing of a plaintext lines using a regex is sufficient. I do however have to use a verbose prompt to get it to comply with the expected format. Combine this with retries in case of rare failures, and the job is done. I think this new feature is more relevant for intricate schemas and dynamic schemas when a text prompt cannot do the job.

faizshah 2y ago

The converse API in AWS bedrock lets you use function calling across a number of different providers (doesn’t support OpenAI): https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html https://docs.aws.amazon.com/bedrock/latest/userguide/convers... I have been using it so that my agents aren’t specific to a particular model or api. Like others have said many other providers already have function calling and json schema for structure outputs.