This feature has existed for quite some time in several inference libraries, like Outlines, under the names "constrained decoding" or "guided decoding". Some ev

This feature has existed for quite some time in several inference libraries, like Outlines, under the names "constrained decoding" or "guided decoding". Some even include it in their OpenAI-compatible API in a very similar form (allowing to pass in a JSON Schema). All this required doing your own inference, though -- so the announcement really just brings this popular feature "to the masses".