4 ms·
For LLMs I recommend just doing NDJSON, that is, newline delimited json. It's much simpler to implement
by rokkamokka 1y ago
For LLMs I recommend just doing NDJSON, that is, newline delimited json. It's much simpler to implement
- rictic 1y agoDo any LLMs support constrained generation of newline delimited json? Or have you found that they're generally reliable enough that you don't need to do constrained sampling?
- sprobertson 1y agonot for the standard hosted APIs using structured output or function calling, best you can get is an array
- stevage 1y agoI love NDJSON in general. I use it a lot for spatial data processing (GDAL calls it GeoJsonSeq).