4 ms·
How much do the labs post-train on the harness inputs & outputs? That's a critical piece to understand if a "generic" harness is at all possible
by shay_ker 3mo ago
How much do the labs post-train on the harness inputs & outputs? That's a critical piece to understand if a "generic" harness is at all possible
- yencabulator 3mo agohttps://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/ https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/ says already too much to use foreign tools.
- simonreiff 3mo agoI think tools have to be flexible enough to accommodate a range of expected inputs. Even assuming models are trained to supply a particular schema, they are stochastic and randomly output variants regularly. My experience has been that it's better to try to auto-normalize the range of anticipated outputs and make tools as forgiving as possible rather than force an overly strict schema that might not match training data on the model.
- edg5000 3mo agoI've experimented with completely novel tool call syntaxes combined with ordering and structuring the context differenly, and they seem to have a high tolerance for this stuff. But you always risk losing performance, probably highly unlikely to exceed stock performance across the board.