3 ms·
I always thought that the frontier coding models were specifically trained to perform well in the harness from their providers. I imagine this will always be th
by prettyblocks 2mo ago
I always thought that the frontier coding models were specifically trained to perform well in the harness from their providers. I imagine this will always be the case to some extent.
- zahrevsky 2mo agoYes, and the article makes the point that this might be changing.
- tudelo 2mo agoFrom what I understand pre-training is totally irrelevant to this and as far as post training goes there will be multiple steps, for claude and codex and the like that ship with a harness, the harness is definitely included in evaluation. However, they will definitely include evaluation from a variety or even none, and settle on something that works the "best" for a release. Disclaimer: No first hand knowledge
- strobe 2mo agoeven if they are is no guarantee that for 'your' specific tasks is beneficial in any means