4 ms·
Good move given some experienced issues and compaction across the 5.6 range is closer to 5.4 than 5.5, i.e solid and reliable. Will say that 5.6-Sol is a minor
by Topfi 3mo ago
Good move given some experienced issues and compaction across the 5.6 range is closer to 5.4 than 5.5, i.e solid and reliable.
Will say that 5.6-Sol is a minor bump in my benchmarks in most areas vs 5.5 but a severe regression in a few specific task focused on rearranging trees, addressing merge conflicts, etc. where the model to accomplish the task does not properly adhere to prompts in a way GPT-5 originally managed, not retaining parts of history in the way prompted despite specific instructions not to as that made the final completion easier…
I am of the conservative and cautious opinion that no model should be able to run destructive tasks at all, I have seen every model do things that make me concerned enough to maintain that opinion and know my evals can’t catch everything. But for 5.6-Sol specifically, I’d caution everyone to reevaluate how you run the model, maybe take a few more precautions you tend to forgo.
It is extremely capable as a reviewer and for extensive tasks, though for the later, the safety net I feel is required to be comfortable limits the utility. The code 5.6-Sol provides also still is a bit harder to parse in reviews.
Release strategy wise, feel it’s have been smarter to release only Luna and Sol now, then Terra a few weeks of posttraining later, I simply cannot see a purpose for it in the current form given how well both Luna and Sol scale up and down respectively with reasoning. Two models from a lab at a time is also the limit I feel one can properly assess at a time.