3 ms·

They are very "stubborn" models Have you found this to be the case even when using the recommended temperature settings (ranging from 0 for math, to 1.5 for cr

• NitpickLawyer 2y ago
  I use 0.05 for math, just did a 5k problem set, trying to fine-tune a smaller model with the outputs. It has some very interesting training, borrowed from r1 per the tech report, where it does the o1/qwq "thinking steps", but a bit shorter. It solves ~80% of the problems in 4k context, while qwq would go on for 8k-16k. It's very good at what it does. But as soon as I need it to do something other than solve a problem - say rewrite the problem in simpler terms, or given a problem + solution provide hints, or rewrite the solution with these <tags>, etc. it kinda stops working. Often times it still goes ahead and solves the problem. That's why I'm saying it's stubborn. If a task looks like a task that it can handle very well, it's really hard to make it perform that other, similar but not quite the same task. In a similar vein - https://github.com/cpldcpu/MisguidedAttention/tree/main/eval#readme https://github.com/cpldcpu/MisguidedAttention/tree/main/eval...