The fine-tuning is nice! It is only SFT, though, which may have limited its improvements. I tested it against the Zephyr 7B model (which used DPO) on a sample q

The fine-tuning is nice! It is only SFT, though, which may have limited its improvements. I tested it against the Zephyr 7B model (which used DPO) on a sample question: How can I code p*log(p) in Julia in a way that doesn't return NaN for any probability value? Yi gave: function safe_p_log(p::Float64) if p > 0.999 && p < 1.001 # or use a more precise threshold if needed return 1 - (1 / p) elseif p > 0.001 && p < 0.01 # or use a more precise threshold if needed return log(p * 1e8) # or use another small number instead of 1e-8 end return p * log(p) end It warned that it would not work for p = 0 or 1. After saying that probabilities can be 0 or 1, it added a `if p == 0.0 || p == 1.0; return NaN` at the start. Zephyr answered: function p_log_p(p::AbstractFloat) :: AbstractFloat return (p <= 0.0) ? 0.0 : p * log(p) end Sometimes it returns an error or minus infinity (I also got `p * log(max(min(p, 1.0), 1e-10))`, which is cute), but it does follow the request.