4 ms·
These seem like amazingly egregious mistakes MS made? Or is it not as bad as it seems? I suppose, I'm curious how these kinds of mistakes happen for a model re
by dorian-graph 2y ago
These seem like amazingly egregious mistakes MS made? Or is it not as bad as it seems? I suppose, I'm curious how these kinds of mistakes happen for a model release.
- danielhanchen 2y agoThis happens quite often sadly - for example I fixed 8 bugs in Gemma https://x.com/danielhanchen/status/1765446273661075609 https://x.com/danielhanchen/status/1765446273661075609, multiple bugs in Llama, Mistral, Qwen, a gradient accumulation bug https://x.com/danielhanchen/status/1846235913443262891 https://x.com/danielhanchen/status/1846235913443262891 etc I wouldn't blame model training teams - sadly it's relatively hard to coordinate large teams so it might have been overlooked. But hey - I'm always here to fix them up :))
- dorian-graph 2y agoFair enough! I haven't been through the train and release of a model like this, so it's new to me. Thanks for your work! ;)