Models move so fast I question if papers like this are worth much. Out of curiosity I ran the clocks through Gemma-2-27B it did ok. Pretty consistently it woul

Models move so fast I question if papers like this are worth much. Out of curiosity I ran the clocks through Gemma-2-27B it did ok. Pretty consistently it would mix up the hour/minute hand but was decent at actually saying what they were pointing at. It never acknowledged the second hand and sometimes mistook that for the hour hand Interestingly it got the roman numeral one spot on. (minus the seconds) I wonder if it had more training data with that style clock.