You would have the same issue with TLS or any other scheme. When OpenNTPD added TLS-authenticated constraints, I immediately began having problems on one of my

You would have the same issue with TLS or any other scheme. When OpenNTPD added TLS-authenticated constraints, I immediately began having problems on one of my PCEngines APU with a broken RTC (same model as all my other ones, but clock would always reset when power removed, no matter how many times I changed the battery). I had to disable TLS constraints. Eventually they added a special "trusted" qualifier to NTP server directives for bypassing constraint checking. Authenticated time synchronization faces intrinsic dilemmas no matter the technology.