I don't know where you got the idea that quantifying something fixes it for all time. Numbers can change. Measures of inter-rater reliability can be applied to

I don't know where you got the idea that quantifying something fixes it for all time. Numbers can change. Measures of inter-rater reliability can be applied to multiple ratings over time by the same person (why not?)