Something like what you describe is more or less how statisticians do in fact reason about statistics: https://en.wikipedia.org/wiki/Sufficient_statistic#Minima

Something like what you describe is more or less how statisticians do in fact reason about statistics: https://en.wikipedia.org/wiki/Sufficient_statistic#Minimal_sufficiency https://en.wikipedia.org/wiki/Sufficient_statistic#Minimal_s..., though only for i.i.d. data from a parametric distribution.