I meant the p-value comparison to cutoff there. It is the test on probability of data being observed given that null hypothesis is true and error model matches

I meant the p-value comparison to cutoff there. It is the test on probability of data being observed given that null hypothesis is true and error model matches your assumption, typically assumed normal or studentized. Such as by chance or because the alternate hypothesis is bunkum or because your experiment lacks power. (It does not distinguish between these, and middle item is only minimally handled by most waste of obtaining it, esp. not by Fischer's or ANOVA which are facts about data only, using error in measurement model.) Thus p-value is a statement about validity of data (type 1 error, false positive) and maybe about the null hypothesis as a secondary notion. (E.g. about the control group of you use that as reference group for null.)