What you describe can be formalized through the notion of Kolmogorov complexity [1], which relates to all sorts of interesting and fundamental concepts like ent

What you describe can be formalized through the notion of Kolmogorov complexity [1], which relates to all sorts of interesting and fundamental concepts like entropy and uncomputability. So I think you're absolutely right to call it fundamental! [1] https://en.wikipedia.org/wiki/Kolmogorov_complexity https://en.wikipedia.org/wiki/Kolmogorov_complexity