The problem with this view is, it ignores that some of what we learn are very strong constraints, such as conservation laws and how they're connected to physica

The problem with this view is, it ignores that some of what we learn are very strong constraints, such as conservation laws and how they're connected to physical symmetries of systems. Strictly speaking, all we know boils down to probabilistic statements. So yes, in some absolute sense, we might discover we're all totally wrong about something really fundamental. But we also know the bound on that probability is very, very, very, very, low.