I agree, but such a "program" wouldn't necessarily look/act anything like the "program" 'sin(1000000τ)'. What's interesting about systems like Turing machines

I agree, but such a "program" wouldn't necessarily look/act anything like the "program" 'sin(1000000τ)'. What's interesting about systems like Turing machines is that the "program" is a specific, isolated thing like a tape of symbols. In contrast, those "foundational mathematical principles" you mention are more of a diffuse web, with lots of interchangable pieces, all mixed up with unrelated facts. This gives lots of different ways to arrive at the answer. With Turing-complete systems, we always have to come back to the code.