There is no dichotomy here. Mathematicians also use heuristics to figure out what is probably true. This is a very important part of mathematical activity - gen

There is no dichotomy here. Mathematicians also use heuristics to figure out what is probably true. This is a very important part of mathematical activity - generating good, not yet worked out ideas based on rough insights. What the physicists are doing here is also how the Fermat's Last Theorem was approached - a conjectural analogy to a different field(elliptic curves). The only difference is that they are more careful about separating heuristics from proofs. If you read physics textbooks, they are less interested in being formally precise. Mathematical activity is not all about formal reasoning, though that is the output.