I think I follow, and I agree. How I view the natural numbers on this fundamental level – and I would say it is the picture that the Peano axioms nail down – is

I think I follow, and I agree. How I view the natural numbers on this fundamental level – and I would say it is the picture that the Peano axioms nail down – is as consisting of a first element and then an infinite chain of elements after it (equivalently: the natural numbers are a chain that extends infinitely in one direction but not in the other). As you say, it does not matter, then, if that first element is called "0", "1", or "17" (until we bestow additional structure on the numbers by introducing addition and multiplication operators).