For uranium, there is 510 million square km of surface area on earth, what's the issue with "destroying" (a lot of the mines are in pretty uninhabitable places)

For uranium, there is 510 million square km of surface area on earth, what's the issue with "destroying" (a lot of the mines are in pretty uninhabitable places) a few sq km out of 510 million to power the human race?