I don't know much about the software-side of nvidia/gpus + LLMs. Can you catch me up on what software they've created means as a differentiator? Is that CUDA?

I don't know much about the software-side of nvidia/gpus + LLMs. Can you catch me up on what software they've created means as a differentiator? Is that CUDA? How does this relate to things like tensorflow with Google's chips?