I was under the impression the driver has to be loaded before the kernel can detect if it can use the hardware (that’s how probing works) so it’s the user space

I was under the impression the driver has to be loaded before the kernel can detect if it can use the hardware (that’s how probing works) so it’s the user space that picks the modules to load. Of course the driver probably won’t do anything if it never gets attached to a device (other than responding to probe requests on the bus it uses.)