In any case, yes: If it's a userspace-based model with kernel modules (like nvidia) - yes, you can insert/remove kernel modules without rebooting and exchange

In any case, yes: If it's a userspace-based model with kernel modules (like nvidia) - yes, you can insert/remove kernel modules without rebooting and exchange userspace libs freely. If it's an in-kernel driver part like the i915 module that comes bundled with your distro's kernel, you can kexec the new kernel, which is simple with systemctl kexec. It will likely require to exit and restart the graphical session though.