Most QLoRAs only target attention layers. This gives pretty good performance for minimal processing time. But this one adds adapters to all linear layers. Also

Most QLoRAs only target attention layers. This gives pretty good performance for minimal processing time. But this one adds adapters to all linear layers. Also I'm not sure what a "step" is but they ran the whole dataset through at least twice.