3 ms·
16 seems like a suspiciously round number ... why not 17 or 13? ... is this just result of some bug in the code they used to do their science? or is it just th
by scotty79 11mo ago
16 seems like a suspiciously round number ... why not 17 or 13? ... is this just result of some bug in the code they used to do their science?
or is it just that 16 was arbitrarily chosen by them as close enough to the actual minimal number of dimensions necessary?
- altairprime 11mo agoThere’s lots of hockey stick charts in the paper that might answer this visually, if that’s of interest.
- woopsn 11mo agoIt's a little arbitrary. Look at the graph on page 6, there's no steep gap in the spectrum there. 16 just about the balance point
- moi2388 11mo agoBut there is a steep gap in the spectrum at 16 on page 7
- yorwba 11mo agoThat's the spectrum of LoRAs, which are LoW RAnk by design.
- moi2388 11mo agoYes. But from their paper: “In our analysis, we present compelling empirical evidence for the existence of universal subspaces within LoRA adapters across different modalities and tasks.” I also don’t understand what they write under figure 2, since resnet50 has 50 layers, not 31.