3 ms·
In general it's an abstract concept like Kolmogrov complexity but there are some practical approaches. People often try to intuit the intrinsic dimensionality
by emcq 11y ago
In general it's an abstract concept like Kolmogrov complexity but there are some practical approaches.
People often try to intuit the intrinsic dimensionality of a dataset by using techniques like looking at singular values above some threshold or reconstruction error versus changing the output dimensionality of a dimensionality reduction/unsupervised technique like PCA, matrix factorization, or an autoencoder.
An info theory person might argue entropy and compression ratios are also insightful.
- TTPrograms 11y agoI'd also note that for the case of typical classification that there's an additional concept of "intrinsic boundary dimensionality" that should be low due to the presence of invariants like scale, rotation and translation which drastically lower the intrinsic boundary dimensionality.