4 ms·
I like the use of compression to find out about repetitions. There is some theory out there called Kolmogorov Complexity [0]. It says that something is as comp
by woliveirajr 9y ago
I like the use of compression to find out about repetitions.
There is some theory out there called Kolmogorov Complexity [0]. It says that something is as complex as how much information you need to express it. In your case, lyrics are as complex as how many symbols (letters? words? bytes?) you need to represent it.
And one good way to calculate it is as you done: compress it. If you're using the same compression method for all the lyrics, you'll find that the ones that are more simple (and more repetitive) are the ones that have a great reduction on their sizes. In that case, the choice of which compression method you use is somehow irrelevant. Had you used Bzip, PPMD, etc., the results probably would be similar.
In case you want to extend your research, for example, as 6stringmerc said, you might consider that the composer matters more than the actual artist.
And, for that, you can use Normalized Compression Distance (NCD) [1]. That way you can measure how two lyrics are similar. Basicaly, you compress those lyrics together. When they are similar, clues from one are used by the compression to also compress the second one, so similar lyrics get more compression than lyrics that aren't related.
And by doing that you can even discover who was the composer of the songs, i.e., the authorship of the lyrics, since each person usually has the same writing style... [2]
[0] https://en.wikipedia.org/wiki/Kolmogorov_complexity https://en.wikipedia.org/wiki/Kolmogorov_complexity
[1] https://en.wikipedia.org/wiki/Normalized_compression_distance https://en.wikipedia.org/wiki/Normalized_compression_distanc...
[2] https://link.springer.com/chapter/10.1007%2F978-3-642-34475-6_76 https://link.springer.com/chapter/10.1007%2F978-3-642-34475-...