The tasks aren't arbitrary. They're meant to be a proxy for some universal concept of competence. This seems at least somewhat wrong to me - the competence is

The tasks aren't arbitrary. They're meant to be a proxy for some universal concept of competence. This seems at least somewhat wrong to me - the competence is not universal but task specific. They compare how your competence to perform task X is related to your ability of assessing your performance of task X in absolute terms and relative to the other participants. They repeat this for different tasks and find that for all tested tasks the same pattern emerges - roughly, the better your performance, the better your ability to accurately assess your own performance and the performance of others. So you can be competent doing task X and provide accurate assessments for task X performances while at the same time being incompetent doing task Y and being less accurate in assessing task Y performances. This essentially means that you can not be universally good at assessing performances of arbitrary tasks, you can only do this well for tasks for which you are yourself competent.