Your point regarding anonymity is highly valid, and I appreciate that you bring it up. There is actually a concept I'm rather fond of that seeks to address this

Your point regarding anonymity is highly valid, and I appreciate that you bring it up. There is actually a concept I'm rather fond of that seeks to address this, usually referred to as peer-reviewed peer review [1]. Essentially, anonymous experts review publications in much the manner they do now, with no risk of their identities being revealed. A separate set of review-reviewers then rates the publication-reviewers themselves, scoring them on a standardized quantitative scale (1-10) based on the validity of their claims and the depth of their assessment. In this manner, experts collect an "aggregate reviewer rating" of sorts analogous to an h-index [2], call it an R-index. This R-index can then serve to select accurate reviewers for future publications. A high R-index could also be included as a positive bullet point on a CV, providing some reward for reviewers to put a high degree of effort into their otherwise unrewarded reviewing work. A few potential flaws are evident: what is one to do if early reviews are scored poorly, and you are no longer able to provide new reviews to increase your R-index? A grace period before scoring could potentially remedy this, ie) an R-index is not assigned until a reviewer has written >10 reviews. [1] - http://www.frontiersin.org/Computational_Neuroscience/10.3389/fncom.2012.00020/full http://www.frontiersin.org/Computational_Neuroscience/10.338... [2] - http://en.wikipedia.org/wiki/H-index‎ http://en.wikipedia.org/wiki/H-index‎