This would be fine on a per user basis, but if your "downvote" informs baseline results, it can also be gamed, with people burying each others' sites by sending

This would be fine on a per user basis, but if your "downvote" informs baseline results, it can also be gamed, with people burying each others' sites by sending malicious negative signals to google about them.