Guidelines are written by product, legal, privacy, comms, policy and other folks in HQ - raters (psychopath or not) do not really influence them. Guidelines in

Guidelines are written by product, legal, privacy, comms, policy and other folks in HQ - raters (psychopath or not) do not really influence them. Guidelines in my experience suffer from "design by committee" syndrome but everyone working on them is pretty empathetic and genuinely tries to do best inside constraints. Acceptable/Unacceptable violence is clearly defined in guidelines with examples - no moral judgement from rater is required. On raters - they are doing pretty mechanical work and we generally do robust quality checks. Some % of tasks are randomly sent to two raters and if there is disagreement, task is sent to auditor. Raters with high disagreement rate with auditor are coached and if disagreement rate contienues to be high, they are fired. Overall quality is controlled by using "golden set" - tasks that have "right" answer decided by HQ. We look at disagreement between raters and right answer. There are other ways to control for quality - this is all pretty standard and everyone is doing the same things. Does not really matter if rater is phychopath or not if he can follow guidelines.