I understand your point, that's why I tried to say that even a budget "evened out" over the study itself could be interesting to look at. There are fixed costs

I understand your point, that's why I tried to say that even a budget "evened out" over the study itself could be interesting to look at. There are fixed costs in the development of the system but there are running costs (owning or renting the servers, improving the ML and the system at all points, gathering more data, curating, maintenance of the data, etc). I don't think anyone reasonably believes that we can just build a box of truth and that nobody need be involved in that project ongoing afterwards (even just to ensure future browser compatibility, security upgrades, system outages, bias in data gathering, etc). What I would be interested in seeing is what the very same money put towards such a project would do for a system that desperately needs much more influx of money towards structural pillars like mental health, housing, legal counsel, etc. Since there's seemingly no A/B testing (with what I mentioned), no double-blind testing (which would test for the impact on the judges), or any form of comparative testing really, it's pretty hard to conclude a lot of things about this project. We surely wouldn't go too quickly to conclusions of usefulness if the subjects of the study were closer to us (i.e. if they weren't already assumed to have done something wrong to be in that situation). -- As for the bad outcomes, I wasn't so much talking at the system at the current instant, but more about the feedback loop this creates: suppose the rating is biased against a community of people such that their measured score is lower than their actual likelihood to show up, then the judge is less likely to give them a chance to be out, which then reinforces the social bias against the category of people and drives what was an initial bias to be a systemic issue. My masters is in Control Theory so I tend to see systems, feedback loops, and stability of trajectories everywhere, I guess.