5 ms·

Not really, but one can report a false positives and false negatives here: https://meta.wikimedia.org/wiki/Research_talk:Revision_scoring_as_a_service#Misclassi

Not really, but one can report a false positives and false negatives here: https://meta.wikimedia.org/wiki/Research_talk:Revision_scoring_as_a_service#Misclassifications https://meta.wikimedia.org/wiki/Research_talk:Revision_scori... Since it is activated on Indonesian Wikipedia (I helped them to extend the tool to Indonesian language), I noticed that this tool can hardly capture an obvious vandalism yet. Some other edits are hard to verify whether it is vandalism or not, even by human. I believe this is a work in progress and hope that it will improve over time. But hey, it is the first time a vandalism detection tool is deployed in Indonesian Wikipedia. :)

jacquesm 11y ago
> Not really That's worrisome then. You'd expect a bit more rigor before putting a bot like that into production.
- tptacek 11y ago
  It's not actually doing reversions, right? It's just scoring edits. Wikipedia editors keep hot-lists of articles they watch edits for, or patrol lists of edits or page creations; ostensibly, all this needs to do is sort those lists of edits.
  - kenrick95 11y ago
    Yup, it's just scoring the revisions. Currently there is no straightforward way to feedback to the tool for learning false positives yet.
  - jacquesm 11y ago
    It's augmentation, not making any decisions on its own (and that's logical given the false positive rate). So it's just an aid at this point but it can probably help quite a bit with triage.
- halfak_wmf 11y ago
  Actually, we openly report a wide variety of statistics about our models. E.g. if you go to https://ores.wmflabs.org/scores/enwiki https://ores.wmflabs.org/scores/enwiki, the service lists out the models that are available for English Wikipedia along with test statistics. You can also browse our test statistics in our documentation. See https://meta.wikimedia.org/wiki/ORES/damaging#English_Wikipedia_.28enwiki.29 https://meta.wikimedia.org/wiki/ORES/damaging#English_Wikipe... Currently, we're optimizing for AUC since it (1) is hard to cheat -- unlike accuracy or precision/recall individually and (2) it represents the ranking problem well. See https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve https://en.wikipedia.org/wiki/Receiver_operating_characteris... "[AUC] is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one"
  - jacquesm 11y ago
    Thank you. 75%, that's not bad but it's also not exactly 'x-ray eyes' grade yet. This is a pretty tough problem you're tackling, I can see now why it is not a bot but an augmentation device. It would be nice to track the accuracy over time, to see if the vandalizers wise up to how the bot makes the decisions to game it.
    - halfak_wmf 11y ago
      Actually our AUC is 90% and that beats the state of the art in the literature. Accuracy is easy to game. We could get 97.5% accuracy by just predicting False all of the time. We don't advise reviewers to use the prediction directly, but rather to set thresholds on the probability. 70% probability is a very conservative filter and will catch about 10% and a little bit less than half of them will actually be damaging. So, this is useful enough to reduce the workload by 90% right now. As we increase the model's fitness, we should be able to drive that up to 95%, but I doubt we'll get much better than that.
      - jacquesm 11y ago
        > 70% probability is a very conservative filter and will catch about 10% and a little bit less than half of them will actually be damaging. Right. This is really interesting. A while ago I wrote a classifier that predicted based on a number of criteria if a new member of a web community would be trouble or not. It also worked as an aid in classification and not as a stand-alone decision maker. You have a lot more data to work with so that should make the problem a bit more tractable. Does it use other bits of knowledge about the edit prior probability of a revert for an edit by that particular user or network range for instance?

jacquesm 11y ago

> Not really That's worrisome then. You'd expect a bit more rigor before putting a bot like that into production.

tptacek 11y ago
It's not actually doing reversions, right? It's just scoring edits. Wikipedia editors keep hot-lists of articles they watch edits for, or patrol lists of edits or page creations; ostensibly, all this needs to do is sort those lists of edits.
- kenrick95 11y ago
  Yup, it's just scoring the revisions. Currently there is no straightforward way to feedback to the tool for learning false positives yet.
- jacquesm 11y ago
  It's augmentation, not making any decisions on its own (and that's logical given the false positive rate). So it's just an aid at this point but it can probably help quite a bit with triage.
halfak_wmf 11y ago
Actually, we openly report a wide variety of statistics about our models. E.g. if you go to https://ores.wmflabs.org/scores/enwiki https://ores.wmflabs.org/scores/enwiki, the service lists out the models that are available for English Wikipedia along with test statistics. You can also browse our test statistics in our documentation. See https://meta.wikimedia.org/wiki/ORES/damaging#English_Wikipedia_.28enwiki.29 https://meta.wikimedia.org/wiki/ORES/damaging#English_Wikipe... Currently, we're optimizing for AUC since it (1) is hard to cheat -- unlike accuracy or precision/recall individually and (2) it represents the ranking problem well. See https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve https://en.wikipedia.org/wiki/Receiver_operating_characteris... "[AUC] is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one"
- jacquesm 11y ago
  Thank you. 75%, that's not bad but it's also not exactly 'x-ray eyes' grade yet. This is a pretty tough problem you're tackling, I can see now why it is not a bot but an augmentation device. It would be nice to track the accuracy over time, to see if the vandalizers wise up to how the bot makes the decisions to game it.
  - halfak_wmf 11y ago
    Actually our AUC is 90% and that beats the state of the art in the literature. Accuracy is easy to game. We could get 97.5% accuracy by just predicting False all of the time. We don't advise reviewers to use the prediction directly, but rather to set thresholds on the probability. 70% probability is a very conservative filter and will catch about 10% and a little bit less than half of them will actually be damaging. So, this is useful enough to reduce the workload by 90% right now. As we increase the model's fitness, we should be able to drive that up to 95%, but I doubt we'll get much better than that.
    - jacquesm 11y ago
      > 70% probability is a very conservative filter and will catch about 10% and a little bit less than half of them will actually be damaging. Right. This is really interesting. A while ago I wrote a classifier that predicted based on a number of criteria if a new member of a web community would be trouble or not. It also worked as an aid in classification and not as a stand-alone decision maker. You have a lot more data to work with so that should make the problem a bit more tractable. Does it use other bits of knowledge about the edit prior probability of a revert for an edit by that particular user or network range for instance?