6 ms·

While the topic is intriguing, I dislike the use of "public services" for this type of research. For instance, adding substances to a water reservoir to study

by hp6 3y ago

While the topic is intriguing, I dislike the use of "public services" for this type of research. For instance, adding substances to a water reservoir to study their effects is unacceptable, without permission or supervision. Similarly, conducting such research without Wikipedia's permission/supervision should not be accepted.

kmeisthax 3y ago
Someone tried something similar but with higher risk: inserting security backdoors into the Linux kernel. They were caught and (AFAIK) their entire school was permabanned from sending pull requests.
- minetest2048 3y ago
  This was also my thought. Search for hypocrite commits, and a link to an lwn article: https://lwn.net/Articles/853717/ https://lwn.net/Articles/853717/ . They did ban their whole school
  - carbocation 3y ago
    The department said they'd report their findings to the community[1]. I wonder if they ever did? 1 = https://cse.umn.edu/cs/statement-cse-linux-kernel-research-april-21-2021 https://cse.umn.edu/cs/statement-cse-linux-kernel-research-a...
    - hakre 3y ago
      Isn't community plural? Perhaps they meant their own community but didn't say specifically which one?
    - Pazzaz 3y ago
      It's the other two links on that page: - https://cse.umn.edu/cs/open-letter-linux-community-april-24-2021 https://cse.umn.edu/cs/open-letter-linux-community-april-24-... - https://cse.umn.edu/cs/statement-computer-science-engineering-confirming-linux-technical-advisory-board-findings-may-9 https://cse.umn.edu/cs/statement-computer-science-engineerin...
      - carbocation 3y ago
        Thanks for pointing that out. That second link seems to close things out.
- hakre 3y ago
  Are you commenting about April of 2021 and do you mean the University of Minnesota? - https://news.ycombinator.com/item?id=26887670 https://news.ycombinator.com/item?id=26887670 - https://news.ycombinator.com/item?id=26949539 https://news.ycombinator.com/item?id=26949539 - https://news.ycombinator.com/item?id=26955414 https://news.ycombinator.com/item?id=26955414
dredmorbius 3y ago
That's frankly one of the first thoughts that came to my mind. I've asked the author about ethical review and processes on the Fediverse. That said, both Wikipedia and the Linux kernel (mentioned in another response to this subthread) should anticipate and defend against either research-based or purely malicious attacks.
- tetris11 3y ago
  If it's a mature product, you should be able to pick it up and rattle it without it breaking. If it's still maturing, then maybe the odd shock here and there will prepare it for maturity?
  - maxbond 3y ago
    It's true that the system must be tolerant to these sorts of faults, but that doesn't mean we have a right to stress it. The margin for error is not infinite, and by consuming some of it we increase the likelihood of errors going undetected for longer. Sometimes it will be worth it anyway, and I don't have an opinion about this Wikipedia example, but I think it's pretty uncontroversial that the Linux example was out of line.
viknesh 3y ago
I think one would have to weigh the pros and cons of this kind of research. In particular, the main cons (IMO) are: * users are misled about facts * trust is lost in Wikipedia * other users/organizations use this as a blueprint to insert false information Harm 3 seems to be the most serious, but I suspect it has happened/will happen irrespective of this research. As opposed to the water reservoir example, these harms seem quite small by contrast. I would have liked to see a section discussing this in the blog post, but perhaps that's included in the original paper.
- hakre 3y ago
  Everything was reverted with 48 hours, your arguments might all apply theoretically but given scope, size, practice and handling, I wonder - apart from the theory - what your opinion is how they practically apply for this case.
  - viknesh 3y ago
    I didn't make it very clear, but I agree that the specific example isn't problematic. The false claims weren't meant to be any sort of targeted disinformation, and like you mention they reverted it in 48 hours.
tobyjsullivan 3y ago
I'm of quite the opposite opinion. Within reason (importantly), I believe any public service, which is also managed by an anonymous, decentralized community, ought to be under test constantly and by anyone. What's the alternative, really? Imagine if it was taboo to independently test the integrity of bitcoin for example. The sibling mentioned the linux kernel case. I admit that one felt wrong. It was a legitimate waste of contributor time and energy, with the potential to open real security holes. I don't pretend to have reconciled why one seems right to me and the other wrong.
- hp6 3y ago
  I think the key difference is supervision, is there another party keeping an eye on what is tested and how. And maybe insuring no permanent damage is done at the end.
- dataflow 3y ago
  > Imagine if it was taboo to independently test the integrity of bitcoin for example. > The sibling mentioned the linux kernel case. I admit that one felt wrong. > I don't pretend to have reconciled why one seems right to me and the other wrong. The "how" is what matters here, not just the "what". "Testing the integrity of Bitcoin" by breaking the hash on your own machine (and publishing the results, or not) is one thing. "Testing" it by sending transactions that might drain someone else's wallet is quite another. Similarly with Linux, hacking it on your own machine and publishing the result is one thing. Introducing a potential security hole on others' machines is another. Similarly with water: messing with your own drinking water is one thing. Messing with someone else's water is quite another.
  - lll-o-lll 3y ago
    > Similarly with Linux, hacking it on your own machine and publishing the result is one thing. Introducing a potential security hole on others' machines is another. Playing devils advocate for a moment. How else do you test the robustness of the human process to prevent bad actors? Don’t you need someone to attempt to introduce a security hole to know that you are robust to this kind of attack?
deleted 3y ago
[deleted]

by hp6 3y ago

kmeisthax 3y ago

Someone tried something similar but with higher risk: inserting security backdoors into the Linux kernel. They were caught and (AFAIK) their entire school was permabanned from sending pull requests.

minetest2048 3y ago
This was also my thought. Search for hypocrite commits, and a link to an lwn article: https://lwn.net/Articles/853717/ https://lwn.net/Articles/853717/ . They did ban their whole school
- carbocation 3y ago
  The department said they'd report their findings to the community[1]. I wonder if they ever did? 1 = https://cse.umn.edu/cs/statement-cse-linux-kernel-research-april-21-2021 https://cse.umn.edu/cs/statement-cse-linux-kernel-research-a...
  - hakre 3y ago
    Isn't community plural? Perhaps they meant their own community but didn't say specifically which one?
  - Pazzaz 3y ago
    It's the other two links on that page: - https://cse.umn.edu/cs/open-letter-linux-community-april-24-2021 https://cse.umn.edu/cs/open-letter-linux-community-april-24-... - https://cse.umn.edu/cs/statement-computer-science-engineering-confirming-linux-technical-advisory-board-findings-may-9 https://cse.umn.edu/cs/statement-computer-science-engineerin...
    - carbocation 3y ago
      Thanks for pointing that out. That second link seems to close things out.
hakre 3y ago
Are you commenting about April of 2021 and do you mean the University of Minnesota? - https://news.ycombinator.com/item?id=26887670 https://news.ycombinator.com/item?id=26887670 - https://news.ycombinator.com/item?id=26949539 https://news.ycombinator.com/item?id=26949539 - https://news.ycombinator.com/item?id=26955414 https://news.ycombinator.com/item?id=26955414

dredmorbius 3y ago

That's frankly one of the first thoughts that came to my mind. I've asked the author about ethical review and processes on the Fediverse. That said, both Wikipedia and the Linux kernel (mentioned in another response to this subthread) should anticipate and defend against either research-based or purely malicious attacks.

tetris11 3y ago
If it's a mature product, you should be able to pick it up and rattle it without it breaking. If it's still maturing, then maybe the odd shock here and there will prepare it for maturity?
- maxbond 3y ago
  It's true that the system must be tolerant to these sorts of faults, but that doesn't mean we have a right to stress it. The margin for error is not infinite, and by consuming some of it we increase the likelihood of errors going undetected for longer. Sometimes it will be worth it anyway, and I don't have an opinion about this Wikipedia example, but I think it's pretty uncontroversial that the Linux example was out of line.

viknesh 3y ago

I think one would have to weigh the pros and cons of this kind of research. In particular, the main cons (IMO) are: * users are misled about facts * trust is lost in Wikipedia * other users/organizations use this as a blueprint to insert false information Harm 3 seems to be the most serious, but I suspect it has happened/will happen irrespective of this research. As opposed to the water reservoir example, these harms seem quite small by contrast. I would have liked to see a section discussing this in the blog post, but perhaps that's included in the original paper.

hakre 3y ago
Everything was reverted with 48 hours, your arguments might all apply theoretically but given scope, size, practice and handling, I wonder - apart from the theory - what your opinion is how they practically apply for this case.
- viknesh 3y ago
  I didn't make it very clear, but I agree that the specific example isn't problematic. The false claims weren't meant to be any sort of targeted disinformation, and like you mention they reverted it in 48 hours.

tobyjsullivan 3y ago

I'm of quite the opposite opinion. Within reason (importantly), I believe any public service, which is also managed by an anonymous, decentralized community, ought to be under test constantly and by anyone. What's the alternative, really? Imagine if it was taboo to independently test the integrity of bitcoin for example. The sibling mentioned the linux kernel case. I admit that one felt wrong. It was a legitimate waste of contributor time and energy, with the potential to open real security holes. I don't pretend to have reconciled why one seems right to me and the other wrong.

hp6 3y ago
I think the key difference is supervision, is there another party keeping an eye on what is tested and how. And maybe insuring no permanent damage is done at the end.
dataflow 3y ago
> Imagine if it was taboo to independently test the integrity of bitcoin for example. > The sibling mentioned the linux kernel case. I admit that one felt wrong. > I don't pretend to have reconciled why one seems right to me and the other wrong. The "how" is what matters here, not just the "what". "Testing the integrity of Bitcoin" by breaking the hash on your own machine (and publishing the results, or not) is one thing. "Testing" it by sending transactions that might drain someone else's wallet is quite another. Similarly with Linux, hacking it on your own machine and publishing the result is one thing. Introducing a potential security hole on others' machines is another. Similarly with water: messing with your own drinking water is one thing. Messing with someone else's water is quite another.
- lll-o-lll 3y ago
  > Similarly with Linux, hacking it on your own machine and publishing the result is one thing. Introducing a potential security hole on others' machines is another. Playing devils advocate for a moment. How else do you test the robustness of the human process to prevent bad actors? Don’t you need someone to attempt to introduce a security hole to know that you are robust to this kind of attack?

deleted 3y ago

[deleted]