3 ms·
Quoting the article's own tl;dr: > I examined all the biorisk-relevant citations from a policy paper arguing that we should ban powerful open source LLMs. > N
by slowmovintarget 3y ago
Quoting the article's own tl;dr:
> I examined all the biorisk-relevant citations from a policy paper arguing that we should ban powerful open source LLMs.
> None of them provide good evidence for the paper's conclusion. The best of the set is evidence from statements from Anthropic -- which rest upon data that no one outside of Anthropic can even see, and on Anthropic's interpretation of that data. The rest of the evidence cited in this paper ultimately rests on a single extremely questionable "experiment" without a control group.
> In all, citations in the paper provide an illusion of evidence ("look at all these citations") rather than actual evidence ("these experiments are how we know open source LLMs are dangerous and could contribute to biorisk").
> A recent further paper on this topic (published after I had started writing this review) continues this pattern of being more advocacy than science.
> Almost all the bad papers that I look at are funded by Open Philanthropy. If Open Philanthropy cares about truth, then they should stop burning the epistemic commons by funding "research" that is always going to give the same result no matter the state of the world.
The rest of the paper supports this thesis with... wait for it... evidence!
- AnimalMuppet 3y ago"burning the epistemic commons". Beautiful phrase. I may steal it.
- deleted 3y ago[deleted]
- artninja1988 3y ago> Almost all the bad papers that I look at are funded by Open Philanthropy. If Open Philanthropy cares about truth, then they should stop burning the epistemic commons by funding "research" that is always going to give the same result no matter the state of the world. I see they are in the effective altruism space. Something about them seems extremely shady to me...
- matheusmoreira 3y ago> policy paper arguing that we should ban powerful open source LLMs This is a corporation lobbying effort, right? I simply can't believe that anyone else would actually want the power of LLMs to be concentrated in the hands of trillionaire corporations.
- quietpain 3y agoI think this is more about the offline aspect of open source LLMs. If someone with bad intentions does his research to harm a group, he’ll leave breadcrumbs that can be sniffed up and eventually ring a bell. If the whole process is offline: no breadcrumbs. In other words: we can kiss offline LLMs goodbye in the near future. My guess is LLMs will be blamed for sole guy building his contraption that gets him in the news and the whole thing will get massive media attention.