6 ms·

The 2023 Hugo nomination statistics have been released and we have questions

80 points by aspenmayer 3y ago

Maskawanian 3y ago
Daniel Greene made a great video explaining this: https://youtube.com/watch?v=85CFDXrORdw https://youtube.com/watch?v=85CFDXrORdw
low_tech_love 3y ago
I know barely nothing about the Hugo process or Sci-Fi awards in general, but unfortunately this rings a bell when I think of my experience as an academic researcher (i.e. "paper writer") regarding some of my Chinese colleagues. No matter how much we want to believe that science (and especially scientific reporting) is objective, the truth is that there are certain unwritten, informal, and subjective rules in the world of academic writing / research reporting that one is simply expected to follow; it is impossible for every reviewer to check every single detail about every paper they accept to review. There are ethical/common sense aspects that are just supposed to be there: the data you report is supposed to be real; the method you report is supposed to have been conducted accurately; etc. However, when it comes to a large part of Chinese-authored publications, one cannot help unfortunately but to have sometimes a kind of a ear-scratching sensation that one is not 100% sure these aspects are really there. First, because we're absolutely flooded daily by Chinese-authored drafts, and I'm not talking about simple, small-scale research results; these are big sprawling research results, involving large datasets and lots of subjects, all coming in at a staggering rate, non-stop. We don't have enough volunteers to review these papers accurately and carefully, especially given that modern open-access journals want to have reviews (100% volunteered, without any pay, mind you) sent back in ridiculous times, like one week. And yes, many of these journals are either Chinese or managed by Chinese colleagues. So in summary we simply cannot manage to review them in time with the care that is necessary, which means that some undesirable work will invariably pass the check (statistically speaking, it will happen). One cannot also help but think that it seems unrealistic that all these research projects are being carried at such incredible pace and with so much efficiency. One thing that compounds this problem is that papers come initially with a very low quality. Slowly they become better; you may find yourself reviewing the same paper 5 or 6 times until you get so sick of it that you have no other choice but to say, "I have nothing else to add, just publish the goddamn thing." This is not how the system was designed to work; usually, the authors (and the PhD supervisor) are supposed to work on the paper a bit before submitting, so that they don't submit a horrible thing for reviewers to work with. I think this works usually on the basis of reputation and "name", but my impression is that the Chinese authors could not care less about this. As I usually say to my local colleagues, we are supervising the Chinese PhD students (via paper reviews) without getting paid for it. Some of the most incredible researchers and smartest people in the world are certainly Chinese and some of their research reports are just amazing. But the numbers are so staggering that invariably the classic system ends up breaking. And one cannot help but think that many of them have found optimized ways to take advantage of the system in exactly those parts were one cannot really point the finger at, because they are based on informal and subjective common-sense rules that are just expected to be there (or else the system does not work).
aspenmayer 3y ago
Original title edited for length: The 2023 Hugo Nomination Statistics Have Finally Been Released – And We Have Questions This post was brought to my attention via Mastodon via HN user cstross, who posted an entire thread about this here, the initial post of which I quote: https://wandering.shop/@cstross/111793448361231714 https://wandering.shop/@cstross/111793448361231714 > Chengdu worldcon Hugo nomination statistics suggest the 2023 Hugo awards were rigged: > [link to OP] > I've also seen it alleged that the concom offered some attendees "special guest" treatment … and stole their Hugo voting rights in return, without telling them about that part of the "deal" first. (I will not cite a source on this until the attendee in question goes public.) [Above repost, including my original comment, upon request via second chance pool. More info about that here: https://news.ycombinator.com/item?id=26998308 https://news.ycombinator.com/item?id=26998308]
- aspenmayer 3y ago
  Even more context via the Mastodon thread: A Comparison of Hugo Nomination Distribution Statistics https://alpennia.com/blog/comparison-hugo-nomination-distribution-statistics https://alpennia.com/blog/comparison-hugo-nomination-distrib... Edit: Even more context from the above link: https://mrphilipslibrary.wordpress.com/2024/01/21/hugo-nominating-stats-rascality-and-a-brief-history-of-where-it-all-started/ https://mrphilipslibrary.wordpress.com/2024/01/21/hugo-nomin... which has a meta-follow-up at: https://mrphilipslibrary.wordpress.com/2024/01/23/the-mess-dave-mccarty-left-behind/ https://mrphilipslibrary.wordpress.com/2024/01/23/the-mess-d...
  - aspenmayer 3y ago
    Related post on HN yesterday, from The Guardian: Science fiction awards held in China under fire for excluding authors https://www.theguardian.com/books/2024/jan/24/science-fiction-awards-held-in-china-under-fire-for-excluding-authors https://www.theguardian.com/books/2024/jan/24/science-fictio... 24 points by Freak_NL, 6 comments https://news.ycombinator.com/item?id=39119799 https://news.ycombinator.com/item?id=39119799
deleted 3y ago
[deleted]
gentleman11 3y ago
There must be some context I'm missing, because the article isn't clear at all about what's going on. It's addressed to people who are insiders I think? This reddit post provides a lot of the missing context, specifically the top comment https://www.reddit.com/r/books/comments/19cvlt3/big_controversy_brewing_over_the_2023_hugo_awards/ https://www.reddit.com/r/books/comments/19cvlt3/big_controve...
- dsr_ 3y ago
  The Hugo Awards are for SFF, and are voted on by members of this year's WorldCon. After an epidemic of slate voting a while back, a ranked choice system called E Pluribus Hugo was installed, and has been reasonably well-accepted. The 2023 WorldCon was held in China, for the first time. (It's traditionally outside North America every other year.) There were lots of Chinese-language works on the ballot, which was to be expected -- lots more Chinese-literate people attending. The rules say that the statistics must be released within 90 days of the convention. Usually they are released the next day. This time, it went all the way to the limit, and arguably beyond. The statistics, at a first glance, look like they have been coerced. Several works were adjudicated to be ineligible. Some of them were plausibly inelegible, and a reason was supplied for each. (First published in a prior year, for example.) Some of them were obviously eligible, and no reason was supplied. E.g. R.F. Kuang's novel _Babel_. The Netflix series The Sandman was ruled ineligible, both in the long form (whole series) and short form (episode 6). The rules say that the admin can choose which category to put a work in if it looks like there's a lot of support for both -- but here the admins decided to drop it from both. Paul Weimer was dropped from the best fan writer award. No explanation. Finally, only one member of the admin staff has answered questions at all, and he has been repeating "We acted in accordance with the constitution and the rules we had to obey" over and over again, without further explanation. So.
  - slaymaker1907 3y ago
    I find it really ironic that they disqualified Babel considering that it was pretty anti-imperialistic and pro-China (though pre-CCP). They just hate Kuang because her grandfather fought against the CCP.
JumpCrisscross 3y ago
The meat: “The seemingly random ineligiblity is believed to be due to the affected nominees being considered politically undesirable in China, especially since two of the affected nominees, R.F. Kuang and Xiran Jay Zhao, are American and Canadian Chinese respectively. However, nothing I have read by R.F. Kuang suggests that she would be overly likely to criticise the Chinese government. … However, if there was political influence on the Hugo ballot (and note that this is a big ‘if’ at this moment) and if Dave McCarty or Ben Yalow allowed this to happen, then fuck them and fuck Chengdu! I don’t blame any of the Chinese organisers for going along with possible political pressure, because they have to live in China and face genuine risks. But McCarty, Yalow or any other western SMOFs involved should have sounded the alarm or at the very least resigned. The Chinese government has no power over them. So shame on them, if they went along with this. Also shame on them, if they thought that burying the data would mean people wouldn’t notice the problem. Note that this is all just theory and conjecture at this point. We don’t know for sure what happened. And this is why we need answers and an explanation and we need it now. The people who were randomly declared ineligible needs answers and the finalists who made the ballot or won also need answers, because this is tainting their nominations and wins as well, if they don’t know if they really made the ballot or won organically.” Hugo (technically, the WSFS) held its Worldcon in Chengdu, China. There is speculation and preliminary evidence the 2023 awards were rigged.
- cstross 3y ago
  you seem to think that Dave and Ben have any control whatsoever over the Chengdu worldcon. They don't. World science fiction conventions are autonomous and independent of one another, apart from (in theory) having to abide by the WSFS constitution. There is no permanent floating worldcon organization that oversees these events. It has worked up until now -- through 81 conventions in a variety of countries -- because everyone abided by an unspoken agreement to observe the rules. The Chengdu concom appeared to be following the rules right up until they didn't any more and went "rules? what rules?" Here's your brief intro to what worldcon is: https://en.wikipedia.org/wiki/Worldcon https://en.wikipedia.org/wiki/Worldcon (Disclaimer: I'm a three-times Hugo award winning author and the author with the most appearances on the Hugo shortlist from outside North America.)
lelanthran 3y ago
I think it's common knowledge by now that the Hugo awards go to whatever is most politically expedient. IOW, there's no surprises here.
- kevinventullo 3y ago
  For what it’s worth, I did not know that, and I find it disappointing. Is there evidence that previous nomination processes were rigged in a way similar to what is suggested here?
  - deleted 3y ago
    [deleted]
  - aspenmayer 3y ago
    I think this is exactly what you’re looking for. I’m reading through it now. Please reply with your findings if you have the time and inclination. https://alpennia.com/blog/comparison-hugo-nomination-distribution-statistics https://alpennia.com/blog/comparison-hugo-nomination-distrib... More info at my original thread: https://news.ycombinator.com/item?id=39132517 https://news.ycombinator.com/item?id=39132517
  - cstross 3y ago
    It's bullshit. There was an extensive campaign in favour of a particular faction circa 2013-17 organized by two loose groups (the Sad Puppies and the Rabid Puppies) which in retrospect were best understood as dry runs for Gamergate, but they flopped (all they could do was rig the shortlist: but the Hugo voters are allowed to vote "no award" if they don't like the menu, and they did so with gusto until the rules could be reformed to make nomination-brigading impractical). https://en.wikipedia.org/wiki/Sad_Puppies https://en.wikipedia.org/wiki/Sad_Puppies
    - bigbillheck 3y ago
      > which in retrospect were best understood as dry runs for Gamergate Gamergate kicked off between the 2nd and 3rd Sad Puppy slates and before the 1st Rabid Puppy.
bondarchuk 3y ago
What exactly is "slate voting" or "slating" in this context? I can find this: https://en.wikipedia.org/wiki/Slate_(elections) https://en.wikipedia.org/wiki/Slate_(elections) but it does not seem to apply directly.
- angry_moose 3y ago
  It grew out of the Sad Puppy movement a few years ago (https://en.wikipedia.org/wiki/Sad_Puppies#2015_campaign https://en.wikipedia.org/wiki/Sad_Puppies#2015_campaign, linked to the year that was most organized and has the best description, but the whole article is worth a read) - a right wing anti-diversity voting campaign the Hugos dealt with from 2013-2017. Slating was a group of voters organizing to all vote for the same group of works (or slates).
- twic 3y ago
  Where a group of people all nominate the same set of candidates. The Hugo system [1] starts with nominations - every voter can nominate up to five works in each category. The six works which get nominated the most (in some sense) become the shortlist. There is then a ranked ballot vote amongst the shortlist in each category. Until 2017, nominations were evaluated by counting the number of nomination ballots that each work appeared on. A hundred people nominate a work, it scores a hundred. The thing is, there are a lot of books published every year, so there are a lot of candidates, and so nominations are quite spread out. In 2015 [2], to get to the top of the shortlist for best novel, a book needed ~400 nominations, from ~1800 ballots; for best short story, ~300 nominations, from ~1200 ballots. So if you can get together a group of 400 voters, and they all agree to nominate the same five works in each category (a slate), then you can get all of those five works onto the shortlist. 400 is not a small number, but it's no more than a third of all voters, and that third can take over the shortlist, and dominate the contest (although their nominations are quite unlikely to actually win - but it is still very annoying!). In 2017, the system was changed, to do some sort of more proportional evaluation [3] that should reduce the power of slates. Consensus seems to be that it worked. [1] https://www.thehugoawards.org/the-voting-system/ https://www.thehugoawards.org/the-voting-system/ [2] https://chaoshorizon.files.wordpress.com/2015/08/2015hugostatistics.pdf https://chaoshorizon.files.wordpress.com/2015/08/2015hugosta... [3] https://electowiki.org/wiki/E_Pluribus_Hugo https://electowiki.org/wiki/E_Pluribus_Hugo
asplake 3y ago
John Scalzi on this a few days ago: https://whatever.scalzi.com/2024/01/21/whats-up-with-babel-and-the-hugos/ https://whatever.scalzi.com/2024/01/21/whats-up-with-babel-a...

80 points by aspenmayer 3y ago

Daniel Greene made a great video explaining this: https://youtube.com/watch?v=85CFDXrORdw https://youtube.com/watch?v=85CFDXrORdw

low_tech_love 3y ago

I know barely nothing about the Hugo process or Sci-Fi awards in general, but unfortunately this rings a bell when I think of my experience as an academic researcher (i.e. "paper writer") regarding some of my Chinese colleagues. No matter how much we want to believe that science (and especially scientific reporting) is objective, the truth is that there are certain unwritten, informal, and subjective rules in the world of academic writing / research reporting that one is simply expected to follow; it is impossible for every reviewer to check every single detail about every paper they accept to review. There are ethical/common sense aspects that are just supposed to be there: the data you report is supposed to be real; the method you report is supposed to have been conducted accurately; etc. However, when it comes to a large part of Chinese-authored publications, one cannot help unfortunately but to have sometimes a kind of a ear-scratching sensation that one is not 100% sure these aspects are really there. First, because we're absolutely flooded daily by Chinese-authored drafts, and I'm not talking about simple, small-scale research results; these are big sprawling research results, involving large datasets and lots of subjects, all coming in at a staggering rate, non-stop. We don't have enough volunteers to review these papers accurately and carefully, especially given that modern open-access journals want to have reviews (100% volunteered, without any pay, mind you) sent back in ridiculous times, like one week. And yes, many of these journals are either Chinese or managed by Chinese colleagues. So in summary we simply cannot manage to review them in time with the care that is necessary, which means that some undesirable work will invariably pass the check (statistically speaking, it will happen). One cannot also help but think that it seems unrealistic that all these research projects are being carried at such incredible pace and with so much efficiency. One thing that compounds this problem is that papers come initially with a very low quality. Slowly they become better; you may find yourself reviewing the same paper 5 or 6 times until you get so sick of it that you have no other choice but to say, "I have nothing else to add, just publish the goddamn thing." This is not how the system was designed to work; usually, the authors (and the PhD supervisor) are supposed to work on the paper a bit before submitting, so that they don't submit a horrible thing for reviewers to work with. I think this works usually on the basis of reputation and "name", but my impression is that the Chinese authors could not care less about this. As I usually say to my local colleagues, we are supervising the Chinese PhD students (via paper reviews) without getting paid for it. Some of the most incredible researchers and smartest people in the world are certainly Chinese and some of their research reports are just amazing. But the numbers are so staggering that invariably the classic system ends up breaking. And one cannot help but think that many of them have found optimized ways to take advantage of the system in exactly those parts were one cannot really point the finger at, because they are based on informal and subjective common-sense rules that are just expected to be there (or else the system does not work).

aspenmayer 3y ago

Original title edited for length: The 2023 Hugo Nomination Statistics Have Finally Been Released – And We Have Questions This post was brought to my attention via Mastodon via HN user cstross, who posted an entire thread about this here, the initial post of which I quote: https://wandering.shop/@cstross/111793448361231714 https://wandering.shop/@cstross/111793448361231714 > Chengdu worldcon Hugo nomination statistics suggest the 2023 Hugo awards were rigged: > [link to OP] > I've also seen it alleged that the concom offered some attendees "special guest" treatment … and stole their Hugo voting rights in return, without telling them about that part of the "deal" first. (I will not cite a source on this until the attendee in question goes public.) [Above repost, including my original comment, upon request via second chance pool. More info about that here: https://news.ycombinator.com/item?id=26998308 https://news.ycombinator.com/item?id=26998308]

aspenmayer 3y ago
Even more context via the Mastodon thread: A Comparison of Hugo Nomination Distribution Statistics https://alpennia.com/blog/comparison-hugo-nomination-distribution-statistics https://alpennia.com/blog/comparison-hugo-nomination-distrib... Edit: Even more context from the above link: https://mrphilipslibrary.wordpress.com/2024/01/21/hugo-nominating-stats-rascality-and-a-brief-history-of-where-it-all-started/ https://mrphilipslibrary.wordpress.com/2024/01/21/hugo-nomin... which has a meta-follow-up at: https://mrphilipslibrary.wordpress.com/2024/01/23/the-mess-dave-mccarty-left-behind/ https://mrphilipslibrary.wordpress.com/2024/01/23/the-mess-d...
- aspenmayer 3y ago
  Related post on HN yesterday, from The Guardian: Science fiction awards held in China under fire for excluding authors https://www.theguardian.com/books/2024/jan/24/science-fiction-awards-held-in-china-under-fire-for-excluding-authors https://www.theguardian.com/books/2024/jan/24/science-fictio... 24 points by Freak_NL, 6 comments https://news.ycombinator.com/item?id=39119799 https://news.ycombinator.com/item?id=39119799

deleted 3y ago

[deleted]

gentleman11 3y ago

There must be some context I'm missing, because the article isn't clear at all about what's going on. It's addressed to people who are insiders I think? This reddit post provides a lot of the missing context, specifically the top comment https://www.reddit.com/r/books/comments/19cvlt3/big_controversy_brewing_over_the_2023_hugo_awards/ https://www.reddit.com/r/books/comments/19cvlt3/big_controve...

dsr_ 3y ago
The Hugo Awards are for SFF, and are voted on by members of this year's WorldCon. After an epidemic of slate voting a while back, a ranked choice system called E Pluribus Hugo was installed, and has been reasonably well-accepted. The 2023 WorldCon was held in China, for the first time. (It's traditionally outside North America every other year.) There were lots of Chinese-language works on the ballot, which was to be expected -- lots more Chinese-literate people attending. The rules say that the statistics must be released within 90 days of the convention. Usually they are released the next day. This time, it went all the way to the limit, and arguably beyond. The statistics, at a first glance, look like they have been coerced. Several works were adjudicated to be ineligible. Some of them were plausibly inelegible, and a reason was supplied for each. (First published in a prior year, for example.) Some of them were obviously eligible, and no reason was supplied. E.g. R.F. Kuang's novel _Babel_. The Netflix series The Sandman was ruled ineligible, both in the long form (whole series) and short form (episode 6). The rules say that the admin can choose which category to put a work in if it looks like there's a lot of support for both -- but here the admins decided to drop it from both. Paul Weimer was dropped from the best fan writer award. No explanation. Finally, only one member of the admin staff has answered questions at all, and he has been repeating "We acted in accordance with the constitution and the rules we had to obey" over and over again, without further explanation. So.
- slaymaker1907 3y ago
  I find it really ironic that they disqualified Babel considering that it was pretty anti-imperialistic and pro-China (though pre-CCP). They just hate Kuang because her grandfather fought against the CCP.

JumpCrisscross 3y ago

The meat: “The seemingly random ineligiblity is believed to be due to the affected nominees being considered politically undesirable in China, especially since two of the affected nominees, R.F. Kuang and Xiran Jay Zhao, are American and Canadian Chinese respectively. However, nothing I have read by R.F. Kuang suggests that she would be overly likely to criticise the Chinese government. … However, if there was political influence on the Hugo ballot (and note that this is a big ‘if’ at this moment) and if Dave McCarty or Ben Yalow allowed this to happen, then fuck them and fuck Chengdu! I don’t blame any of the Chinese organisers for going along with possible political pressure, because they have to live in China and face genuine risks. But McCarty, Yalow or any other western SMOFs involved should have sounded the alarm or at the very least resigned. The Chinese government has no power over them. So shame on them, if they went along with this. Also shame on them, if they thought that burying the data would mean people wouldn’t notice the problem. Note that this is all just theory and conjecture at this point. We don’t know for sure what happened. And this is why we need answers and an explanation and we need it now. The people who were randomly declared ineligible needs answers and the finalists who made the ballot or won also need answers, because this is tainting their nominations and wins as well, if they don’t know if they really made the ballot or won organically.” Hugo (technically, the WSFS) held its Worldcon in Chengdu, China. There is speculation and preliminary evidence the 2023 awards were rigged.

cstross 3y ago
you seem to think that Dave and Ben have any control whatsoever over the Chengdu worldcon. They don't. World science fiction conventions are autonomous and independent of one another, apart from (in theory) having to abide by the WSFS constitution. There is no permanent floating worldcon organization that oversees these events. It has worked up until now -- through 81 conventions in a variety of countries -- because everyone abided by an unspoken agreement to observe the rules. The Chengdu concom appeared to be following the rules right up until they didn't any more and went "rules? what rules?" Here's your brief intro to what worldcon is: https://en.wikipedia.org/wiki/Worldcon https://en.wikipedia.org/wiki/Worldcon (Disclaimer: I'm a three-times Hugo award winning author and the author with the most appearances on the Hugo shortlist from outside North America.)

lelanthran 3y ago

I think it's common knowledge by now that the Hugo awards go to whatever is most politically expedient. IOW, there's no surprises here.

kevinventullo 3y ago
For what it’s worth, I did not know that, and I find it disappointing. Is there evidence that previous nomination processes were rigged in a way similar to what is suggested here?
- deleted 3y ago
  [deleted]
- aspenmayer 3y ago
  I think this is exactly what you’re looking for. I’m reading through it now. Please reply with your findings if you have the time and inclination. https://alpennia.com/blog/comparison-hugo-nomination-distribution-statistics https://alpennia.com/blog/comparison-hugo-nomination-distrib... More info at my original thread: https://news.ycombinator.com/item?id=39132517 https://news.ycombinator.com/item?id=39132517
- cstross 3y ago
  It's bullshit. There was an extensive campaign in favour of a particular faction circa 2013-17 organized by two loose groups (the Sad Puppies and the Rabid Puppies) which in retrospect were best understood as dry runs for Gamergate, but they flopped (all they could do was rig the shortlist: but the Hugo voters are allowed to vote "no award" if they don't like the menu, and they did so with gusto until the rules could be reformed to make nomination-brigading impractical). https://en.wikipedia.org/wiki/Sad_Puppies https://en.wikipedia.org/wiki/Sad_Puppies
  - bigbillheck 3y ago
    > which in retrospect were best understood as dry runs for Gamergate Gamergate kicked off between the 2nd and 3rd Sad Puppy slates and before the 1st Rabid Puppy.

bondarchuk 3y ago

What exactly is "slate voting" or "slating" in this context? I can find this: https://en.wikipedia.org/wiki/Slate_(elections) https://en.wikipedia.org/wiki/Slate_(elections) but it does not seem to apply directly.

angry_moose 3y ago
It grew out of the Sad Puppy movement a few years ago (https://en.wikipedia.org/wiki/Sad_Puppies#2015_campaign https://en.wikipedia.org/wiki/Sad_Puppies#2015_campaign, linked to the year that was most organized and has the best description, but the whole article is worth a read) - a right wing anti-diversity voting campaign the Hugos dealt with from 2013-2017. Slating was a group of voters organizing to all vote for the same group of works (or slates).
twic 3y ago
Where a group of people all nominate the same set of candidates. The Hugo system [1] starts with nominations - every voter can nominate up to five works in each category. The six works which get nominated the most (in some sense) become the shortlist. There is then a ranked ballot vote amongst the shortlist in each category. Until 2017, nominations were evaluated by counting the number of nomination ballots that each work appeared on. A hundred people nominate a work, it scores a hundred. The thing is, there are a lot of books published every year, so there are a lot of candidates, and so nominations are quite spread out. In 2015 [2], to get to the top of the shortlist for best novel, a book needed ~400 nominations, from ~1800 ballots; for best short story, ~300 nominations, from ~1200 ballots. So if you can get together a group of 400 voters, and they all agree to nominate the same five works in each category (a slate), then you can get all of those five works onto the shortlist. 400 is not a small number, but it's no more than a third of all voters, and that third can take over the shortlist, and dominate the contest (although their nominations are quite unlikely to actually win - but it is still very annoying!). In 2017, the system was changed, to do some sort of more proportional evaluation [3] that should reduce the power of slates. Consensus seems to be that it worked. [1] https://www.thehugoawards.org/the-voting-system/ https://www.thehugoawards.org/the-voting-system/ [2] https://chaoshorizon.files.wordpress.com/2015/08/2015hugostatistics.pdf https://chaoshorizon.files.wordpress.com/2015/08/2015hugosta... [3] https://electowiki.org/wiki/E_Pluribus_Hugo https://electowiki.org/wiki/E_Pluribus_Hugo

asplake 3y ago

John Scalzi on this a few days ago: https://whatever.scalzi.com/2024/01/21/whats-up-with-babel-and-the-hugos/ https://whatever.scalzi.com/2024/01/21/whats-up-with-babel-a...