4 ms·

Currently I am doing 6 requests a minute

by janmo 5y ago

mistermann 5y ago
I wonder, might you consider implementing something like what's asked for here? https://news.ycombinator.com/item?id=27093483 https://news.ycombinator.com/item?id=27093483 > Does it track things that were on the front page but were flagged? I've been looking for that. ...and... > I sort of think there should be an 'important flagged discussions' link, ones with otherwise lots of engagement that got flagged by people who were opposed to the viewpoint of the post. An example of such a post: The not-so-drastic truth behind climate change https://news.ycombinator.com/item?id=27097434 https://news.ycombinator.com/item?id=27097434 This article was on my frontpage, I read the 10 or so comments that were there and then forwarded it to myself as I love reading discussions about culture war topics, but when I refreshed HN immediately after reading the comments, I discovered that not only was it no longer on the front page, but it couldn't be found on any of the 17 pages provided when hitting next and searching for it. This is far from the first time I've noticed this happening with articles of this nature, and I think it would be very interesting to be able to see what ideas the HN moderators don't want the minds of their users exposed to. I think there are also several specific (and influential) subreddits that would also yield some very interesting results. Of course, you run the risk of getting your scraper blocked which would ruin your entire project...but on the other hand, if you were able to find a technical way around such blocking (I assume there are services that allow one to scrape using a large pool of ip's?) I think you could make this project significantly more interesting than it already is. Being able to see the things you are already showing is mildly interesting, but being able to see a list of particular topics that the powerful gatekeepers of information (in a very real sense, demi-gods who exert significant control over the publics' very perception of reality) do not want people to see is far more interesting, and in turn may make your service far more popular than it would be in its current form. EDIT: And it keeps getting more interesting...now the parent comment to the one I made[1] in that thread is [deleted] - not sure if that's a user invoked delete or moderator - I assume the former, as if it was a moderator I assume mine would have been toasted also. https://news.ycombinator.com/item?id=27098186 https://news.ycombinator.com/item?id=27098186 >> Even re-production and communication should be discussed. > I think the most interesting and valuable/useful perspective to discuss this topic from (aside from "the science") is from a human psychology (which is heavily dependent on communication) perspective. Ultimately, actually doing something about climate change (regardless of what the consensus decision of that is) requires political support, and from what we've seen on that front, that seems to be the weakest link within the entire multidimensional problem. > Speaking of human psychology (and communication, and perception of reality), note how quickly moderators were able to memory hole this post - it was on my front page 10 minutes ago, and now it can no longer be found within the first 17 pages of HN. I've seen this phenomenon occur on several occasions over the years, and I don't even check HN all that often, and in each case (of the ones I've noticed) it tends to be particularly controversial topics like this one, where the perspective of the article is "controversial". Human beings (and all the clever mischievousness they get up to) are completely fascinating creatures, I can't understand why so few people find them interesting. EDIT 2: And just now does the post have a [flagged] tag, wonder why it disappeared before even being flagged.
- walterbell 5y ago
  There is good research to be done throughout social media on flag/dupe/downvote trends and countermeasures, whether inspired by the zeitgeist, unpaid advocates for a viewpoint, paid reputation management firms and their clients, or social-bot experiments on influence.
  - mistermann 5y ago
    Agreed...going even further, I think ~collective observation, analysis, and self/meta-reflection of the internet hivemind/zeitgeist/simulation/matrix could be a huge new internet platform space...and not just popular and fun/interesting, but I truly believe that the near total absence of skilled self-reflection is just what the doctor ordered for the period of social and political polarization that we find ourselves in. I think once people "got" the idea, it would catch on like wildfire (I also think that if a nefarious actor got there first (Facebook, Google, etc), they could possibly capture and nullify the space). I know of absolutely zero startups that are active in this space, about the closet thing I've come across are platforms like https://www.kialo.com/ https://www.kialo.com/ that are trying to improve the quality and structure of conversation. Do you know of anything along the lines of what I'm describing here (assuming it makes sense what I'm describing)?
    - walterbell 5y ago
      > Do you know of anything along the lines of what I'm describing here I'm only aware of organizations that have a business incentive to influence at-scale sensemaking or decisions, so their analysis is private, and their public participation is often unattributable and oriented towards a specific side/outcome. Examples could include Cambridge Analytica, public-private partnerships on semi-automated data analytics for psychological warfare, influence operations and persona management, and PR firms (http://www.techsoc.com/grassroots.html http://www.techsoc.com/grassroots.html). As for consumer-accessible data visualization and models for interactions within specific communities, it's the eternal challenge of gamified semantics in any sufficiently-contested online space, with Google search rankings/algo/gaming being one of the most visible battlegrounds. Every meta-observation and subsequent algo/ranking update leads to new outcome (e.g. profit) variability, which changes the live experiment and starts a new iteration of the ongoing game. The most successful historical examples have been some decentralized combination of human moderation, participant voting and usually-private analytics tools available to moderators. Over time, we've seen the most successful communities move towards increased transparency of manual moderation. Independent, adversarial analytics (e.g. hnrankings or this tool) provide valuable feedback to both users and moderators. Kialo looks fun, thanks for the pointer. Do you know their business model? Here's a recent paper on community governance automation: https://www.csee.umbc.edu/courses/graduate/691/spring21/pdf/2008.04236.pdf https://www.csee.umbc.edu/courses/graduate/691/spring21/pdf/... > When online communities desire other forms of government, such as ones that take many members’ opinions into account or that distribute power in non-trivial ways, communities must resort to laborious manual effort. In this paper, we present PolicyKit, a software infrastructure that empowers online community members to concisely author a wide range of governance procedures and automatically carry out those procedures on their home platforms ... We demonstrate the expressivity of PolicyKit through implementations of governance models such as a random jury deliberation, a multi-stage caucus, a reputation system, and a promotion procedure inspired by Wikipedia’s Request for Adminship (RfA) process.

Currently I am doing 6 requests a minute

by janmo 5y ago

Currently I am doing 6 requests a minute

I wonder, might you consider implementing something like what's asked for here? https://news.ycombinator.com/item?id=27093483 https://news.ycombinator.com/item?id=27093483 > Does it track things that were on the front page but were flagged? I've been looking for that. ...and... > I sort of think there should be an 'important flagged discussions' link, ones with otherwise lots of engagement that got flagged by people who were opposed to the viewpoint of the post. An example of such a post: The not-so-drastic truth behind climate change https://news.ycombinator.com/item?id=27097434 https://news.ycombinator.com/item?id=27097434 This article was on my frontpage, I read the 10 or so comments that were there and then forwarded it to myself as I love reading discussions about culture war topics, but when I refreshed HN immediately after reading the comments, I discovered that not only was it no longer on the front page, but it couldn't be found on any of the 17 pages provided when hitting next and searching for it. This is far from the first time I've noticed this happening with articles of this nature, and I think it would be very interesting to be able to see what ideas the HN moderators don't want the minds of their users exposed to. I think there are also several specific (and influential) subreddits that would also yield some very interesting results. Of course, you run the risk of getting your scraper blocked which would ruin your entire project...but on the other hand, if you were able to find a technical way around such blocking (I assume there are services that allow one to scrape using a large pool of ip's?) I think you could make this project significantly more interesting than it already is. Being able to see the things you are already showing is mildly interesting, but being able to see a list of particular topics that the powerful gatekeepers of information (in a very real sense, demi-gods who exert significant control over the publics' very perception of reality) do not want people to see is far more interesting, and in turn may make your service far more popular than it would be in its current form. EDIT: And it keeps getting more interesting...now the parent comment to the one I made[1] in that thread is [deleted] - not sure if that's a user invoked delete or moderator - I assume the former, as if it was a moderator I assume mine would have been toasted also. https://news.ycombinator.com/item?id=27098186 https://news.ycombinator.com/item?id=27098186 >> Even re-production and communication should be discussed. > I think the most interesting and valuable/useful perspective to discuss this topic from (aside from "the science") is from a human psychology (which is heavily dependent on communication) perspective. Ultimately, actually doing something about climate change (regardless of what the consensus decision of that is) requires political support, and from what we've seen on that front, that seems to be the weakest link within the entire multidimensional problem. > Speaking of human psychology (and communication, and perception of reality), note how quickly moderators were able to memory hole this post - it was on my front page 10 minutes ago, and now it can no longer be found within the first 17 pages of HN. I've seen this phenomenon occur on several occasions over the years, and I don't even check HN all that often, and in each case (of the ones I've noticed) it tends to be particularly controversial topics like this one, where the perspective of the article is "controversial". Human beings (and all the clever mischievousness they get up to) are completely fascinating creatures, I can't understand why so few people find them interesting. EDIT 2: And just now does the post have a [flagged] tag, wonder why it disappeared before even being flagged.

walterbell 5y ago
There is good research to be done throughout social media on flag/dupe/downvote trends and countermeasures, whether inspired by the zeitgeist, unpaid advocates for a viewpoint, paid reputation management firms and their clients, or social-bot experiments on influence.
- mistermann 5y ago
  Agreed...going even further, I think ~collective observation, analysis, and self/meta-reflection of the internet hivemind/zeitgeist/simulation/matrix could be a huge new internet platform space...and not just popular and fun/interesting, but I truly believe that the near total absence of skilled self-reflection is just what the doctor ordered for the period of social and political polarization that we find ourselves in. I think once people "got" the idea, it would catch on like wildfire (I also think that if a nefarious actor got there first (Facebook, Google, etc), they could possibly capture and nullify the space). I know of absolutely zero startups that are active in this space, about the closet thing I've come across are platforms like https://www.kialo.com/ https://www.kialo.com/ that are trying to improve the quality and structure of conversation. Do you know of anything along the lines of what I'm describing here (assuming it makes sense what I'm describing)?
  - walterbell 5y ago
    > Do you know of anything along the lines of what I'm describing here I'm only aware of organizations that have a business incentive to influence at-scale sensemaking or decisions, so their analysis is private, and their public participation is often unattributable and oriented towards a specific side/outcome. Examples could include Cambridge Analytica, public-private partnerships on semi-automated data analytics for psychological warfare, influence operations and persona management, and PR firms (http://www.techsoc.com/grassroots.html http://www.techsoc.com/grassroots.html). As for consumer-accessible data visualization and models for interactions within specific communities, it's the eternal challenge of gamified semantics in any sufficiently-contested online space, with Google search rankings/algo/gaming being one of the most visible battlegrounds. Every meta-observation and subsequent algo/ranking update leads to new outcome (e.g. profit) variability, which changes the live experiment and starts a new iteration of the ongoing game. The most successful historical examples have been some decentralized combination of human moderation, participant voting and usually-private analytics tools available to moderators. Over time, we've seen the most successful communities move towards increased transparency of manual moderation. Independent, adversarial analytics (e.g. hnrankings or this tool) provide valuable feedback to both users and moderators. Kialo looks fun, thanks for the pointer. Do you know their business model? Here's a recent paper on community governance automation: https://www.csee.umbc.edu/courses/graduate/691/spring21/pdf/2008.04236.pdf https://www.csee.umbc.edu/courses/graduate/691/spring21/pdf/... > When online communities desire other forms of government, such as ones that take many members’ opinions into account or that distribute power in non-trivial ways, communities must resort to laborious manual effort. In this paper, we present PolicyKit, a software infrastructure that empowers online community members to concisely author a wide range of governance procedures and automatically carry out those procedures on their home platforms ... We demonstrate the expressivity of PolicyKit through implementations of governance models such as a random jury deliberation, a multi-stage caucus, a reputation system, and a promotion procedure inspired by Wikipedia’s Request for Adminship (RfA) process.