10 ms·
A Graph of Related Subreddits
- rewq4321 7y agoSee also: https://subredditstats.com/subreddit-user-overlaps/programming https://subredditstats.com/subreddit-user-overlaps/programmi...
- Glosster 7y agoThis one seems to be doing a worse job. Compare what I'm getting for /r/longevity: https://subredditstats.com/subreddit-user-overlaps/longevity https://subredditstats.com/subreddit-user-overlaps/longevity vs https://anvaka.github.io/sayit/?query=longevity https://anvaka.github.io/sayit/?query=longevity
- anvaka 7y agoOh hey there, I'm the author of this project. Just wanted to say thank you for sharing this! Would be happy to answer any questions - graphs are my long time hobby, and I love them! PS: You can find more recent graphs and fun projects here: https://twitter.com/search?q=from%3Aanvaka%20min_retweets%3A20&src=typed_query https://twitter.com/search?q=from%3Aanvaka%20min_retweets%3A...
- bobosha 7y agoHow does this infer subreddit similarity? For instance, I checked for r/AskHistorians and the results don't seem that relevant. edit: never mind, I just read your GitHub readme. But the question still stands as if "users posting in x, also posted in y" is a good way to infer similarity. Could comparing top-ranked posts be a better comparator?
- I_am_tiberius 7y agoI guess the amazon principle: Users who posted here, also posted there?
- catach 7y agoThe method seems surprisingly effective for a lot of subs, but also bothers me.
- anvaka 7y agoI used my own metric that is based on jaccard similarity. Which in turn is based on "users who posted to X also posted to Y" metric. That said, there are a few subreddits that are too popular and similarity results were too saturated (/r/videos, /r/funny, etc.) so I did a manual override by looking into most commonly mentioned other subreddits, and sometimes into `about` blurb of subreddit). Please don't consider these recommendation as source of truth! It's just a fun way to discover other subreddits :). I'm also very open to change this metric to something else - please let me know if you have any recommendations! [1]: https://github.com/anvaka/sayit#the-data https://github.com/anvaka/sayit#the-data - describes the data, indexing scripts are here: https://github.com/anvaka/sayit/tree/master/scripts https://github.com/anvaka/sayit/tree/master/scripts [2]: Manual overrides can be found here https://github.com/anvaka/sayit-data#sayit---recommendation-data https://github.com/anvaka/sayit-data#sayit---recommendation-...
- ketralnis 7y agoI work for reddit and here I’ve used a similar technique to your jaccard distance but with one twist: divide by the size of the smaller subreddit (in your case, the number of unique posters that you’ve recorded). That gives you a directional relatedness, that is programming->python but not necessarily python->programming. Used this way you account for the giant subreddit problem automatically but now the results are less “amitheasshole is related to askreddit” and more like “linguisticshumor is a more niche version of linguistics”. The great thing is that it’s actually more actionable as far as recommendations go! Everybody has already heard of the bigger version of this subreddit, but they probably haven’t heard of the smaller versions. And it’s self-correcting. As a subreddit gets bigger we are less likely to recommend it (which is great because it needs our help less)
- jcims 7y agoCan people still make their upvotes/downvotes public? I found that useful for sussing out related subreddits when it was possible back in the day.
- anvaka 7y agoThis is super awesome, thank you for sharing! If you guys are interested in seeing how your recommendation work for the entire reddit, I'd be happy to build you a spaceship similar to this one https://github.com/anvaka/word2vec-graph https://github.com/anvaka/word2vec-graph . I couldn't find an easy way to download the entire recommendation graph, but it would be awesome if we could make it work. My email is the same as this account at gmail, and twitter is all open: https://twitter.com/anvaka https://twitter.com/anvaka
- varjag 7y agoHave to say the link between The_Donald and TwoXChromosomes was a surprising one.
- Mountain_Skies 7y agoProbably due to both subreddits going on trolling raids on each other.
- varjag 7y agoReddit tries to keep a lid on brigading, it's actively policed. Surprising to see this as a major component.
- MagnumOpus 7y agoReddit execs give lip service to fighting brigading. It is only enforced in rare cases.
- ptah 7y agoany information about coverage? r/electribe is not in auto complete and generates a graph with only itself
- stared 7y agoThank you for creating this project! I share your interest in creating graphs (vide some older project of mine, a map of Stack Exchange tags https://p.migdal.pl/tagoverflow/ https://p.migdal.pl/tagoverflow/). When I dived into various visualization of Subreddits (as a background reading for this side-project https://observablehq.com/@stared/tree-of-reddit-sex-life https://observablehq.com/@stared/tree-of-reddit-sex-life), your vis was the best for exploration (out of many, many). The only other one that I found and this level was a Sigma.js-based https://www.jacobsilterra.com/subreddit_map/network/ https://www.jacobsilterra.com/subreddit_map/network/ (beautiful, but more static).
- anvaka 7y agoYay! Thank you for sharing the links and thank you for your kind words! I see you are doing something with quantum tensors, which is very impressive! I'm still struggling with concept of regular tensors, and would love to have a good intuition/visualization for them - do you have any pointers?
- stared 7y agoIt depends on what you are looking for. Tensors as in physics (typically with each dimension being related to space(time)), tensors for quantum information & computing, tensors for deep learning? I've found it illuminating to see tensor diagrams. - https://medium.com/@pmigdal/in-the-topic-of-diagrams-i-did-write-a-review-simple-diagrams-of-convoluted-neural-networks-6418a63f9281 https://medium.com/@pmigdal/in-the-topic-of-diagrams-i-did-w... - https://www.math3ma.com/blog/matrices-as-tensor-network-diagrams https://www.math3ma.com/blog/matrices-as-tensor-network-diag... Also, right now, we are developing a matrix visualization in https://github.com/Quantum-Game https://github.com/Quantum-Game. However, it is pretty much work in progress; for a slightly more mature one, go to Quantum Game 2 website and in the element encyclopedia there is one. I am always up for talking about tensors, so feel invited to drop me an email.
- ddoeth 7y agoI just saw your city roads project, that is so awesome! Thanks for doing that
- flanbiscuit 7y agoI love your npm package dependency tree graph, I've actually posted links to it very recently on both HN and Reddit. Awesome tool! https://npm.anvaka.com/#/ https://npm.anvaka.com/#/
- anvaka 7y agoOh thank you! I'm very glad you liked it :)!
- throw_m239339 7y agoThank you, it's quite a useful app.
- hooande 7y agogreat UI for this. very novel and intuitive
- anvaka 7y agoThank you! I'm very happy to hear this!
- papln 7y agoSmall request: Can you turn off the initial jumble of nodes before the graph is built and laid out? It's distracting clutter that adds no information. Animations are fun, but only if they convey artistic or intellectual meaning. Also, the default zoom level is zoomed in too far (large text, not showing the wohle graph). perhaps this is because viewport resolution/size (phone vs desktop) is not taken into account when chooseing a zoom level (font size)
- wsinks 7y agoThis is so cool! You've basically made a different, better version of r/findasubreddit. Which led me to think... https://anvaka.github.io/sayit/?query=findasubreddit https://anvaka.github.io/sayit/?query=findasubreddit Super interesting! So many small and unknown subreddits, and one of the main connections is another subreddit called "Somebody Make This". I'll take a peek under the hood later, but this on the surface is very cool.
- anvaka 7y agoThank you :)! Under the hood it is all very naive counting of users who posted to A also posted to B.
- chx 7y agoThanks, this is very interesting. Mind if I ask how you determine relatedness? https://anvaka.github.io/sayit/?query=UsbCHardware https://anvaka.github.io/sayit/?query=UsbCHardware This subreddit is very small, I am a very active poster there and yet I recognize none of the related ones.
- anvaka 7y agoHi there. Here is a similar question and answer: https://news.ycombinator.com/item?id=22178373 https://news.ycombinator.com/item?id=22178373
- rewq4321 7y agoI think for smaller subreddit's there's not as much data, so results are more noise. Apparently Jaccard similarity was used, except for the really big subreddits: https://github.com/anvaka/sayit https://github.com/anvaka/sayit Unrelated, but had to say it somewhere in this discussion: Thanks @anvaka for your open source graph libs! Way more performant than anything else I've tried.
- DLA 7y agoanvaka: your work is just amazing! The viz you did of software repos is outstanding. I just wanted to express thanks to you for sharing as I am a big fan of all things graph as well.
- neiman 7y agoWhat a fantastic idea! I always keep on looking for new interesting subreddits. This tool is the best I saw for this task so far.
- anvaka 7y agoI'm so glad to know this! Thank you!
- mckirk 7y agoThe most interesting graph I could find so far: https://anvaka.github.io/sayit/?query=chairsunderwater https://anvaka.github.io/sayit/?query=chairsunderwater Speaking of which, it'd be awesome if the site could automatically generate multi-reddits from the results. I think a multi-reddit constructed from that graph would be quite interesting ;)
- semipro 7y agoThanks, I knew about r/chairsunderwater, but never heard about r/breadstapedtotrees. My life will never be the same.
- appleflaxen 7y agotry https://anvaka.github.io/sayit/?query=ocaml https://anvaka.github.io/sayit/?query=ocaml the algorith is non-deterministic, so if you follow the link more than once, you get a different spatial arrangement each time. some are very pleasing, with identifiable clusters, and others are just a seemingly random scatter of names. also interesting: https://anvaka.github.io/sayit/?query=lostredditors https://anvaka.github.io/sayit/?query=lostredditors (no lines are generated here; is this a feature or a bug?)
- sgentle 7y agoThis cluster is pretty spectacular: https://anvaka.github.io/sayit/?query=DarkFuturology https://anvaka.github.io/sayit/?query=DarkFuturology Edit: oh wow: https://anvaka.github.io/sayit/?query=Jung https://anvaka.github.io/sayit/?query=Jung
- shanth 7y agowould be nice to keep that renderer as separate library.
- orf 7y agoThe_donald members often post in TwoXChromosomes and TropicalWeather? https://anvaka.github.io/sayit/?query=the_donald https://anvaka.github.io/sayit/?query=the_donald
- blablablerg 7y agoRant: I am no Trump bigot, but it annoys me to no end that an feminist echochamber like TwoXChromosomes is on r/all. Any kind of criticism, for example when all men are blamed for not doing the dishes as if all men are dirty beasts: prepare to get burned down by horde of feminists. "This is a support group, a safe space". No it is a toxic space that I can't block in my Reddit feed (RES firefox addon doesn't work). If I can't block it, I should be allowed to give my opinion. I don't hate females, but goddamn do I hate that place.
- rkop 7y agoPrepare to be banned here. You can't say anything bad about anything even remotely related to women in this safe space powered by Silicon Valley™.
- fifnir 7y ago> I am no Trump bigot ....but let me regurgitate biggoted trumpist talking points which paint me as some sort of victim to the somehow oppressing women. RES on firefox works perfectly well for me, if twoXchromosome of all subs bothers you on /r/all just try harder to get it working.
- np_tedious 7y agoIs there some reason you can't / are reluctant to make an account? Then you can take this off your subscriptions and add non-default subs that interest you
- jccalhoun 7y ago> for example when all men are blamed for not doing the dishes as if all men are dirty beasts: prepare to get burned down by horde of feminists. citation needed
- morceauxdebois 7y agoSomeone should make a graph all the moderators letting subreddits turn to garbage bot farms
- mapleboi 7y agothis is so sick! finally an easy way to find some new subreddits. thanks!
- anvaka 7y agoYay! Thank you!
- dredmorbius 7y agohttps://anvaka.github.io/sayit/?query=hackernews https://anvaka.github.io/sayit/?query=hackernews
- petey283 7y agoGreat project.. and great implementation.
- anvaka 7y agoThank you!
- FredrikMeyer 7y agoEarlier discussion (june 2019) https://news.ycombinator.com/item?id=18866800 https://news.ycombinator.com/item?id=18866800
- itsmhuang 7y agoI'm not able to see the contents of a subreddit in the sidebar after clicking on it. I'm on MacOS Catalina using Google Chrome 79 (pretty modern).
- anvaka 7y agoI heard this might be caused by some adblocking extensions - they consider reddit to be an ad/tracking system, so they block all javascript requests to it. Do you happen to have one of those extensions?
- justaman 7y agoSearching for r/aww links to lots of porn.
- ProbablyRyaan 7y ago*immediately types in porn subreddit.
- rednerrus 7y agoThe link between r/Android and r/iamverysmart seems legit.
- soylentcola 7y agoLink also exists with r/apple and r/ios (but not r/windows or r/linux).
- iblaine 7y agoIf I pick a relatively obscure subreddit like dataengineering, then the results are noisy. May increase the distance/decrease the charge between nodes as the number of children increases on a node?
- GloriousKoji 7y agoBon Appetit is a New York city based food magazine that also has pretty entertaining YouTube content. The graph is extremely shocking and not at all what I would have expected. https://anvaka.github.io/sayit/?query=bon_appetit https://anvaka.github.io/sayit/?query=bon_appetit
- deleted 7y ago[deleted]