7 ms·
Hundreds of millions of stars turned into a map of GitHub projects
- Abimelex 3y agojust came here to give it my upvote. That's so insane cool!
- mxmilkiib 3y agoEasier than maintaining a wiki! https://wiki.thingsandstuff.org/Audio https://wiki.thingsandstuff.org/Audio etc.
- irrational 3y agoIs there a page that explains what the island names mean? I understand Minecraftria, but Fluttopia (and most others)? No clue.
- speedgoose 3y agoFrom the project readme on GitHub: > A lot of country labels were generated with help of ChatGPT. If you find something wrong, you can right click it, edit, and send a pull request - I'd be grateful.
- detaro 3y agoit usually becomes clear when zooming in and looking at the examples. E.g. "Fluttopia" is Flutter-related repos.
- anvaka 3y agoYes! Thank you Note that some country are missing their names because I didn't have enough knowledge to assign a name to them. Here is my naming process if anyone wants to help with the rest: https://github.com/anvaka/map-of-github#country-names https://github.com/anvaka/map-of-github#country-names
- swyx 3y agovery cool! how did you decide on the boundaries and placement of the continents?
- zX41ZdbW 3y agoI found many ClickHouse-related projects in a dedicated cluster - "House of Clicks" :) But ClickHouse itself somehow appears in Kubernation, and ClickBench - in Datapolis. Nice names btw.
- anvaka 3y agoThank you! It could be a curse or popularity? Some projects are so popular that you can place them nearly anywhere, and they would still find a lot of densely connected group of neighbors
- paulddraper 3y agoNice!! But how do I zoom in/out?
- bluefishinit 3y agoScroll wheel works for me.
- paulddraper 3y agoIDK how I feel about Kubernation: Firecracker, Grafana, Localstack, Podman, Terraform, Vault? Seems like a really overreaching power. Wait, never mind, that makes sense.
- anvaka 3y ago:D btw, if you find any country label to be missing or could be improved - right click it, change it, and then send a PR (it's two clicks away)
- swyx 3y agohow exactly did you name and cluster these things? i love how you presented it and dont know of any existing quantitative methodology
- anvaka 3y agoCheck this out https://github.com/anvaka/map-of-github#how-was-it-made https://github.com/anvaka/map-of-github#how-was-it-made - I gave a short overview there - does this help?
- divan 3y agoCheck out other anvaka's projects, there are gems: https://news.ycombinator.com/from?site=anvaka.github.io https://news.ycombinator.com/from?site=anvaka.github.io
- tomthe 3y agoOr check them out directly on the map: https://anvaka.github.io/map-of-github/#12.36/-31.74456/14.87526 https://anvaka.github.io/map-of-github/#12.36/-31.74456/14.8...
- bluefishinit 3y agoThis is really neat! Are there any insights you gleaned from doing this that were unexpected?
- pvaldes 3y agoSomebody has been doing a lot of Covid-19 plots in Gnu-R
- anvaka 3y agoI knew GitHub is not a tiny website, but I didn't imagine how big it actually is. Each of those dots are giant parts of someone's life. There are a lot of interests that I didn't know exist. For example https://github.com/cat-milk/Anime-Girls-Holding-Programming-Books https://github.com/cat-milk/Anime-Girls-Holding-Programming-... - someone collects anime girls holding programming books. https://github.com/tylertreat/Comcast https://github.com/tylertreat/Comcast - and here is someone who is amazing at coming up with funny project names =)
- tomthe 3y agoWhat else did you use to calculate similarities? When I tried something similar with HN co-commenters, I just got one big blob with dang and other prolific commenters in the middle. That was not very interesting or insightful. How long did it take to calculate the Leiden clustering and the force-layout? Do you think it would be somehow possible to compute a force-layout of the whole graph?
- edrxty 3y agoThis is amazing, the GitHub ham radio and hardware scenes are way bigger than I previously thought. Shout-out to OpenRTX and M17 along with all the awesome SDR applications!
- junon 3y agoThe data seems a bit dated, but this is really cool. I know this because I moved visionmedia/debug to debug-js/debug in November of 2021.
- anvaka 3y agoI've got data from 2020 to end of March 2023
- junon 3y agoAh okay, that makes more sense. Did you scrape it yourself? Incredible work by the way :)
- anvaka 3y agoThank you! Google BigQuery has it readily available, so probably took 10 minutes to fetch it =) https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code https://cloud.google.com/blog/topics/public-datasets/github-...
- zX41ZdbW 3y agoI recommend checking https://ghe.clickhouse.tech/ https://ghe.clickhouse.tech/ It explains the full pipeline - how to download, collect, and analyze this sort of data.
- rightbyte 3y agoThis is really neat. Clustering of data like this makes for so interesting graphics.
- OmarShehata 3y agoThis is amazing! I love this effect of showing me that my little project that I was working on fairly solitarily is actually part of a community of other people doing similar work that I can reach out to/collaborate with etc! What does it mean if, I click on a repo, and it shows 5-6 links to specific projects? Does that just mean the jaccard similarity index was below a threshold?
- anvaka 3y agoYes! I picked only the highest scores to form an edge in relationships graph. Typically a sigma (std deviation) or two away from the mean. So if there is a direct link between your project and others - the similarities are abnormally high. Note that I'm not rendering direct links outside of the country yet, there might be more there. Will probably add a "focused" view to see those better
- PedroBatista 3y agoAILandia was big before but now has been growing at an astounding rate almost surpassing Frontera. Some say it's a cancer others say it's inevitable and here to stay. Probably both are right. Also, why is Swiftoria so big?
- anvaka 3y agoI've been watching changelog nightly emails for a while - they summarize most starred daily repositories, and growth of AI there (subjectively) seems to be even higher than frontend tech
- mftb 3y agoTo me this is proof that doing a good visualization is really an art. I saw this on hn for a couple hours, but really didn't think much of it. As soon as I clicked through the funny names, and "continents" brought a smile to my face. I guess my little project is too small to make it, but now I too aspire to join the great nation of Golandia.
- binarysneaker 3y ago"Homelabia" made me smile like a schoolboy.
- anvaka 3y agoThank you!
- sdflhasjd 3y agoIf you type a capital letter into the search box, it gets stuck on "downloading index" indefinitely. I was waiting for about 10 minutes wondering how big these indexes were...
- pizza 3y agoDamn that's good
- matt_trentini 3y agoVery cool visualisation! Though I noticed a couple of odd groupings - like that MicroPython was clustered in Arduinoria rather than the adjacent MicroPythonia... :P
- jxf 3y agoI'm embarrassed to admit that I read "stars" in the title and spent far too long trying to figure out how these were arranged into constellations.
- __MatrixMan__ 3y agoThis is so fun, thanks for making it. I bet that several of these regions have a common image in their readme (the python logo, the nix logo, etc). Imagine little flags popping out of each region...
- anvaka 3y agohaha, I love the flags idea :). I wish I had the prowess to implement it in a way that is visually appealing and not obscuring the map. I would also love to have a giant octocat hugging the archipelago, with some radial gradient emitting inside of it. Alas my design-gl foo is not there yet
- internetguy 3y agoThis is frickin awesome. Made me really see how the programming community can come together and contribute to something powerful.
- data_ders 3y ago1. this is so cool. i'd love to see more name for the regions! 2. nit: one repo dear to my heart dbt-labs/dbt-core was previously dbt-labs/dbt as well as fishtownanalytics/dbt. they show up as unique nodes on your map. what most interesting to me is that each name of this repo links to a different set of related repos?
- anvaka 3y agoThank you! The connections are inferred from stargazers. If they lead to different set of related projects it might be a sign that different group of people gave stars different things at that time. Github of course has much more dimensions than a flat 2d surface can show
- danking00 3y agoThis reveals an interesting dichotomy in the project I work on, Hail. I think of hail as a serverless workflow, relational, and linear algebra system most similar to Dask, BigQuery, Snowflake, Spark, etc. but this map is constructed from the perspective of the user so Hail lands squarely in the world of bioinformatics. Also neat to see how bioinformatics is such a splitbrained community. They land next to R but are filled with Python projects.
- rmorey 3y agoThis is amazing
- sethkim 3y agoVery cool! If you work on a dynamic version that allows users to understand changes in the open-source topography over time and detect/predict new clusters, this could be a very powerful tool for investment intelligence.
- coffeeshopgoth 3y agoI love these things - to see everything sort of classified, and all at once, helps me find things I would never think to look for. Forgive me if this has been posted somewhere else, but if you want to see a reddit map, here it is (maybe you found it already in their github, but I just have it as a bookmark): https://anvaka.github.io/map-of-reddit/?x=18083.096950551575&y=19980.1478781304&z=19152.50551819642&v=2 https://anvaka.github.io/map-of-reddit/?x=18083.096950551575...
- topspin 3y agoIs Vue over represented in this? Seems like there would be a Reactistan island somewhere before Vue would show up.
- tomschlick 3y agoVue is a pretty big deal in the PHP/Laravel community. Evan the creator even gave a talk at Laracon a few months in, showing the hockey stick growth after it started to get traction in the community.
- anvaka 3y agoTrue. Though I also expected to "Reactistan" somewhere more explicitly, I wonder if it is scattered in Fronterra continent? I am also puzzled myself why the algorithm has separated Fronterra from another large island to the west, seemingly related to node and some other JS libraries. Decided to keep it all as is in case I'm missing some reason
- silverwind 3y agoMy feeling is that the React community is at least double to triple the size of the Vue one, so yes, I feel React should definitely be it's own island.
- shon 3y agoWow.. that’s cool
- metalliqaz 3y agoDid it break under the load? I can't search
- arthurcolle 3y agoReminds me of dotlan Eve maps
- phirschybar 3y agoincredible and so inspiring. thank you!!
- VitoVan 3y agoI'm quite unhappy with SBCL located in Schemaria: https://anvaka.github.io/map-of-github/#12/13.469/-8.175 https://anvaka.github.io/map-of-github/#12/13.469/-8.175 It should be Lispaña.
- anvaka 3y agoShould I rename Schemaria to Lispania?
- VitoVan 3y agoThe Scheme people would be unhappy then.
- anvaka 3y agoTrue... Scheming Lispania :)?
- andreareina 3y agoParentensia
- VitoVan 3y agoI like this one.
- ReleaseCandidat 3y agoNo, as there are all/most/some (Haskell, OCaml, Purescript, Idris, Coq, ...) functinal languages. Lambdania would be a possibility. Although Prolog and Forth are located there too.
- defanor 3y agoML-family languages are placed in Schemaria as well; apparently it is a catch-all for functional languages.
- paxo 3y agoHaha, very nice. I was able to locate multiple projects even ones with less than 1k stars easily.
- kelsolaar 3y agoThis is very cool and was pleasant albeit surprising to see colour in Gamedonia!
- speak 3y agoW'
- casey2 3y agoAmazing way to see related projects in an environment you aren't familiar with.
- geokon 3y agoThis gives a really cool top level view into the whole Chinese opensource community - which has always been very mysterious to me. Bc it's absolutely massive and almost entirely on its own doing its own thing due to the language barrier
- anvaka 3y agoIndeed! A lot of amazing things happens there. A few are getting world wide adoption (echarts are awesome if you are into frontend viz, so is element-plus library for vue components). There are giant communities of ML too
- _nalply 3y agoThe title is clickbaity in a very clever way, and I mean this as a compliment! I thought about what it says for almost a minute before clicking. Stars being made into Github projects? Would I find Orion and Alpha Centauri? I thought hard on about how to pull this off. Then I clicked and I realized that it is the opposite way, in reverse. And then this realization brought a smile on my face.
- anvaka 3y agoYou are very kind and your compliment put smile on my face too :)
- pmontra 3y agoThe Ruby world is part of an island labeled Art of node, west of Fronterra. Strange choice.
- anvaka 3y agoI agree. I'm not sure why they are together. Maybe all that tooling stuff that was happening in early days?
- samsquire 3y agoThis is absolutely awesome. Such a good job well done! The search is fast! The sidebar is a great feature. I also love how you highlight the repositories you search for and draw red lines between things. I found my journal repositories in the "Land of Node" in "Frontartia". I am surprised by that because I didn't realise I was associated with the node community! I am so impressed with your visualization, it is intuitive and interesting the different communities of GitHub.
- anvaka 3y agoThank you so much! I'm puzzled by some countries in Frontartia's island too. I'm not sure why it was even pulled away, as if there is something I'm missing.
- mellosouls 3y agoThis is great, not just nice to look at but useful too. Top work! How it was constructed (also really interesting) is described at the repo: https://github.com/anvaka/map-of-github#readme https://github.com/anvaka/map-of-github#readme
- anvaka 3y agothank you!
- amrb 3y agoGroup names are 10/10
- specproc 3y agoThis is a beautiful project. It's useful, aesthetically pleasing, and based on a clever methodology. Absolutely love it. Great work.
- anvaka 3y agoThank you so much for your compliment!