4 ms·
Somehow torvalds/linux is in Fronterra, next to JS projects, awesome-X lists, and frontend checklists. Either kernel hackers unexpectedly love frontend, or mor
by tux3 2y ago
Somehow torvalds/linux is in Fronterra, next to JS projects, awesome-X lists, and frontend checklists.
Either kernel hackers unexpectedly love frontend, or more likely the people that write the code don't overlap much with the people that star Github projects!
- revskill 2y agoBecause of react ?
- jensenbox 2y agoThat was my first reaction.
- moffkalast 2y agoWhat's your angle?
- dbrans 2y agoCan you elaborate?
- odo1242 2y agoWait, why React?
- dataviz1000 2y agoPerhaps the same reason heat maps are often really the underlining population map https://xkcd.com/1138/ https://xkcd.com/1138/
- wodenokoto 2y agoThat’s why in NLP we use term frequency over inverse document frequency. It gives you a measure of common uncommon things are. Wonder how you’d implement that in a heat map. Just call each pixel a document and see where it takes you?
- supriyo-biswas 2y agoWinsorize the data points to remove outliers and then divide it by the population count for the case of the heatmap?
- bravura 2y agoPeople have been critiquing the collaborative filtering aspect of this work vs content analysis ("[why use stars instead of code similarity]") but there's something elegant about the simplicity of using less priors here. A tf*idf matrix could be applied to the star-feature matrix too. Document = github repo. Term = name of user who starred it. THUS, users who overstar are simply less important for computing similarities. This would mitigate the phenomenon of massively popular github repos being clustered together because of folks who blithely star the most well known stuff.
- supriyo-biswas 2y agoI wonder if code embeddings might have been a better way to organize the projects, although probably infeasible given the amount of resources required to download and compute embeddings for each file.
- machiaweliczny 2y agoEmbeddings are super cheap to compute
- anvaka 2y agoJaccard similarity is not particularly good for "celebrity" projects. They are similar because they are popular, not because there is semantic relationship. It's the same problem I faced with the map of reddit (https://anvaka.github.io/map-of-reddit/ https://anvaka.github.io/map-of-reddit/ ) - all popular subreddits are just "similar" to each other. Stil works great for smaller, non-celebrity projects :D