5 ms·

FWIW many projects written in web languages like PHP seem to be treated as javascript by Github because they also have javascript code (which happens to be more

by sheetjs 12y ago

FWIW many projects written in web languages like PHP seem to be treated as javascript by Github because they also have javascript code (which happens to be more significant / larger than the underlying code). It's unfortunate that there is no way to specify the primary language of a project. The Github language system is also somewhat unpredictable: https://github.com/SheetJS/test_files https://github.com/SheetJS/test_files seems to alternate between AppleScript and Shell with each commit (even if no .scpt or .sh file was changed or added)

flohofwoe 12y ago
Indeed, Github's language detection is absolutely useless, and a manual setting is very much needed. One of my projects is a header-only C++ lib, which is detected as 100% C, another one is a fairly typical C++11 codebase, which is detected as 50% C and 50% C++, third is a Python project but has some big example 3D asset files in three.js format which is then listed as 1.2% Python and 98.8% Javascript. So at least for my github projects, the language classification is completely off.
zongitsrinzler 12y ago
If a project mainly consists of JavaScript it should be a JavaScript project, no?
- deleted 12y ago
  [deleted]
- ubernostrum 12y ago
  Well, to take an example: I work on the Mozilla Developer Network (MDN). MDN is mostly a wiki, and the underlying codebase for that is all in Python. There's also the Demo Studio, which lets people upload web technology demos, and that's Python too. But the wiki also has an embedded script-ish language (letting people create macros/templates for specific purposes and re-use them across multiple articles). That's JavaScript-based, and so is implemented in node.js. And there's a WYSIWYG editor for the wiki pages, which is a couple of MDN-specific plugins we've developed, plus off-the-shelf components (the editor + jQuery, which we also use for a few other things on-site). So MDN is a Python project with a couple JS utilities attached. And if you set up a local copy, you can pretty easily see that Python is by far the heaviest-used language. But due to the way GitHub counts and reports statistics, it shows up as a JavaScript project. To pick on an easy example: the copy of jQuery we have in our repository weighs in at just over 9,000 lines. So just having jQuery means you need to write over 10k lines of code in order to get the "real" language of your project recognized.
  - unfunco 12y ago
    One could argue that jQuery, and other libraries shouldn't be included in the repository proper, instead it could be a submodule or included in a package manager manifest, such as bower.
    - Hovertruck 12y ago
      From the bower docs: "N.B. If you aren't authoring a package that is intended to be consumed by others (e.g., you're building a web app), you should always check installed packages into source control."
      - munro 12y ago
        The Node core dev actually went back and forth on this, and came to the conclusion to not commit dependencies to the repo, and introduced "shrinkwrap" [1]. It felt wrong committing dependencies to the repo, which I agree with. I hate my diffs being drowned in external changes, I'd rather see that someone simply upgraded a dependency. Plus it does skew the project, both for what the primary language is, as well as how much a committer is contributing. I would love to see this eventually in Bower! They have an issue for it [2]. [1] "Why not just check node_modules into git?" http://blog.nodejs.org/2012/02/27/managing-node-js-dependencies-with-shrinkwrap/ http://blog.nodejs.org/2012/02/27/managing-node-js-dependenc... [2] https://github.com/bower/bower/issues/505 https://github.com/bower/bower/issues/505
        unfunco 12y ago
        I agree. Hovertruck quoted a piece from the Bower documentation, which I can see the reasoning behind, if you're working on a web application then committing the library removes the possibility that the dependencies cannot be resolved because a library has been removed from the package manager. There are benefits to both options, but my personal preference is to keep third party libraries out of the VCS.
        munro 12y ago
- thatthatis 12y ago
  No. The way a lot of languages handle 3rd party dependencies is via a manifest file of some kind -- requirements.txt. These get installed on the system when deployed. JavaScript, however, tends to require (or at the least follow the practice) of having a copy of your JS 3rd party dependencies in your repo. So you may have 30,000 lines of python dependencies and 20,000 lines of JavaScript dependencies, but only the JS shows up in your project files to get counted. This isn't even asking the question of if the new code in the project or the total enabling code is what should be counted. If we count total enabling code, do we count the Linux kernel implicitly too?
- return0 12y ago
  Most HTML projects would then be "Images". My plain html project is marked as 'CSS', my arduino project (which contains a python file, but no java) is marked as 'Java'; that's 2/5 errors. The files that get changed/updated are the ones that matter. If most of your diffs are PHP, then you're mostly workign with PHP.
yaph 12y ago
You can avoid 3rd party libraries being counted in by following some conventions, see the patterns ignored by Github's linguist https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml https://github.com/github/linguist/blob/master/lib/linguist/...
- notwedtm 12y ago
  Why not just hash popular files and don't count ones that match?
chimeracoder 12y ago
Until December 2013, if you included non-minified Bootstrap Javascript files in your project, Linguist (the library Github uses to detect languages) would count them towards the Javascript LOC count for your project[0]. I had a number of small web server projects that were incorrectly classified as >95% Javascript for this reason. Fortunately this was fixed, but I imagine there are a number of other low-hanging fruit that are causing projects to be misclassified as well. [0] https://github.com/github/linguist/pull/856 https://github.com/github/linguist/pull/856

by sheetjs 12y ago

flohofwoe 12y ago

Indeed, Github's language detection is absolutely useless, and a manual setting is very much needed. One of my projects is a header-only C++ lib, which is detected as 100% C, another one is a fairly typical C++11 codebase, which is detected as 50% C and 50% C++, third is a Python project but has some big example 3D asset files in three.js format which is then listed as 1.2% Python and 98.8% Javascript. So at least for my github projects, the language classification is completely off.

zongitsrinzler 12y ago

If a project mainly consists of JavaScript it should be a JavaScript project, no?

deleted 12y ago
[deleted]
ubernostrum 12y ago
Well, to take an example: I work on the Mozilla Developer Network (MDN). MDN is mostly a wiki, and the underlying codebase for that is all in Python. There's also the Demo Studio, which lets people upload web technology demos, and that's Python too. But the wiki also has an embedded script-ish language (letting people create macros/templates for specific purposes and re-use them across multiple articles). That's JavaScript-based, and so is implemented in node.js. And there's a WYSIWYG editor for the wiki pages, which is a couple of MDN-specific plugins we've developed, plus off-the-shelf components (the editor + jQuery, which we also use for a few other things on-site). So MDN is a Python project with a couple JS utilities attached. And if you set up a local copy, you can pretty easily see that Python is by far the heaviest-used language. But due to the way GitHub counts and reports statistics, it shows up as a JavaScript project. To pick on an easy example: the copy of jQuery we have in our repository weighs in at just over 9,000 lines. So just having jQuery means you need to write over 10k lines of code in order to get the "real" language of your project recognized.
- unfunco 12y ago
  One could argue that jQuery, and other libraries shouldn't be included in the repository proper, instead it could be a submodule or included in a package manager manifest, such as bower.
  - Hovertruck 12y ago
    From the bower docs: "N.B. If you aren't authoring a package that is intended to be consumed by others (e.g., you're building a web app), you should always check installed packages into source control."
    - munro 12y ago
      The Node core dev actually went back and forth on this, and came to the conclusion to not commit dependencies to the repo, and introduced "shrinkwrap" [1]. It felt wrong committing dependencies to the repo, which I agree with. I hate my diffs being drowned in external changes, I'd rather see that someone simply upgraded a dependency. Plus it does skew the project, both for what the primary language is, as well as how much a committer is contributing. I would love to see this eventually in Bower! They have an issue for it [2]. [1] "Why not just check node_modules into git?" http://blog.nodejs.org/2012/02/27/managing-node-js-dependencies-with-shrinkwrap/ http://blog.nodejs.org/2012/02/27/managing-node-js-dependenc... [2] https://github.com/bower/bower/issues/505 https://github.com/bower/bower/issues/505
      - unfunco 12y ago
        I agree. Hovertruck quoted a piece from the Bower documentation, which I can see the reasoning behind, if you're working on a web application then committing the library removes the possibility that the dependencies cannot be resolved because a library has been removed from the package manager. There are benefits to both options, but my personal preference is to keep third party libraries out of the VCS.
        munro 12y ago
thatthatis 12y ago
No. The way a lot of languages handle 3rd party dependencies is via a manifest file of some kind -- requirements.txt. These get installed on the system when deployed. JavaScript, however, tends to require (or at the least follow the practice) of having a copy of your JS 3rd party dependencies in your repo. So you may have 30,000 lines of python dependencies and 20,000 lines of JavaScript dependencies, but only the JS shows up in your project files to get counted. This isn't even asking the question of if the new code in the project or the total enabling code is what should be counted. If we count total enabling code, do we count the Linux kernel implicitly too?
return0 12y ago
Most HTML projects would then be "Images". My plain html project is marked as 'CSS', my arduino project (which contains a python file, but no java) is marked as 'Java'; that's 2/5 errors. The files that get changed/updated are the ones that matter. If most of your diffs are PHP, then you're mostly workign with PHP.

yaph 12y ago

You can avoid 3rd party libraries being counted in by following some conventions, see the patterns ignored by Github's linguist https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml https://github.com/github/linguist/blob/master/lib/linguist/...

notwedtm 12y ago
Why not just hash popular files and don't count ones that match?

chimeracoder 12y ago

Until December 2013, if you included non-minified Bootstrap Javascript files in your project, Linguist (the library Github uses to detect languages) would count them towards the Javascript LOC count for your project[0]. I had a number of small web server projects that were incorrectly classified as >95% Javascript for this reason. Fortunately this was fixed, but I imagine there are a number of other low-hanging fruit that are causing projects to be misclassified as well. [0] https://github.com/github/linguist/pull/856 https://github.com/github/linguist/pull/856