9 ms·

Interesting results, but I think the farther you go down the list the more the fact that you're using the Computer Language Benchmark Game affects what you're s

• gameswithgo 8y ago
  submit a better implementation. it is not possible for any one entity to produce optimal implementations for many languages, it is up to us. or, if you dont like their rules, make your own benchmark system with new rules.
  • kbenson 8y ago
    I don't use Typescript, and use Javascript only for what I have to. I don't care about the ranking of either of them, I'm just pointing out a possible flaw in the methodology that should be taken into account when looking at the numbers presented, and what I suspect is a concrete example of that. This also isn't a criticism of the benchmark game, it's well known that not every implementation is equivalent in the time and effort put into optimizing it. It serves its purpose about as well as can be expected. Unfortunately, using it as the base of further calculations can lead to some of the known quirks of the benchmarks being exaggerated into results that are not always obviously an artifact of the underlying system, as I suspect this is. Making that obvious by pointing it out can be useful.
    • gameswithgo 8y ago
      it is absolutely a flaw in the methodology, but one I think is unavoidable without enormous resources.
      • kbenson 8y ago
        I agree. I just wanted to point it out in case people took the analysis as entirely accurate without looking a little closer. I suspect the entries towards the top of the list are fairly accurate, by nature of their competitive standing in the benchmark games and their focus as languages on performance. It's much easier for an implementation difference to hide in the natural drift you see in the languages with VMs and interpreters that don't receive as much attention. That's actually one of the reasons why the TypeScript/Javascript divide jumped out at me. They were mentioning that the bottom of the list was dominated by interpreted languages, and mentioned TypeScript by name (which surprised my given the focus JavaScript VMs have gotten), and then when I reviewed JavaScript's standing (which was more in-line with what I expected), I noticed the difference between it and TypeScipt was very pronounced, which is odd when (to my knowledge) TypseScript compiles to JavaScript, and not because it's doing a lot of convenience stuff that would slow it down. That said, I don't use TypeScript, so maybe I'm overlooking something.
        • 52-6F-62 8y ago
          I, by nature of my current job, have to write JavaScript regularly. In those cases I almost always opt for TypeScript for my sanity. So as a regular writer of TypeScript I had the exact same question as you.
      • pvg 8y ago
        It took me about a minute looking at their published detailed data to notice the problem. They should have noticed a factor-of-15ish outlier and checked why on earth it was there.
        • andrewprock 8y ago
          That's not how the game works. The community "plays the game" by submitting better benchmarks when they come up with better solutions.
          • pvg 8y ago
    • igouy 8y ago
      > Unfortunately, using it as the base of further calculations can lead to some of the known quirks of the benchmarks being exaggerated You seem not to have considered the possibility that the authors may have simply made a mistake, unrelated to the origin of the programs. The authors presented at an Oct 2017 conference. Archived benchmarks game web-pages from 2017 do not show the 10x fannkuch-redux differences that the authors report -- https://web.archive.org/web/20170918163900/http://benchmarksgame.alioth.debian.org/u64q/typescript.html https://web.archive.org/web/20170918163900/http://benchmarks...
      • kbenson 8y ago
        > Archived benchmarks game web-pages from 2017 do not show the 10x fannkuch-redux differences that the authors report I think the Sep 1st 2017 benchmark does, though.[1] At that point it's 1,204.93 seconds, compared to the 131.39 seconds on September 18th you referenced. That makes sense, since the paper could have been finished quite a bit prior to the conference. 1: https://web.archive.org/web/20170901020804/http://benchmarksgame.alioth.debian.org/u64q/typescript.html https://web.archive.org/web/20170901020804/http://benchmarks...
        • deleted 8y ago
          [deleted]
        • igouy 8y ago
          That's interesting. Do you agree that those very different times were measurements of the same TypeScript fannkuch-redux program? 5 July, Node 8.1.3, TypeScript 2.4.1 https://web.archive.org/web/20170715120038/http://benchmarksgame.alioth.debian.org/u64q/program.php?test=fannkuchredux&lang=typescript&id=2 https://web.archive.org/web/20170715120038/http://benchmarks... 1 Sep, Node 8.4.0, TypeScript 2.5.2 https://web.archive.org/web/20170922144419/http://benchmarksgame.alioth.debian.org:80/u64q/program.php?test=fannkuchredux&lang=typescript&id=2 https://web.archive.org/web/20170922144419/http://benchmarks... ---- How should we now assess your "suspiciously like entirely different algorithms were used in each implementation" comment?
• dahart 8y ago
  That’s an interesting discrepancy, I might have had the same gut reaction. But you’re using that assumption to cast slippery-slope doubt on the whole project without knowing anything specific. It’s possible you’re right, but maybe find out which algorithms are being used first? Perhaps there is a reason that TypeScript actually is an order of magnitude slower than JavaScript. The Benchmark Game project was specifically set up to allow people to scrutinize the algorithms used and make fair comparisons, so it might be unwise to start with the assumption that the most basic aspect of the project failed spectacularly and that something really obviously stupid and easy to fix is happening. I am at least giving benefit of the doubt and wondering what might be wrong with TypeScript. It’s not implausible since other languages take just as long.
  • nicoburns 8y ago
    You can write literally any JavaScript program with TypeScript (falling back to the `any` type if you really need to), so this doesn't really work in this case. Also, typical TypeScript programs are faster than typical JavaScript programs, because JavaScript JITs like predictable object shapes and monomorphic functions for the same reasons that other language implementations require them. Libraries like lodash and bluebird take advantage of this fact without using TypeScript, but TypeScript steers you towards these patterns.
    • snek 8y ago
      >typical TypeScript programs are faster than typical JavaScript programs Person who works on js engines here :) You would think this is the case, but in actuality, js and ts are about on par in performance (assuming similarly written code). This is in fact due to the fact engines optimize for idiomatic js patterns, not idiomatic typescript patterns. often these will align, but in some cases (usually revolving around generics and inheritance) well written js will actually fall through the optimization pipeline faster due to following more patterns that have specific optimization checks.
      • truncate 8y ago
        Any articles/resources on writing high-performance JS that you would recommend? I don't use JS for work, but just for random fun stuff. So just curious to learn more about it.
• zawazzi 8y ago
  Javascript in many cases runs on v8 which is a just in time compiler. Just in time compilers provide substantial performance benefits for lines of code executed multiple times. Perhaps v8 team (google) doesn't support typescript (microsoft). https://softwareengineering.stackexchange.com/questions/275497/why-doesnt-v8-compile-typescript-instead-of-javascript https://softwareengineering.stackexchange.com/questions/2754...
  • ben-schaaf 8y ago
    No one executes TypeScript directly... It's a language that compiles to JS, which can then be run using V8 among others.
• dnautics 8y ago
  I'd be very interested in seeing a few energy benchmarks not here, for example: energy to route 1B hits to a web endpoint, energy to parse a Json, energy to grab an auth token from a web request and forward to an auth server. Energy to forward 1M 1k files from the filesystem to the network, etc.
  • kbenson 8y ago
    Well, I think the reason they don't do that, and focused on the benchmark game (which I think is a good solution to their problem of needing representative sample, it just comes with its own caveats), is because much of that is often handled by libraries (possible written in C), code in the Interpreter or VM (also possibly written in C), or handled by the OS (in the case of connection handling and file access). Those are useful things to know, but don't inherently speak to the features of the language itself.
    • makapuf 8y ago
      Sure but even a language neutral ballpark could be useful to compare relative sizes. Is network or disk energy used insignificant wrt CPU or ram ? Is it roughly equivalent ?
    • rienbdj 8y ago
      The language and the runtime are basically one package. I don’t care that node is fast because it is all C under the hood. To the end user, it’s just fast.
  • TickleSteve 8y ago
    Since those operations will typically be I/O bound instead of CPU bound, that will be a system-level test, with the language having less effect. In that case, the characteristics of the system will dominate.
• jlarocco 8y ago
  > Theoretically, I would assume those to be very close, since one compiles to the other. I don't think that's a safe assumption. The Javascript results would be a lower bound (assuming the same algorithm, etc.), but there's no telling what "extra" Javascript (and thus overhead) might be inserted by the Typescript compiler. As far as individual results go, there's no point jumping to conclusions when the results and code are available online and easy to try for yourself: https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/typescript.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Typescript is still 2-3x slower than Node in several of the benchmarks, and the results could have been different in May 2018 when the article was written.
  • markmark 8y ago
    Typescript is a superset of Javascript, you can literally submit the faster JS programmes as the TS ones.
    • TallGuyShort 8y ago
      This may be a dumb question as I know nothing about Typescript, but is it idiomatic to do that, though? You can, for the most part, use C in C++. But if C was beating the pants off of C++ in benchmarks it wouldn't tell me anything useful if someone submitted a C program as the C++ benchmark. If I wanted to use idiomatic C, I'd use C. I expect a C++ vs. C comparison to compare the encouraged features of C++ with what's available in C.
      • alangpierce 8y ago
        Idiomatic TypeScript and idiomatic JavaScript are nearly the same; it's not like the C vs. C++ situation. A design goal of TypeScript is to make it possible to define types for idiomatic JavaScript programs, and I think they meet that goal pretty well. It's one reason the type system is much more advanced and flexible than, say, Java. You might see little differences here and there due to limitations in typechecking and the enum feature, but fundamentally TypeScript is just about adding structure and safety to your code rather than changing the way you code.
• userbinator 8y ago
  as not all languages get the same attention. ...which also seems to be reflective of how much programmers using language X care about (execution) efficiency, so I'd say these results are not far off from reality. C and C++ are another interesting pair --- proponents of the latter always love to claim "zero cost abstractions" that allow you to write very abstract code which they say the compiler can then optimise (or clean up the mess, depending on your viewpoint...) to the same output as if you did it manually in C, but the results show a very different picture.
  • rienbdj 8y ago
    I think it calls into question the code samples. There are very few C features not in C++.
    • fnrslvr 8y ago
      I haven't seen the code, but maybe c&p-ing the C entrant into the C++ box to get performance parity would make the C++ entrant run afoul of typical C++ coding conventions. (You can then have a debate about whether you want to see idiomatic C++ in benchmarks or purely the most performant C++ possible. Personally if I'm shopping for a language for a project then I'm far more likely to be interested in the former.)