7 ms·

Duetto: a C++ compiler for the Web going beyond Emscripten and Node.js

46 points by apignotti 13y ago

blueblob 13y ago
Are there any tutorials on this so that we can compare it to emscripten?
- multimillion 13y ago
  We will publish extensive tutorials soon, for now you can find some examples on http://leaningtech.com/duetto/examples/ http://leaningtech.com/duetto/examples/ In addition, the full source code of Nontetris ( http://allievi.sssup.it/jacopone/cnontetris/ http://allievi.sssup.it/jacopone/cnontetris/ ) will be published in a couple of days, with a 'guide' on porting a C++ game to Javascript with duetto. Stefano of Leaningtech
  - kirab 13y ago
    Hi Stefano, there are four important things required for this to work out: - List of benefits of your solutions compared to other C++ → JS compilers (emscripten) - Great documentation and good community support (forum, wiki, dynamic list of projects using Duetto) - Very good debugging capabilities in the produced JS (Developers need to work with the code, an invisible layer sometimes means a lot of trouble) - Big tech partners who will do complex projects with Duetto to iron-out the compiler and show "real work" can be done with it rgds, Kira
  - flohofwoe 13y ago
    FYI: http://leaningtech.com/duetto/examples/ http://leaningtech.com/duetto/examples/ is broken on OSX in Chrome and FF for me, it looks like a mobile page is served which makes the code examples unreadable. It looks good in Chrome on Windows7, but also broken in FF on Windows7. Haven't test other platforms. Wow: and a minute after I typed this, my Windows machine gave me a blue screen, not sure if it's related but it's the first in years ;)
  - flohofwoe 13y ago
    For games I think you need to show that duetto is just as fast as emscripten when running. emscripten-generated code is very light on the garbage collector (inside emscripten generated code no garbage is produced at all, only in some wrapper code which needs to talk to JS libs), and having the "heap" in a plain typed array usually also gives a performance advantage, since as in C++ you have exact control of the memory layout of your data structures. Not having to construct and destroy JS objects all the time, and keeping all data in typed arrays is essential for performance I think. This is even true without diving into asm.js.
    - apignotti 13y ago
      We have written a detailed post a few months ago comparing duetto to emscripten. We also explain why we think it's not a good idea to use a pre-allocated typed array heap: http://leaningtech.com/duetto/blog/2013/05/28/Comparing-to-asm.js/ http://leaningtech.com/duetto/blog/2013/05/28/Comparing-to-a...
      - flohofwoe 13y ago
        Disclaimer: speaking strictly from a game-devs perspective: I remember that post and from a game dev's perspective the pre-allocated-memory vs. reclaimed-through-garbage-collection argument is a bit weak IMHO. Games have predictable memory usage, and game devs are used to see memory as a static, finite resource anyway (since that's how it is on consoles). Grabbing a chunk of memory upfront is quite common for games. And as for the (big) size of the pre-allocated memory: either your game needs that much memory at a time or it doesn't, if the user doesn't have that much free you're f*cked either way. The perf comparison is missing a "asm.js + V8" bar, asm.js code is usually faster then non-asm.js code, even if the JS engine isn't implementing AoT compilation as FF does. Edit: fixed "...uncommon -> common..." above
        angersock 13y ago
        From the link: "Since I do not expect native platforms (i.e. GLibc) to preallocate gigs of memory at application startup to make it faster when dynamic memory is actually used, I do not expect this from a compiler for the JavaScript target either." So, it would actually make very good sense to have the thing slab allocate a big honking array upfront, and then manage it with brk/sbrk/malloc, and perhaps just add more memory as needed. Like, if we're going to be writing C/C++ in the browser, why not do this? Why make us deal with the GC at all?
  - icefox 13y ago
    You should include a link to the examples in the copy
mercurial 13y ago
I'm not sure why you would want something like that, unless you really have a lot of existing code C++ business code you want to reuse, or you really love C++.
- kirab 13y ago
  I really love C++ and I bet with the currently developing standard (C++11, C++14) more people will start loving C++ again :-)
  - trimbo 13y ago
    I've been away from it for a few years and just recently picked up a new project in C++11. I'm loving the improvements.
  - chrismorgan 13y ago
    Have you tried Rust? It's not an entirely stable language yet, but I've been absolutely loving it thus far in my few months of close involvement. C++11 and C++14 are vast improvements, but I reckon Rust is still considerably better than them. As far as Rust for the web is concerned, I've got pretty big ideas which Rust is instrumental in (I've been primarily a Python programmer hitherto, FYI), and so I'm working on the lowest layer at present, in rust-http (https://githib.com/chris-morgan/rust-http https://githib.com/chris-morgan/rust-http). By the time it's entirely ready for production use, I expect someone to have got Rust working with Emscripten too, and then we'll be ready to do even better than Duetto.
  - betterunix 13y ago
    "I bet with the currently developing standard (C++11, C++14) more people will start loving C++ again" I doubt it. C++11 takes C++ even further from where it needs to be. Every high-level feature is bogged down by low-level annoyances. Do you capture variables by value or by reference? Do you need a weak pointer to break this cycle, or can you use a weak pointer somewhere else (or maybe just not both with smart pointers at all?)? Your destructor has an error to report -- now what? The reason nobody likes C++ is that it is overly complicated for low-level programming and too low-level and annoying for high-level programming. At best the only thing people can say about C++ is that it is "good enough" and popular.
chrismorgan 13y ago
That cnontetris example seemed to be having dire effects on Firefox's memory usage. It was using 21% of my 4GB, but then when I started that it went jumping up to over 80% and back down again and back and forth. Is it really causing the allocation and freeing of over 2GB of RAM just like that every so often? Oh, by the way—the game was entirely unplayable for reasons likely to be related to this memory thrashing.
- apignotti 13y ago
  Our memory management approach is still not tuned and optimized. There are quite a few low hanging fruits that should provide great improvements, but we have not yet implemented them - Alessandro of Leaningtech
  - chrismorgan 13y ago
    Would you mind if I suggested that arbitrarily using a couple of gigabytes of memory like that was a fairly severe problem? :-) Seriously, I would like to see a warning put on links to that example to the effect that at present it will make your browser use up to 2½ GB more RAM and will thrash your memory. Because it did render my browser (and to a lesser degree my system) unresponsive for a couple of minutes while things swapped.
    - apignotti 13y ago
      You are absolutely right, it is a severe problem and I apologize for this. We are adding a warning on our examples page right now.
nrser 13y ago
i think you're out of memories brah: Fatal error: Out of memory (allocated 21757952) (tried to allocate 7680 bytes) in /var/www/techblog/wp-content/plugins/disqus-comment-system/lib/api/disqus/url.php on line 55
angersock 13y ago
It's a slow morning, so this was good for a chuckle: " Bring the robustness and proven scalability of C++ programming to the Web " Ha. Ha ha. Ha ha ha.
- okamiueru 13y ago
  A bit unrelated, but does anyone know at what comment score one gets to downvote non-constructive comments, or is that what the flagging is for?
  - angersock 13y ago
    Downvoting comes around 500 maybe? Not actually sure. Anyways, lest you think my observation is just a "non-constructive comment": The great thing about the browser is that it is free of a lot of garbage we have to deal with in C++. The antics required to make C++ work in the browser, while laudable, are absurd. If you need cross-platform code, write in Java and avoid native extensions. If you need cross-platform code that has to be fast (i.e., you actually need to have inline assembly), write in C or C++ and actually do your proper cross-platform work (proper abstraction of I/O, proper defines, proper selection of types, etc.). If you need to draw shit in the browser, use the wonderful ecosystem of tools that have evolved using the standard affordances of Javascript and CSS. Here are the bits I take issue with: "write web applications in C++, reusing existing code and making porting of whole applications and games to the browser plausible." Why? What type of code is worth reusing? What type of performance is this going to get me? Most C++ codebases are janky and sad--game engines being some of the worst offenders. " code both the frontend and the backend of a web application in the same language and codebase " Why the fuck would you ever want to use C++ as that language? It's terrible, so much so in fact that it lost out to PHP and Perl. Let that sink in for a second. People said, "I would really like a language that's easier to work in!" and then picked PHP and Perl. PHP and Perl. Moreover, go look at their docs: http://www.leaningtech.com/duetto/examples/ http://www.leaningtech.com/duetto/examples/ " Duetto does not attempt at emulating numerical types which are not supported by JavaScript float is supported with 64-bit precision, like double. long long is defined as a 32-bit integer. 64-bit integers are not supported at all in JavaScript and duetto does not attempt to emulate them." Well thank Christ we never will need numerical types not in Javascript--I mean, I've never used bit-twiddling in my engine code, or my decompression code, or my audio code. ~ Look, it's a really cool use of LLVM, but we don't need C++ in the browser.
apignotti 13y ago
As many have noticed the blog is being seriously crushed by the heavy traffic. We are mirroring the content here: http://leaningtech.com/duetto/blog/2013/10/31/Duetto-Released/ http://leaningtech.com/duetto/blog/2013/10/31/Duetto-Release...
- angersock 13y ago
  snip EDIT: After some consideration, I've removed my post. Though the irony is great, the fact is that this is a cool use of LLVM and a pretty interesting thought-experiment made live. I know I wouldn't appreciate the kind of heckling I'd just posted were our roles reversed. Good work on the technical stuff folks.
  - mtinkerhess 13y ago
    Cut them some slack, jeez.
    - angersock 13y ago
      Done.
      - apignotti 13y ago
        I really appreciate it. We are doing our best with the limited capabilities of three persons. Thanks a lot. - Alessandro of Leaningtech
        angersock 13y ago
        You guys are doing great work! Please don't confuse my disdain for the concept of C++ in the browser for a lack of appreciation for your work.
utopcell 13y ago
I wonder how this compares to wt (http://www.webtoolkit.eu/wt http://www.webtoolkit.eu/wt) in terms of performance.
nonchalance 13y ago
> reducing the limitations and obstacles given by Javascript What are these perceived limitations? There are fairly standard idioms and pure JS libraries for getting around most problems people find

46 points by apignotti 13y ago

blueblob 13y ago

Are there any tutorials on this so that we can compare it to emscripten?

multimillion 13y ago
We will publish extensive tutorials soon, for now you can find some examples on http://leaningtech.com/duetto/examples/ http://leaningtech.com/duetto/examples/ In addition, the full source code of Nontetris ( http://allievi.sssup.it/jacopone/cnontetris/ http://allievi.sssup.it/jacopone/cnontetris/ ) will be published in a couple of days, with a 'guide' on porting a C++ game to Javascript with duetto. Stefano of Leaningtech
- kirab 13y ago
  Hi Stefano, there are four important things required for this to work out: - List of benefits of your solutions compared to other C++ → JS compilers (emscripten) - Great documentation and good community support (forum, wiki, dynamic list of projects using Duetto) - Very good debugging capabilities in the produced JS (Developers need to work with the code, an invisible layer sometimes means a lot of trouble) - Big tech partners who will do complex projects with Duetto to iron-out the compiler and show "real work" can be done with it rgds, Kira
- flohofwoe 13y ago
  FYI: http://leaningtech.com/duetto/examples/ http://leaningtech.com/duetto/examples/ is broken on OSX in Chrome and FF for me, it looks like a mobile page is served which makes the code examples unreadable. It looks good in Chrome on Windows7, but also broken in FF on Windows7. Haven't test other platforms. Wow: and a minute after I typed this, my Windows machine gave me a blue screen, not sure if it's related but it's the first in years ;)
- flohofwoe 13y ago
  For games I think you need to show that duetto is just as fast as emscripten when running. emscripten-generated code is very light on the garbage collector (inside emscripten generated code no garbage is produced at all, only in some wrapper code which needs to talk to JS libs), and having the "heap" in a plain typed array usually also gives a performance advantage, since as in C++ you have exact control of the memory layout of your data structures. Not having to construct and destroy JS objects all the time, and keeping all data in typed arrays is essential for performance I think. This is even true without diving into asm.js.
  - apignotti 13y ago
    We have written a detailed post a few months ago comparing duetto to emscripten. We also explain why we think it's not a good idea to use a pre-allocated typed array heap: http://leaningtech.com/duetto/blog/2013/05/28/Comparing-to-asm.js/ http://leaningtech.com/duetto/blog/2013/05/28/Comparing-to-a...
    - flohofwoe 13y ago
      Disclaimer: speaking strictly from a game-devs perspective: I remember that post and from a game dev's perspective the pre-allocated-memory vs. reclaimed-through-garbage-collection argument is a bit weak IMHO. Games have predictable memory usage, and game devs are used to see memory as a static, finite resource anyway (since that's how it is on consoles). Grabbing a chunk of memory upfront is quite common for games. And as for the (big) size of the pre-allocated memory: either your game needs that much memory at a time or it doesn't, if the user doesn't have that much free you're f*cked either way. The perf comparison is missing a "asm.js + V8" bar, asm.js code is usually faster then non-asm.js code, even if the JS engine isn't implementing AoT compilation as FF does. Edit: fixed "...uncommon -> common..." above
      - angersock 13y ago
        From the link: "Since I do not expect native platforms (i.e. GLibc) to preallocate gigs of memory at application startup to make it faster when dynamic memory is actually used, I do not expect this from a compiler for the JavaScript target either." So, it would actually make very good sense to have the thing slab allocate a big honking array upfront, and then manage it with brk/sbrk/malloc, and perhaps just add more memory as needed. Like, if we're going to be writing C/C++ in the browser, why not do this? Why make us deal with the GC at all?
- icefox 13y ago
  You should include a link to the examples in the copy

mercurial 13y ago

I'm not sure why you would want something like that, unless you really have a lot of existing code C++ business code you want to reuse, or you really love C++.

kirab 13y ago
I really love C++ and I bet with the currently developing standard (C++11, C++14) more people will start loving C++ again :-)
- trimbo 13y ago
  I've been away from it for a few years and just recently picked up a new project in C++11. I'm loving the improvements.
- chrismorgan 13y ago
  Have you tried Rust? It's not an entirely stable language yet, but I've been absolutely loving it thus far in my few months of close involvement. C++11 and C++14 are vast improvements, but I reckon Rust is still considerably better than them. As far as Rust for the web is concerned, I've got pretty big ideas which Rust is instrumental in (I've been primarily a Python programmer hitherto, FYI), and so I'm working on the lowest layer at present, in rust-http (https://githib.com/chris-morgan/rust-http https://githib.com/chris-morgan/rust-http). By the time it's entirely ready for production use, I expect someone to have got Rust working with Emscripten too, and then we'll be ready to do even better than Duetto.
- betterunix 13y ago
  "I bet with the currently developing standard (C++11, C++14) more people will start loving C++ again" I doubt it. C++11 takes C++ even further from where it needs to be. Every high-level feature is bogged down by low-level annoyances. Do you capture variables by value or by reference? Do you need a weak pointer to break this cycle, or can you use a weak pointer somewhere else (or maybe just not both with smart pointers at all?)? Your destructor has an error to report -- now what? The reason nobody likes C++ is that it is overly complicated for low-level programming and too low-level and annoying for high-level programming. At best the only thing people can say about C++ is that it is "good enough" and popular.

chrismorgan 13y ago

That cnontetris example seemed to be having dire effects on Firefox's memory usage. It was using 21% of my 4GB, but then when I started that it went jumping up to over 80% and back down again and back and forth. Is it really causing the allocation and freeing of over 2GB of RAM just like that every so often? Oh, by the way—the game was entirely unplayable for reasons likely to be related to this memory thrashing.

apignotti 13y ago
Our memory management approach is still not tuned and optimized. There are quite a few low hanging fruits that should provide great improvements, but we have not yet implemented them - Alessandro of Leaningtech
- chrismorgan 13y ago
  Would you mind if I suggested that arbitrarily using a couple of gigabytes of memory like that was a fairly severe problem? :-) Seriously, I would like to see a warning put on links to that example to the effect that at present it will make your browser use up to 2½ GB more RAM and will thrash your memory. Because it did render my browser (and to a lesser degree my system) unresponsive for a couple of minutes while things swapped.
  - apignotti 13y ago
    You are absolutely right, it is a severe problem and I apologize for this. We are adding a warning on our examples page right now.

nrser 13y ago

i think you're out of memories brah: Fatal error: Out of memory (allocated 21757952) (tried to allocate 7680 bytes) in /var/www/techblog/wp-content/plugins/disqus-comment-system/lib/api/disqus/url.php on line 55

angersock 13y ago

It's a slow morning, so this was good for a chuckle: " Bring the robustness and proven scalability of C++ programming to the Web " Ha. Ha ha. Ha ha ha.

okamiueru 13y ago
A bit unrelated, but does anyone know at what comment score one gets to downvote non-constructive comments, or is that what the flagging is for?
- angersock 13y ago
  Downvoting comes around 500 maybe? Not actually sure. Anyways, lest you think my observation is just a "non-constructive comment": The great thing about the browser is that it is free of a lot of garbage we have to deal with in C++. The antics required to make C++ work in the browser, while laudable, are absurd. If you need cross-platform code, write in Java and avoid native extensions. If you need cross-platform code that has to be fast (i.e., you actually need to have inline assembly), write in C or C++ and actually do your proper cross-platform work (proper abstraction of I/O, proper defines, proper selection of types, etc.). If you need to draw shit in the browser, use the wonderful ecosystem of tools that have evolved using the standard affordances of Javascript and CSS. Here are the bits I take issue with: "write web applications in C++, reusing existing code and making porting of whole applications and games to the browser plausible." Why? What type of code is worth reusing? What type of performance is this going to get me? Most C++ codebases are janky and sad--game engines being some of the worst offenders. " code both the frontend and the backend of a web application in the same language and codebase " Why the fuck would you ever want to use C++ as that language? It's terrible, so much so in fact that it lost out to PHP and Perl. Let that sink in for a second. People said, "I would really like a language that's easier to work in!" and then picked PHP and Perl. PHP and Perl. Moreover, go look at their docs: http://www.leaningtech.com/duetto/examples/ http://www.leaningtech.com/duetto/examples/ " Duetto does not attempt at emulating numerical types which are not supported by JavaScript float is supported with 64-bit precision, like double. long long is defined as a 32-bit integer. 64-bit integers are not supported at all in JavaScript and duetto does not attempt to emulate them." Well thank Christ we never will need numerical types not in Javascript--I mean, I've never used bit-twiddling in my engine code, or my decompression code, or my audio code. ~ Look, it's a really cool use of LLVM, but we don't need C++ in the browser.

apignotti 13y ago

As many have noticed the blog is being seriously crushed by the heavy traffic. We are mirroring the content here: http://leaningtech.com/duetto/blog/2013/10/31/Duetto-Released/ http://leaningtech.com/duetto/blog/2013/10/31/Duetto-Release...

angersock 13y ago
snip EDIT: After some consideration, I've removed my post. Though the irony is great, the fact is that this is a cool use of LLVM and a pretty interesting thought-experiment made live. I know I wouldn't appreciate the kind of heckling I'd just posted were our roles reversed. Good work on the technical stuff folks.
- mtinkerhess 13y ago
  Cut them some slack, jeez.
  - angersock 13y ago
    Done.
    - apignotti 13y ago
      I really appreciate it. We are doing our best with the limited capabilities of three persons. Thanks a lot. - Alessandro of Leaningtech
      - angersock 13y ago
        You guys are doing great work! Please don't confuse my disdain for the concept of C++ in the browser for a lack of appreciation for your work.

utopcell 13y ago

I wonder how this compares to wt (http://www.webtoolkit.eu/wt http://www.webtoolkit.eu/wt) in terms of performance.

nonchalance 13y ago

> reducing the limitations and obstacles given by Javascript What are these perceived limitations? There are fairly standard idioms and pure JS libraries for getting around most problems people find