7 ms·

Nodejs+Fabric as fast as multi-threaded C++

102 points by FabricPaul 15y ago

Srirangan 15y ago
Extremely interesting. Since we're bench marking it with C++, is it safe to assume it performs much better than the JVM?
- spullara 15y ago
  Yes, I ported it and though the JVM performed well, it is beaten by the optimizing C++ compiler. https://github.com/spullara/Benchmarks/blob/master/Server/ValueAtRisk/VarMT.java https://github.com/spullara/Benchmarks/blob/master/Server/Va...
nphase 15y ago
Fabric Server isn't available to play with?
- FabricPaul 15y ago
  Hi there - not yet. We are going to start the alpha release very soon though. You can play with the client-side stuff though - it's essentially the same system. If you sign up to the newsletter you will get notification when we drop the FE Server stuff
VMG 15y ago
The C++ versions were compiled using gcc version 4.4.5 using the compiler flags “-O6 – lpthread”. Isn't -O6 the same as -O3 ?
- apaprocki 15y ago
  Yes -O3 is the maximum. Anything higher than (-O1000) just runs -O3.
- Game_Ender 15y ago
  Yes it was a very WTF moment for me as well. The "C++" code is pretty much straight C, with references being the only C++ feature being used.
  - pjmlp 15y ago
    It might look like C, but the last time I looked to the C++ standard that is also valid C++ code.
- pjin 15y ago
  Fortran compilers (gcc has a Fortran 95 frontend) have long offered optimization levels beyond -O3; i.e., -O6 is a decent upper bound among popular free and commercial compilers, not just gcc/gfortran. The author probably came from a HPC background where -O6 is commonly used for compatibility among such a variety of compilers, even when writing C and C++ programs. Technically not the most portable way to build, but it works.
  - FabricPaul 15y ago
    I asked the engineer for a response: "For reference, I used -O6 because it's a historical convention (more of a joke, really) for "optimize the crap out of it". UNIX geeks have been using -O6 in this way for about 30 years."
allertonm 15y ago
Not that it diminishes the interestingness of this post (quite the opposite), but it's worth noting that this is not achieved using plain-vanilla JavaScript. From the product page: "The high-performance parts of the application are written using a performance-specific extension to JavaScript, called KL (Kernel Language). This language is similar in scope and syntax to JavaScript, but has some key differences that optimize it for writing high-performance code. Fabric applications are described as a dependency graph (think of a flow chart). The dependency graph describes data, and the transformations that must happen to that data. Fabric analyses the graph to discover where it is possible to perform tasks in parallel. (task-based decomposition) Fabric analyses the graph to discover where it is possible to perform the same instruction on lots of data at the same time. All of this is possible because Fabric has the LLVM compiler embedded within it. This means that applications are dynamically compiled on target. The Fabric plug-in has to be written for each platform, but Fabric applications only have to be written once. Fabric handles CPU multi-threading automatically, but the developer must explicitly write code for the GPU using OpenCL."
- sehugg 15y ago
  In other words, a better headline would be "Fabric Engine fast as multithreaded C++ (and can be glued to JS)"
  - copper 15y ago
    And, apart from a stray use of new and delete, this looks almost like valid C99 code to me. Not that that diminishes the achievement any, of course. I'd quibble mildly about his choice of compile flags and make some minor stylistic changes to the code (like the tacky transpose :), but I don't think it would affect the benchmark results any. Given the code used, it looks like it might be straightforward to use openmp rather than pthreads. I'd be interested in seeing how that works out.
- master 15y ago
  Any ideas on how this model compares to HipHop (Facebook's PHP-to-C system)?
apaprocki 15y ago
While this is interesting, not all CPUs are created equal. I can run the var-mt.cpp example on POWER hardware in: real 0m25.680s My point being, that node.js doesn't work on this box, nor does Fabric. So when developing specialized algorithms you sometimes might want to run them in specialized environments.
- FabricPaul 15y ago
  Right - we would have to build a version of the engine to run on that hardware. Once that's done though, the applications would run. i.e. we prototyped on ARM earlier this year, and our unit tests worked. That said - if you're building for specialized environments, you're probably going to want to hand-optimize code rather than rely on LLVM to do it. LLVM does a pretty good job though :)
ravloony 15y ago
So this software involves installing a plugin in the browser? In an age where users are being warned by the browser makers themselves to be careful about that sort of thing? I can't see this ending well.
- allertonm 15y ago
  In this instance - since this is a node app - they are running the engine server side. But yes, they do have a plugin version of Fabric. I have similar misgivings about that.
- FabricPaul 15y ago
  Hi there - it depends on who you're targeting. Our client-side focus is on native developers looking for high-performance in web applications e.g. medical visualization http://vimeo.com/31970502 http://vimeo.com/31970502 - the benefits of web applications are great, but performance is the current limiting factor. We are likely to build support for NaCl in the future. On the server-side, there is no requirement for a client to install a plug-in - it's for high-performance on the server i.e. semantic analysis, compute bound problems etc Last point - we also think a lot about hybrid models where performance can be accessed on both client and server. This is much longer term, but the design of Fabric allows for it.
  - FabricPaul 15y ago
    p.s. you can see our sample client-side apps here: http://vimeo.com/groups/fabric/videos http://vimeo.com/groups/fabric/videos and play with them at http://demos.fabric-engine.com http://demos.fabric-engine.com You can how we've extended Fabric to include existing C++ libraries - great for custom data types, streaming data etc It is not really a consumer web technology - we're acutely aware of the plug-in friction/antipathy. However - if you want HPC in the browser, this is one of the only ways to do it.
blub 15y ago
I wouldn't call that code C++ at all, it's C with a few C++ keywords/niceties. [1],[2] [1] https://github.com/fabric-engine/Benchmarks/blob/f11cf6cc8cf4b3cab8048bf48e14dc651532be98/Server/ValueAtRisk/var-st.cpp https://github.com/fabric-engine/Benchmarks/blob/f11cf6cc8cf... [2] https://github.com/fabric-engine/Benchmarks/blob/f11cf6cc8cf4b3cab8048bf48e14dc651532be98/Server/ValueAtRisk/var-mt.cpp https://github.com/fabric-engine/Benchmarks/blob/f11cf6cc8cf...
- Game_Ender 15y ago
  Yep and here is the C version: http://pastebin.com/z7apLBHb http://pastebin.com/z7apLBHb and it's diff: http://pastebin.com/YHjW6AHp http://pastebin.com/YHjW6AHp On my machine the C and C++ versions essentially have the same speed.
berkut 15y ago
Could probably squeeze a bit more out of the C++ version by targeting the specific architecture of the CPU to make use of SSE. Also, what floating point type is KL using? float or double? - and is double necessary? - converting the C++ code to use floats would probably provide a fair speedup on the divides and due to squeezing more data into cache lines...
- gcp 15y ago
  Could probably squeeze a bit more out of the C++ version by targeting the specific architecture of the CPU to make use of SSE. Not only that, from browsing the code, the critical loop is likely matrix multiplication. If that's the case, any kind of engine who is smart about SSE, cache lines, etc is going to be able to outperform simple C/C++ code. Of course there's excellent matrix maths libraries for C/C++ that could be used instead.
  - to3m 15y ago
    More than half of the running time seems to be taken up by the generation of normally-distributed random numbers. Sort of makes sense, I suppose, since that bit has a loop and a `sqrt' and a `log' in it. The repeated calls to `exp' seem to take up some time too. As for the matrix multiplication, that only happens on startup, so it's surely irrelevant. The bit that runs a lot just does matrix*vector. It is rather hard to make that cache-incoherent, as it just walks forwards through all inputs and outputs. In any event I would think that the program's entire working set will fit in L1. I was merely fiddling with this out of interest, so I didn't spend ages SSE2ifying it. The VC++ x64 compiler doesn't do inline assembly language anyway. But if you halve the number of multiply-adds `multMatVec' does, under the assumption that this would make it twice as fast, and that twice as fast would be what an SSE2 implementation would be like, it makes no noticeable difference. (I was fiddling with the single-threaded version, using Visual Studio 2010, compiling for x64.)
- berkut 15y ago
  Actually seems like it doesn't...: Using: -O3 -march=corei7 -msse -msse2 -msse3 -msse4 -fipa-matrix-reorg -fwhole-program made negligible (if any) difference running var-mt: Original: 30.464 30.211 30.646 with g++ options: 30.381 30.487 30.277 However, using ICC 12.0.4 (-o3 -xssse3) 25.418 25.613 25.392 On the same machine (SB 2.2Ghz).
tantalor 15y ago
Is Fabric Engine commercial software? If so, the authors of this post ought to disclose their interests as the owners.
- FabricPaul 15y ago
  Hi there - apologies. Fabric is a commercial company. Fabric Engine will be free for non-commercial use - we're currently in beta so pricing is not yet finalized. I assumed the username 'FabricPaul' was a good indication, but thanks for calling it out.
  - foobarbazetc 15y ago
    "free for non-commercial use" == commercial.
schiptsov 15y ago
a most inflated buzzword since Java and closed-source proprietary solution trying to make hype..
dubajj 15y ago
Defeating poorly optimized c++ code is far from impressive. I would also recommend NOT running any performance tests on EC2 since they throttle cpu unpredictably.
spullara 15y ago
Porting the C++ code directly to Java: macpro:ValueAtRisk sam$ time java -cp . VarMT VaR = -43.7173372179254300 real 0m36.863s user 4m48.944s sys 0m0.978s macpro:ValueAtRisk sam$ time ./var-mt VaR = -43.7173372179254329 real 0m27.048s user 3m33.971s sys 0m0.145s Pretty good showing for such a low level benchmark. https://github.com/spullara/Benchmarks/blob/master/Server/ValueAtRisk/VarMT.java https://github.com/spullara/Benchmarks/blob/master/Server/Va... It would be moderately interesting to see how well this benchmark would do in OpenCL or the like. (Benchmarks ran on a 3.33 GHz, 6-core MacPro, JDK 7 Developer Preview)
- FabricPaul 15y ago
  cool :) Thanks for doing that - we can merge in at later date and include Java results. We expose OpenCL in Fabric as an extension, as we wanted to be able to target heterogeneous hardware architectures. We didn't use this for benchmarking as we wanted to show CPU performance first. For clarity - KL does not compile down to OpenCL, you have to write for the GPU explicitly.
FabricPaul 15y ago
I've had a few email/tweet exchanges regarding KL, so I thought it would be useful to clarify a few things: - KL is a language with a syntax that is very close to Javascript. It borrows syntax from JavaScript, but not the rules of the language itself. Just like JavaScript borrows syntax from C, and OpenCL borrows from C. - There are many things in Javascript we don't support. Some things we don't currently support (eg. in-line initialization of arrays) but will probably support in the future; other things we probably won't support (eg. regular expressions as language objects) and finally there are things we will never support (ie. closures). - There are nice features of KL that Javascript does not have, for instance arithmetic operator overloading. These features are included because they are particularly useful for computational problems. - KL is not JavaScript++ - It's designed for writing high-performance operator code, not to handle everything that JavaScript can do. We don't want to reinvent the wheel :) We will work on a post to cover KL in more detail, including roadmap. You can email info at fabric-engine dot com if you have any questions you want to take offline. Thanks, Paul
agoder 15y ago
In a quick test I did, using a better compiler (gcc 4.6.2 or icc) makes it 15% faster.
- FabricPaul 15y ago
  Cool - can you contribute the code to the repository so we can test and merge it in? Thanks

102 points by FabricPaul 15y ago

Extremely interesting. Since we're bench marking it with C++, is it safe to assume it performs much better than the JVM?

spullara 15y ago
Yes, I ported it and though the JVM performed well, it is beaten by the optimizing C++ compiler. https://github.com/spullara/Benchmarks/blob/master/Server/ValueAtRisk/VarMT.java https://github.com/spullara/Benchmarks/blob/master/Server/Va...

nphase 15y ago

Fabric Server isn't available to play with?

FabricPaul 15y ago
Hi there - not yet. We are going to start the alpha release very soon though. You can play with the client-side stuff though - it's essentially the same system. If you sign up to the newsletter you will get notification when we drop the FE Server stuff

VMG 15y ago

The C++ versions were compiled using gcc version 4.4.5 using the compiler flags “-O6 – lpthread”. Isn't -O6 the same as -O3 ?

apaprocki 15y ago
Yes -O3 is the maximum. Anything higher than (-O1000) just runs -O3.
Game_Ender 15y ago
Yes it was a very WTF moment for me as well. The "C++" code is pretty much straight C, with references being the only C++ feature being used.
- pjmlp 15y ago
  It might look like C, but the last time I looked to the C++ standard that is also valid C++ code.
pjin 15y ago
Fortran compilers (gcc has a Fortran 95 frontend) have long offered optimization levels beyond -O3; i.e., -O6 is a decent upper bound among popular free and commercial compilers, not just gcc/gfortran. The author probably came from a HPC background where -O6 is commonly used for compatibility among such a variety of compilers, even when writing C and C++ programs. Technically not the most portable way to build, but it works.
- FabricPaul 15y ago
  I asked the engineer for a response: "For reference, I used -O6 because it's a historical convention (more of a joke, really) for "optimize the crap out of it". UNIX geeks have been using -O6 in this way for about 30 years."

allertonm 15y ago

Not that it diminishes the interestingness of this post (quite the opposite), but it's worth noting that this is not achieved using plain-vanilla JavaScript. From the product page: "The high-performance parts of the application are written using a performance-specific extension to JavaScript, called KL (Kernel Language). This language is similar in scope and syntax to JavaScript, but has some key differences that optimize it for writing high-performance code. Fabric applications are described as a dependency graph (think of a flow chart). The dependency graph describes data, and the transformations that must happen to that data. Fabric analyses the graph to discover where it is possible to perform tasks in parallel. (task-based decomposition) Fabric analyses the graph to discover where it is possible to perform the same instruction on lots of data at the same time. All of this is possible because Fabric has the LLVM compiler embedded within it. This means that applications are dynamically compiled on target. The Fabric plug-in has to be written for each platform, but Fabric applications only have to be written once. Fabric handles CPU multi-threading automatically, but the developer must explicitly write code for the GPU using OpenCL."

sehugg 15y ago
In other words, a better headline would be "Fabric Engine fast as multithreaded C++ (and can be glued to JS)"
- copper 15y ago
  And, apart from a stray use of new and delete, this looks almost like valid C99 code to me. Not that that diminishes the achievement any, of course. I'd quibble mildly about his choice of compile flags and make some minor stylistic changes to the code (like the tacky transpose :), but I don't think it would affect the benchmark results any. Given the code used, it looks like it might be straightforward to use openmp rather than pthreads. I'd be interested in seeing how that works out.
master 15y ago
Any ideas on how this model compares to HipHop (Facebook's PHP-to-C system)?

apaprocki 15y ago

While this is interesting, not all CPUs are created equal. I can run the var-mt.cpp example on POWER hardware in: real 0m25.680s My point being, that node.js doesn't work on this box, nor does Fabric. So when developing specialized algorithms you sometimes might want to run them in specialized environments.

FabricPaul 15y ago
Right - we would have to build a version of the engine to run on that hardware. Once that's done though, the applications would run. i.e. we prototyped on ARM earlier this year, and our unit tests worked. That said - if you're building for specialized environments, you're probably going to want to hand-optimize code rather than rely on LLVM to do it. LLVM does a pretty good job though :)

ravloony 15y ago

So this software involves installing a plugin in the browser? In an age where users are being warned by the browser makers themselves to be careful about that sort of thing? I can't see this ending well.

allertonm 15y ago
In this instance - since this is a node app - they are running the engine server side. But yes, they do have a plugin version of Fabric. I have similar misgivings about that.
FabricPaul 15y ago
Hi there - it depends on who you're targeting. Our client-side focus is on native developers looking for high-performance in web applications e.g. medical visualization http://vimeo.com/31970502 http://vimeo.com/31970502 - the benefits of web applications are great, but performance is the current limiting factor. We are likely to build support for NaCl in the future. On the server-side, there is no requirement for a client to install a plug-in - it's for high-performance on the server i.e. semantic analysis, compute bound problems etc Last point - we also think a lot about hybrid models where performance can be accessed on both client and server. This is much longer term, but the design of Fabric allows for it.
- FabricPaul 15y ago
  p.s. you can see our sample client-side apps here: http://vimeo.com/groups/fabric/videos http://vimeo.com/groups/fabric/videos and play with them at http://demos.fabric-engine.com http://demos.fabric-engine.com You can how we've extended Fabric to include existing C++ libraries - great for custom data types, streaming data etc It is not really a consumer web technology - we're acutely aware of the plug-in friction/antipathy. However - if you want HPC in the browser, this is one of the only ways to do it.

blub 15y ago

I wouldn't call that code C++ at all, it's C with a few C++ keywords/niceties. [1],[2] [1] https://github.com/fabric-engine/Benchmarks/blob/f11cf6cc8cf4b3cab8048bf48e14dc651532be98/Server/ValueAtRisk/var-st.cpp https://github.com/fabric-engine/Benchmarks/blob/f11cf6cc8cf... [2] https://github.com/fabric-engine/Benchmarks/blob/f11cf6cc8cf4b3cab8048bf48e14dc651532be98/Server/ValueAtRisk/var-mt.cpp https://github.com/fabric-engine/Benchmarks/blob/f11cf6cc8cf...

Game_Ender 15y ago
Yep and here is the C version: http://pastebin.com/z7apLBHb http://pastebin.com/z7apLBHb and it's diff: http://pastebin.com/YHjW6AHp http://pastebin.com/YHjW6AHp On my machine the C and C++ versions essentially have the same speed.

berkut 15y ago

Could probably squeeze a bit more out of the C++ version by targeting the specific architecture of the CPU to make use of SSE. Also, what floating point type is KL using? float or double? - and is double necessary? - converting the C++ code to use floats would probably provide a fair speedup on the divides and due to squeezing more data into cache lines...

gcp 15y ago
Could probably squeeze a bit more out of the C++ version by targeting the specific architecture of the CPU to make use of SSE. Not only that, from browsing the code, the critical loop is likely matrix multiplication. If that's the case, any kind of engine who is smart about SSE, cache lines, etc is going to be able to outperform simple C/C++ code. Of course there's excellent matrix maths libraries for C/C++ that could be used instead.
- to3m 15y ago
  More than half of the running time seems to be taken up by the generation of normally-distributed random numbers. Sort of makes sense, I suppose, since that bit has a loop and a `sqrt' and a `log' in it. The repeated calls to `exp' seem to take up some time too. As for the matrix multiplication, that only happens on startup, so it's surely irrelevant. The bit that runs a lot just does matrix*vector. It is rather hard to make that cache-incoherent, as it just walks forwards through all inputs and outputs. In any event I would think that the program's entire working set will fit in L1. I was merely fiddling with this out of interest, so I didn't spend ages SSE2ifying it. The VC++ x64 compiler doesn't do inline assembly language anyway. But if you halve the number of multiply-adds `multMatVec' does, under the assumption that this would make it twice as fast, and that twice as fast would be what an SSE2 implementation would be like, it makes no noticeable difference. (I was fiddling with the single-threaded version, using Visual Studio 2010, compiling for x64.)
berkut 15y ago
Actually seems like it doesn't...: Using: -O3 -march=corei7 -msse -msse2 -msse3 -msse4 -fipa-matrix-reorg -fwhole-program made negligible (if any) difference running var-mt: Original: 30.464 30.211 30.646 with g++ options: 30.381 30.487 30.277 However, using ICC 12.0.4 (-o3 -xssse3) 25.418 25.613 25.392 On the same machine (SB 2.2Ghz).

tantalor 15y ago

Is Fabric Engine commercial software? If so, the authors of this post ought to disclose their interests as the owners.

FabricPaul 15y ago
Hi there - apologies. Fabric is a commercial company. Fabric Engine will be free for non-commercial use - we're currently in beta so pricing is not yet finalized. I assumed the username 'FabricPaul' was a good indication, but thanks for calling it out.
- foobarbazetc 15y ago
  "free for non-commercial use" == commercial.

schiptsov 15y ago

a most inflated buzzword since Java and closed-source proprietary solution trying to make hype..

dubajj 15y ago

Defeating poorly optimized c++ code is far from impressive. I would also recommend NOT running any performance tests on EC2 since they throttle cpu unpredictably.

spullara 15y ago

Porting the C++ code directly to Java: macpro:ValueAtRisk sam$ time java -cp . VarMT VaR = -43.7173372179254300 real 0m36.863s user 4m48.944s sys 0m0.978s macpro:ValueAtRisk sam$ time ./var-mt VaR = -43.7173372179254329 real 0m27.048s user 3m33.971s sys 0m0.145s Pretty good showing for such a low level benchmark. https://github.com/spullara/Benchmarks/blob/master/Server/ValueAtRisk/VarMT.java https://github.com/spullara/Benchmarks/blob/master/Server/Va... It would be moderately interesting to see how well this benchmark would do in OpenCL or the like. (Benchmarks ran on a 3.33 GHz, 6-core MacPro, JDK 7 Developer Preview)

FabricPaul 15y ago
cool :) Thanks for doing that - we can merge in at later date and include Java results. We expose OpenCL in Fabric as an extension, as we wanted to be able to target heterogeneous hardware architectures. We didn't use this for benchmarking as we wanted to show CPU performance first. For clarity - KL does not compile down to OpenCL, you have to write for the GPU explicitly.

FabricPaul 15y ago

I've had a few email/tweet exchanges regarding KL, so I thought it would be useful to clarify a few things: - KL is a language with a syntax that is very close to Javascript. It borrows syntax from JavaScript, but not the rules of the language itself. Just like JavaScript borrows syntax from C, and OpenCL borrows from C. - There are many things in Javascript we don't support. Some things we don't currently support (eg. in-line initialization of arrays) but will probably support in the future; other things we probably won't support (eg. regular expressions as language objects) and finally there are things we will never support (ie. closures). - There are nice features of KL that Javascript does not have, for instance arithmetic operator overloading. These features are included because they are particularly useful for computational problems. - KL is not JavaScript++ - It's designed for writing high-performance operator code, not to handle everything that JavaScript can do. We don't want to reinvent the wheel :) We will work on a post to cover KL in more detail, including roadmap. You can email info at fabric-engine dot com if you have any questions you want to take offline. Thanks, Paul

agoder 15y ago

In a quick test I did, using a better compiler (gcc 4.6.2 or icc) makes it 15% faster.

FabricPaul 15y ago
Cool - can you contribute the code to the repository so we can test and merge it in? Thanks