4 ms·
OpenSSL memory use in Node.js
- HerraBRE 16y agoThe article also covers the same issue in Python and shows how to drastically reduce memory usage. Pretty awesome stuff!
- lloeki 16y agoThis is great not because Python is my language of choice but because this shows how to generalize it to other cases and not stay in node land. I wish more articles would take this approach.
- asolove 16y agoThis is a really good writeup. I have never used Instruments before, and while I'm not an expert in OpenSSL, there was enough detail to point me in the right direction when dealing with this in the future. Thanks.
- AngryParsley 16y agoI think a more accurate title would be "How to reduce memory usage 10x in almost anything that uses OpenSSL." Bottom line: All those bookkeeping data structures for compression take up a lot of space. Disable SSL compression and you can go from 1MB per connection down to 100kB.
- JoachimSchipper 16y agoSummary: when using OpenSSL with a lot of connections, use a shared SSL_CTX or you'll have a large per-connection overhead.
- agl 16y agoDon't forget to enable SSL_MODE_RELEASE_BUFFERS with OpenSSL 1.0.0 too.
- pquerna 16y agoAh, good point, Node already does this, but I'll append the post to mention it.
- otterley 16y agoI'd like NodeJS to add a tls.createCTX() method that generates an OpenSSL CTX object that can be manipulated and passed as an optional parameter to the various other tls methods. This would allow great flexibility in the TLS configuration - everything from CA paths to CRL checking to RFC 5077 support could be configured by the application once and the settings reused for all qualifying subsequent connections. Compare with, e.g., AnyEvent::TLS (http://search.cpan.org/~mlehmann/AnyEvent-5.31/lib/AnyEvent/TLS.pm http://search.cpan.org/~mlehmann/AnyEvent-5.31/lib/AnyEvent/...).
- nathanwdavis 16y agoI love posts like this. There are too many blog posts that make it to the top of HN that are just complaining or speculating of some type. I would much rather read a detailed write-up on the discovery and solution to a problem.
- pronoiac 16y agoEep! Skipping compression before you encrypt is a bad idea. Using it saves bandwidth, & makes attacks much more complicated. Can Node.js do some compression before it hands off data to OpenSSL?
- newman314 16y agoProbably if you use something like node-compress or node-zippy.
- caf 16y agoIf you want to debug an issue like this on Linux, the Massif tool (part of Valgrind) is a decent heap profiler.
- shushan 16y agoWhat is the windows variation to those/Instruments.app? (generally speaking, I am aware that it might not apply to debugging node apps running on top of cygwin)
- MadWombat 15y agoIt is interesting, how he is using OpenSSL, Node.js, Python, Twisted framework and still manages to take a swing at OSS development tools in favor of closed ones. Does this issue even exists on serious operating systems?