4 ms·

There is no reason for using native extensions in scripting languages like Ruby when program load is pretty low (especially when they are used for performance r

by rgtk 10y ago

There is no reason for using native extensions in scripting languages like Ruby when program load is pretty low (especially when they are used for performance reason, not binding). Great insight, though. Author described this experience as it was a great venture... in hindsight I suppose :-)

mnarayan01 10y ago
> when program load is pretty low The leak was in a gem.
- rgtk 10y ago
  Gem is still library albeit its name.
  - mnarayan01 10y ago
    1. AFAIK it's the most popular Markdown gem around (ref. https://www.ruby-toolbox.com/categories/markup_processors https://www.ruby-toolbox.com/categories/markup_processors) so using it in a project is more "default" than it is "optimization". 2. With over 10M downloads, I assume at least some of them spend large percentages of time using it. 3. It's Markdown; replicating the idiosyncrasies of SmartyPants (https://daringfireball.net/projects/smartypants/ https://daringfireball.net/projects/smartypants/) in a different language is not something many people are going to want to do. I'm not saying that people should or should not use Redcarpet; just that it's unlikely that anyone engaged in "premature optimization" here.
teej 10y ago
I'd say it's appropriate for anything in the critical path - talking to databases being the only major thing that comes to mind. I agree that a native extension for markdown parsing is way overkill. Stick with pure-language extensions (they're so much easier to work with!) until you can measure and quantify the cost and benefit of switching.
yxhuvud 10y ago
The reason for the binding can be that no-one can be assed to reimplement the functionality. It is not as if anyone expect the scripting languages to implement their own versions of openssl.
scott_s 10y ago
Your criticism really applies to the gem authors, which to me, does not make sense. If I, as a library developer, want my library to be used on the critical path in heavy-load situations, and the library is for a language like Ruby, I'm going to figure out which part of my library are the performance critical parts, and implement them in a native language.
- haimez 10y ago
  You would think that you (in this case, "you" == "average library author in the ruby ecosystem") would, but you won't. I'm fairly confident the problem is similar, but likely worse in the javascript ecosystem. Necessity is the mother of invention, and when you're the first person to notice a memory leak, you're on your own. Both ecosystems tout the low barrier to entry as a great benefit, but memory leaks, mostly benign inefficiencies, and poor algorithmic efficiency becomes at least par for the course if not a crippling liability when reality hits your application like a freight train (or say a 2 order-of-magnitude spike).
amalag 10y ago
Now that people are figuring out how to do this in Rust things should get a lot better.

by rgtk 10y ago

mnarayan01 10y ago

> when program load is pretty low The leak was in a gem.

rgtk 10y ago
Gem is still library albeit its name.
- mnarayan01 10y ago
  1. AFAIK it's the most popular Markdown gem around (ref. https://www.ruby-toolbox.com/categories/markup_processors https://www.ruby-toolbox.com/categories/markup_processors) so using it in a project is more "default" than it is "optimization". 2. With over 10M downloads, I assume at least some of them spend large percentages of time using it. 3. It's Markdown; replicating the idiosyncrasies of SmartyPants (https://daringfireball.net/projects/smartypants/ https://daringfireball.net/projects/smartypants/) in a different language is not something many people are going to want to do. I'm not saying that people should or should not use Redcarpet; just that it's unlikely that anyone engaged in "premature optimization" here.

teej 10y ago

I'd say it's appropriate for anything in the critical path - talking to databases being the only major thing that comes to mind. I agree that a native extension for markdown parsing is way overkill. Stick with pure-language extensions (they're so much easier to work with!) until you can measure and quantify the cost and benefit of switching.

yxhuvud 10y ago

The reason for the binding can be that no-one can be assed to reimplement the functionality. It is not as if anyone expect the scripting languages to implement their own versions of openssl.

scott_s 10y ago

Your criticism really applies to the gem authors, which to me, does not make sense. If I, as a library developer, want my library to be used on the critical path in heavy-load situations, and the library is for a language like Ruby, I'm going to figure out which part of my library are the performance critical parts, and implement them in a native language.

haimez 10y ago
You would think that you (in this case, "you" == "average library author in the ruby ecosystem") would, but you won't. I'm fairly confident the problem is similar, but likely worse in the javascript ecosystem. Necessity is the mother of invention, and when you're the first person to notice a memory leak, you're on your own. Both ecosystems tout the low barrier to entry as a great benefit, but memory leaks, mostly benign inefficiencies, and poor algorithmic efficiency becomes at least par for the course if not a crippling liability when reality hits your application like a freight train (or say a 2 order-of-magnitude spike).

amalag 10y ago

Now that people are figuring out how to do this in Rust things should get a lot better.