4 ms·

Slightly related: I'm in the process of choosing the best way to give our Go binaries some scripting capabilities (it's entreprise stuff, the idea is "let's the

by bpizzi 9y ago

Slightly related: I'm in the process of choosing the best way to give our Go binaries some scripting capabilities (it's entreprise stuff, the idea is "let's the customer script this behaviour"). I'm aware of Lua and JS interpreters, some fully native, some made of binding against existing interpreters. Native JS interpreters have my preference right now: easier to build/maintain, I can bear the loss of perfs, and JS seems easier to sell than Lua (remember it's entreprise stuff). Does anyone have some insight or experience in that area and care to give feedback to a fellow HNer? And did I miss other scripting languages?

Derbasti 9y ago
Lua is incredibly easy to embed. The whole Lua interpreter including the standard library is about 150kb. That's Kilo, not mega! You can swap out the memory allocator or the number type on one simple header file. Lua was built specifically for embedding! What is more, it is a joyously well designed language that is well suited for building DSLs. And it is pretty fast and memory efficient for a dynamic language. Other options are TKL, but the syntax is a bit unorthodox. Or Python, but it is much bigger, and harder to embed. JS is not a great fit, I would say, mostly because it is comparatively HUGE and not particularly optimized for embedding.
- hnlmorg 9y ago
  Perl 5 is another option. There's a few embedded Perl libraries out there. But Perl has a bad reputation (undeservingly in my personal opinion) so potentially another hard sell for enterprise.
  - hnlmorg 9y ago
    I guess it's too much to ask for whoever downvoted my comment to explain why they did? Otherwise it's going to look little more than a knee-jerk reaction from some Perl hater.
    - Yaggo 9y ago
      I didn't downvote you, but I'd like to know what makes Perl a good option for embedded language and what implementations are available.
      - hnlmorg 9y ago
        Like with Python, Javascript, etc Perl has got a huge and collaborative eco-system (CPAN is the godfather (in terms of age) of npm / pip / et al). It's also terse enough that writing scripts doesn't become a bigger job than it needs to be. The language also offers better type safety than Javascript and most sysadmins are already proficient in Perl because, until recently, it used to be the goto scripting language for servers (these days it seems to be more Python) so Perl makes some sense for enterprise systems. There are big drawbacks for using Perl though. It's appallingly bad for writing OO code (objects in Perl are really just hashes / maps) so larger code bases or complex code can quickly become a pain to maintain. While it has one of the better approaches for comparing data of different types (imo), Perl still has more than it's fair share of hidden traps. Ultimately though, it's a better language than many give it credit for even if it is considered old fashioned these days. So if languages like Python and Javascript are to be considered then I think Perl deserves a mention as well. As for implementations, the standard Perl interpreter can be linked to which makes it very easy to embed into other programs.
    - wahern 9y ago
      I didn't downvote you, but IME Perl is a poor language for embedding. Perl XS is itself pretty bad, whether embedding a C application with Perl, extending a Perl application with C, and especially if you're doing C->Perl->C->Perl mixes--the Perl XS interfaces quickly become leaky abstractions. And don't even get me started on the build process.... I've written far more than my fair share of Perl XS code over the past nearly 15 years. I still use Perl and the last time I wrote Perl XS module was just last month. I could go on for pages about the negative aspects, without having much to say about the positive aspects. But here's a very concrete reason why nobody should contemplate embedding Perl: I filed this bug report (Thread race in dist/IO/IO.xs) over two years ago: https://rt.perl.org/Public/Bug/Display.html?id=125685 It's still not fixed. The fix isn't easy, either. It was introduced in an attempt to fix some other stuff. But maybe the biggest reason it hasn't yet been fixed is because the bug is only likely to be encountered when creating multiple Perl interpreter instances in separate threads--a very uncommon usage for Perl.[1] Uncommon usages will not be (and are not) well supported. Non-threaded Perl embedding may be more common, but ultimately you don't want to be embedding any interpreter that has shared state, especially if the fact of this shared state leaks into interfaces or breaks the interface contracts. Python is also still problematic in terms of not being able to completely erase vestigial assumptions of one interpreter per process. Neither Perl nor Python were originally designed for embedding, and even their extension APIs were bolted on. While they've both been improved over the years and work well for straight-forward C bindings, the fact that neither the languages nor implementations were originally designed for embedding eventually comes through and cause headaches when you want to do anything sophisticated--sometimes even when you're not doing anything sophisticated. [1] The only significant use of Perl in the open source world that I'm familiar with that embeds Perl this way (multiple instances, multiple threads) is Apache mod_perl. But for other reasons mod_perl creates and initializes all Perl instances from a single thread. This is why the bug went undiscovered for so long after it was introduced, and why there's little impetus to fix it.
- cheaprentalyeti 9y ago
  TKL? Is that something new?
nicpottier 9y ago
Would love if there was another option between those two as well. I know lots of people love Lua but I find it different enough that using it as an embedded scripting language is annoying. IE, I'm not programming in it all the time, so things like 1-indexing of arrays never becomes feels natural and I make indexing mistakes. Other differences like arrays and dictionaries being the same type cause other context switches which I dislike. (and can cause super subtle bugs in JSON marshaling and unmarshaling if you aren't careful) I would really love a fast, lightweight stripped down "C Flavored Syntax" dynamic language to use instead of JS or Lua but I haven't found either yet.
- acqq 9y ago
  I have almost the same feeling as you: I'd love most of the Lua more with zero indexing and more of C like syntax (without these "end" literals, I really like C math-formula-like minimalism). I don't care however if arrays and dicts are the same behind the scenes.
  - nicpottier 9y ago
    Ya the weird case we ran into with the dict / list was when unmarshaling some JSON which had an empty list `[]` then marshaling it back to JSON you end up with `{}`. Which is pretty broken and given the prevalence of JSON as an on the wire format is seriously disappointing. Not sure if there's an easy fix to that apart from using `null` instead of empty lists.
- tangent128 9y ago
  Have you encountered Squirrel yet? It seems to be targeting a "C-flavored Lua" niche: http://www.squirrel-lang.org/ http://www.squirrel-lang.org/

by bpizzi 9y ago

Derbasti 9y ago

Lua is incredibly easy to embed. The whole Lua interpreter including the standard library is about 150kb. That's Kilo, not mega! You can swap out the memory allocator or the number type on one simple header file. Lua was built specifically for embedding! What is more, it is a joyously well designed language that is well suited for building DSLs. And it is pretty fast and memory efficient for a dynamic language. Other options are TKL, but the syntax is a bit unorthodox. Or Python, but it is much bigger, and harder to embed. JS is not a great fit, I would say, mostly because it is comparatively HUGE and not particularly optimized for embedding.

hnlmorg 9y ago
Perl 5 is another option. There's a few embedded Perl libraries out there. But Perl has a bad reputation (undeservingly in my personal opinion) so potentially another hard sell for enterprise.
- hnlmorg 9y ago
  I guess it's too much to ask for whoever downvoted my comment to explain why they did? Otherwise it's going to look little more than a knee-jerk reaction from some Perl hater.
  - Yaggo 9y ago
    I didn't downvote you, but I'd like to know what makes Perl a good option for embedded language and what implementations are available.
    - hnlmorg 9y ago
      Like with Python, Javascript, etc Perl has got a huge and collaborative eco-system (CPAN is the godfather (in terms of age) of npm / pip / et al). It's also terse enough that writing scripts doesn't become a bigger job than it needs to be. The language also offers better type safety than Javascript and most sysadmins are already proficient in Perl because, until recently, it used to be the goto scripting language for servers (these days it seems to be more Python) so Perl makes some sense for enterprise systems. There are big drawbacks for using Perl though. It's appallingly bad for writing OO code (objects in Perl are really just hashes / maps) so larger code bases or complex code can quickly become a pain to maintain. While it has one of the better approaches for comparing data of different types (imo), Perl still has more than it's fair share of hidden traps. Ultimately though, it's a better language than many give it credit for even if it is considered old fashioned these days. So if languages like Python and Javascript are to be considered then I think Perl deserves a mention as well. As for implementations, the standard Perl interpreter can be linked to which makes it very easy to embed into other programs.
  - wahern 9y ago
    I didn't downvote you, but IME Perl is a poor language for embedding. Perl XS is itself pretty bad, whether embedding a C application with Perl, extending a Perl application with C, and especially if you're doing C->Perl->C->Perl mixes--the Perl XS interfaces quickly become leaky abstractions. And don't even get me started on the build process.... I've written far more than my fair share of Perl XS code over the past nearly 15 years. I still use Perl and the last time I wrote Perl XS module was just last month. I could go on for pages about the negative aspects, without having much to say about the positive aspects. But here's a very concrete reason why nobody should contemplate embedding Perl: I filed this bug report (Thread race in dist/IO/IO.xs) over two years ago: https://rt.perl.org/Public/Bug/Display.html?id=125685 It's still not fixed. The fix isn't easy, either. It was introduced in an attempt to fix some other stuff. But maybe the biggest reason it hasn't yet been fixed is because the bug is only likely to be encountered when creating multiple Perl interpreter instances in separate threads--a very uncommon usage for Perl.[1] Uncommon usages will not be (and are not) well supported. Non-threaded Perl embedding may be more common, but ultimately you don't want to be embedding any interpreter that has shared state, especially if the fact of this shared state leaks into interfaces or breaks the interface contracts. Python is also still problematic in terms of not being able to completely erase vestigial assumptions of one interpreter per process. Neither Perl nor Python were originally designed for embedding, and even their extension APIs were bolted on. While they've both been improved over the years and work well for straight-forward C bindings, the fact that neither the languages nor implementations were originally designed for embedding eventually comes through and cause headaches when you want to do anything sophisticated--sometimes even when you're not doing anything sophisticated. [1] The only significant use of Perl in the open source world that I'm familiar with that embeds Perl this way (multiple instances, multiple threads) is Apache mod_perl. But for other reasons mod_perl creates and initializes all Perl instances from a single thread. This is why the bug went undiscovered for so long after it was introduced, and why there's little impetus to fix it.
cheaprentalyeti 9y ago
TKL? Is that something new?

nicpottier 9y ago

Would love if there was another option between those two as well. I know lots of people love Lua but I find it different enough that using it as an embedded scripting language is annoying. IE, I'm not programming in it all the time, so things like 1-indexing of arrays never becomes feels natural and I make indexing mistakes. Other differences like arrays and dictionaries being the same type cause other context switches which I dislike. (and can cause super subtle bugs in JSON marshaling and unmarshaling if you aren't careful) I would really love a fast, lightweight stripped down "C Flavored Syntax" dynamic language to use instead of JS or Lua but I haven't found either yet.

acqq 9y ago
I have almost the same feeling as you: I'd love most of the Lua more with zero indexing and more of C like syntax (without these "end" literals, I really like C math-formula-like minimalism). I don't care however if arrays and dicts are the same behind the scenes.
- nicpottier 9y ago
  Ya the weird case we ran into with the dict / list was when unmarshaling some JSON which had an empty list `[]` then marshaling it back to JSON you end up with `{}`. Which is pretty broken and given the prevalence of JSON as an on the wire format is seriously disappointing. Not sure if there's an easy fix to that apart from using `null` instead of empty lists.
tangent128 9y ago
Have you encountered Squirrel yet? It seems to be targeting a "C-flavored Lua" niche: http://www.squirrel-lang.org/ http://www.squirrel-lang.org/