8 ms·
My experience binding a couple of scripting engines with C++
- germandiago 5y agoConstructive feedback for the article is welcome. Thanks!
- Rochus 5y agoDid you have a look at e.g. https://root.cern.ch/root/html534/guides/users-guide/CINT.html https://root.cern.ch/root/html534/guides/users-guide/CINT.ht... and https://root.cern/cling/ https://root.cern/cling/? > Lua ... It has unfamiliar syntax ... start indexing at 1 Syntax is not unfamiliar, just more Pascal like; if you use LuaJIT you can use zero based indices and a powerful FFI for direct C code integration.
- germandiago 5y agoWell, by this I mean "unfamiliar to me", of course. Lol. Actually Lua is something to consider from the point of view of usage: it is an industry standard actually. However, all those small quirks in semantics... and classes can be done in many ways (that is what I understand, via metatables)... In ChaiSCript or Wren there is one true way and you are done. You might like it or not, but it leads to less confusion, especially if you use most of the time what is in the mainstream. This is by no means a bad thing in itself, it is just about how ergonomic or time-consuming it could be for myself: I just feel more comfortable with ChaiScript, Wren or Squirrel than with Lua. Even AngelsCript is also more similar to what you already have. So when exposing APIs there is much less friction. Truth to be told, there is also https://github.com/ThePhD/sol2 https://github.com/ThePhD/sol2 which looks great and something to consider. It makes binding things quite easier and gives you object-oriented Lua. You could rely on that. It was just my subjective choice. There is no 100% right choice. Probably, if I found people that are comfortable with Lua I would use that. But the case is that this is a project of mine as it stands now.
- jcelerier 5y ago> Syntax is not unfamiliar, just more Pascal like; how is that not unfamiliar
- Rochus 5y agoWell, why would you then consider Python to be familiar?
- oblio 5y agoBecause Python is 100000x more popular than Pascal in 2021?
- germandiago 5y agoYou make a good point. I started my Computer Science and Engineering degree (I am european, not american, so the equivalent looks like kind of a merge of both areas) with Python, C and C++ on the programming side. Pascal was discarded a few years back in my university. And yes, by familiar I mean exactly what you mean: you see nowadays Java, Python, C++, C, C#, but Pascal is disappeared. Disappeared since long ago since I do not know even the syntax myself by casual reading around.
- Rochus 5y agoPython syntax is more similar to Pascal than to e.g. Java or JS. "Modula-3 is the origin of the syntax and semantics used for exceptions, and some other Python features." (from https://docs.python.org/3/faq/general.html#why-was-python-created-in-the-first-place https://docs.python.org/3/faq/general.html#why-was-python-cr...). Also the predecessor languages ABC and SETL were in the Algol tradition. > Python is 100000x more popular than Pascal It's about factor 8 on https://www.tiobe.com/tiobe-index/ https://www.tiobe.com/tiobe-index/ or factor 3 (in score) on https://spectrum.ieee.org/static/interactive-the-top-programming-languages-2020 https://spectrum.ieee.org/static/interactive-the-top-program..., whatever you prefer as a reference. Delphi (which is Object Pascal) is still a widely used language.
- 5y ago
- germandiago 5y agoAs for looking at CINT, CLing, yes I did. I prefer to use a dynamic language with coroutines out of the box. It is actually what I was looking for besides ease of binding it and "familiarity" in semantics/syntax in a broad, imprecise way I defined for myself.
- pierrec 5y ago> if you use LuaJIT you can use zero based indices and a powerful FFI Not entirely, Lua standard libraries still expect everything to be one-indexed, while FFI structures are zero indexed. So with LuaJIT you often end up with a mix of 0 and 1 indexed code, which in my experience was workable but definitely a pain point.
- Rochus 5y agoYou should avoid the Lua C API in LuaJIT because it is not supported by the JIT (i.e. it makes your code running in the interpreter instead of the JIT). Using zero based indices in Lua code running on LuaJIT works well.
- pierrec 5y agoI'm not talking about the Lua C API, but things as simple as this: > stuff = {"one", "two", "three"} > stuff[1] one These native Lua structures are baked in, and they're a lot more flexible than FFI structures - presumably if you're using Lua, it's because you want to take advantage of that flexibility and those affordances. FWIW I've written a lot of LuaJIT, and usually I kept the lower-level FFI stuff separate from the higher-level code using Lua data structures, so I rarely encountered that discrepancy between them, but still something to keep in mind.
- Rochus 5y agoYou can e.g. do > stuff = { [0]="one", [1]="two" } > print(stuff[0]) one Works well; I wrote e.g. https://github.com/rochus-keller/Smalltalk#a-smalltalk-80-interpreted-virtual-machine-on-luajit https://github.com/rochus-keller/Smalltalk#a-smalltalk-80-in... that way. EDIT: even this works > stuff = { [0]="one", "two", "three" } > print(stuff[0]) -> one > print(stuff[1]) -> two
- pansa2 5y ago> stuff = { [0]="one", "two", "three" } In this case, is it possible to make iteration start with the element at index 0? Maybe by implementing a custom version of `ipairs`?
- Zababa 5y ago> Prefer dynamic to static typing, since static typing can remove the coding speed: it makes you think about types. I'm wondering what you mean exactly by this. Do you not think about types when programming with a language with dynamic typing?
- germandiago 5y agoNo, I think I expressed myself the wrong way. What I mean is that if you have to annotate all your code with types (like in AngelScript), this will slow you down for two reasons. First, you need to think about types, and second, refactoring is more rigid. If it is optional, it is ok, you can take advantage of it at will (ChaiScript supports types in parameters, but optionally).
- Zababa 5y agoThanks, that clarifies it and makes sense.
- olvy0 5y agoI'm late to the party here, but just wanted to add that AngelScript added support for auto declaration like in C++11 some time ago. https://www.angelcode.com/angelscript/sdk/docs/manual/doc_datatypes_auto.html https://www.angelcode.com/angelscript/sdk/docs/manual/doc_da...
- heinrichhartman 5y ago> [lua] was discarded because of 3. It has unfamiliar syntax, but worse, unfamiliar semantics: no classes, use tables, start indexing at 1 and other oddities, just as being able to call functions with the wrong number of arguments and returning nil on the way. Also, use tables for both hash tables and arrays. It is powerful, do not misunderstand me, and Lua supports good concurrency. It was just not what I was looking for because of the mentioned things. I can fully understand why lua is not a good fit for this case, however, I would like to add some color to the picture. The most powerful way to for C(++) - lua interop, is not the official CAPI but luajit/FFI: https://luajit.org/ext_ffi.html https://luajit.org/ext_ffi.html This allows for allocation of C objects on the heap and on the stack and FAST function calls. Doing the same for C++ is possible but requires some work e.g. http://lua-users.org/lists/lua-l/2011-07/msg00492.html http://lua-users.org/lists/lua-l/2011-07/msg00492.html Furthermore: - unfamiliar syntax -- The syntax is tiny -- and I found nothing unexpected about it. - no classes -- There are many class libraries available for lua. Just pick one. Used penlight classes quite a bit, without running into major issues. - use tables for both hash tables and arrays. -- Yes. This is on the API side, under the hood hashes and arrays are used where appropriate.
- pwdisswordfish8 5y agoAlso, > unfamiliar semantics: no classes, use tables, start indexing at 1 and other oddities, just as being able to call functions with the wrong number of arguments and returning nil on the way Other than 1-based indexing, those semantics should be very familiar from JavaScript.
- germandiago 5y agoIt still feels weird to me to call table an array, a hash table and do classes via metatables in very different ways. For sure it is powerful. But it does not fullfill well the zero-friction I was looking for when integrating. You have to change a bit your mindset when integrating and using this stuff. In Wren you have lists, which are basically arrays and dictionaries, ranges and all things I already know how to use from Python/C++. That said, inheritance is not working smooth and ChaiScript does not even have inheritance itself. But for my purposes a class, concurrency and familiar data structures and patters was enough.
- pansa2 5y ago> Python [...] could be difficult to port to Web Assembly down the road The Pyodide project has already compiled CPython to WebAssembly - why is that a worse solution than compiling one of these other scripting language interpreters to WASM?
- zurn 5y agoIndeed, Python is one of the most well behaved scripting languages for WebAssembly, and people were running in browser for a good while already with WebAsembly predecessors (emscripten and asm.js).
- germandiago 5y agoThis was not my information at the time. But thanks for the info, it is helpful. With https://github.com/pybind/pybind11 https://github.com/pybind/pybind11 there is really great integration with C++ and Python is my second home after C++ actually. Anyway, I am quite happy with Wren and it seems to be fast (not a requirement for my project, though)
- pansa2 5y agoOne issue could be size - CPython's native binary is an order of magnitude larger than Lua's, and the same is probably true when using WASM. Perhaps something like MicroPython could solve that, though.
- fulafel 5y agoYep, that exists: https://github.com/micropython/micropython/tree/master/ports/javascript https://github.com/micropython/micropython/tree/master/ports...
- tyingq 5y agoWhile it "works" python under WASM means downloading a very large interpreter and runtime and waiting quite a long time for it to start up. On my i5 laptop, this demo downloads about 8Mb and takes a couple of seconds to load up: http://karay.me/truepyxel/demo.html http://karay.me/truepyxel/demo.html Lua, by comparison, is very small and has a fast startup under WASM.
- TonyTrapp 5y agoDevelopers working with C-like languages might not like Lua because of its different looks and behaviour. I was in the same situation and started looking into Squirrel because of that. But eventually I went back to Lua, for one important reason: The ecosystem. Lua has a lot of adjacent tools and a huge community. If you want to do something in Squirrel, you will be much more on your own, which can be frustrating. Lots of Lua's quirks can be easily worked around, but lack of a community supporting the language can't. This is especially important if you want to open your scripting API to users of your software.
- germandiago 5y agoI think what you say is true. Anyway, I do not need state-of-the-art technology in my case. Basically I really wanted, and I think (in order of importance, but being the two former points the most important with a big difference), this: 1. Concurrency support 2. Easy to bind into C++ 3. Familiarity, etc. Namely, if only Lua had existed and provided that Sol2 exists and makes it bind it to C++ easy, I would have chosen that. But since Wren + WrenBind17 existed, Wren had more familiar syntax and it was a viable choice, I went for that. I was trying to find the past of least resistance (lower learning curve, easier to bind, concurrency making my code easier, since I am familiar already with most patterns) As for ChaiScript, it was the first thing I took since it was so easy to embed. But it had its own problems: lack of concurrency and it does not point the file and line of errors, which is very painful because it drops your productivity. And scripting... scripting is about productivity, at least that is what I was using it for.
- pwdisswordfish8 5y agoI kind of expected to see QuickJS or Duktape here; if the author considers ‘something like javascript-ey’ to be just fine, he might as well have used JavaScript itself, with all its strengths and faults.
- germandiago 5y agoWell. I think I was a bit inaccurate. When I said javascript-ey what I meant is also familiar. Something like Squirrel and Wren do a good job. The ones you mentioned, as far as I investigated, were not dead-easy to integrate into C++, one of the top requirements. Take into account that I have to expose my own types, not just ints and basic types. The original API was coded in a natural C++ way. APIs that wrap well in that sense could be Sol2 for Lua, Chaiscript, Wrenbind... which integrate with custom types and smart pointers. With other scripting languages and their libraries you need additional work
- pwdisswordfish8 5y agoDuktape’s host API is pretty much a ripoff of Lua’s, and the latter wasn’t rejected on those grounds, so… QuickJS’s API isn’t particularly well-documented, but it’s not hard to find your way around it either if you dig into the source (the engine is very hackable, too; you might even fix some of the language’s design flaws – obligatory wat talk reference – if you’re so inclined). The host API follows the CPython model, with objects represented by pointers and explicit reference counting on the C side. There are some predefined macros to ease defining built-in classes. Some type-level hackery in C++ might ease things even further. I don’t know how much deader-easier you want it.
- germandiago 5y agoTake a look at how pybind11, wrenbind17, sol2 or Chaiscript do it. That is how easy I want it: I can expose custom types and global state easily. I do not want just ints and const char */double. These bindings do from decent to great.
- gigel82 5y agoJavaScript would definitely be my first choice if I had to integrate another scripting language into a native program today (doubly so if the eventual target is web, like the author implies with WebAssembly support). Depending on your needs (small vs. fast, low memory usage vs. full JIT, etc.) you can pick anything from JerryScript / QuickJS all the way to V8. Perhaps the only thing missing is a universal C/C++ API for embedding JavaScript engines that lets you swap out easily to test different trade-offs.
- richardwhiuk 5y agoHow well established is JavaScript FFI support?
- mschuster91 5y agoI do wonder why PHP was not included on that list? It offers an embed SAPI module - which some adventurous person even managed to bind to NodeJS: https://www.npmjs.com/package/php-embed https://www.npmjs.com/package/php-embed Only thing I can imagine is that it's not widely known and even more less documented.
- germandiago 5y agoMy game is not a web-first game. I do not want to put stones on its way to WebAssembly but I was not looking for web-first tech, just a small embeddable language to complement its C++ codebase.
- mingodad 5y agoFor a C/C++ like syntax there is also https://github.com/mingodad/ljs https://github.com/mingodad/ljs and ljsjit and also https://github.com/mingodad/squilu https://github.com/mingodad/squilu a variant of Squirrel with more C/C++/Java/CSharp/Javascript syntax.