Hacker News Search

fwsgonzo 2y ago

You're right! I actually did try to integrate a compiler, but I quickly realized that it wasn't worth it for me, personally. Iterating on the script is very quick right now. It's mostly a matter of compiler flags and ccache, believe it or not. And I can use mold for linking RISC-V!

boffinAudio 2y ago
Its great work, and you've inspired me to put libriscv in places where I usually put the LuaVM, but I wish you'd update your article to s/Scripting/Dynamic modification of resources at runtime/ .. or something. We can see, already here in this thread, the dangers of people mis-interpreting your technology. Now, if you get the riscv compiler integrated into your projects such that we can indeed just load a text file containing C code, dynamically at runtime, (a la shaders), I'll revisit all of this and go along with your program. ;) Anyway, thanks for the great series of articles - really gave me an interesting read on the way to work this morning, and I've put libriscv (as well as your sample projects) in my stack of Lab Todo's for the week .. now if someone produces LuaRISCV that targets the riscv VM instead of Lua Bytecode, that's gonna break a few molds, in and of itself .. ;) (In the context of your work so far, this seems like a low-hanging fruit, actually..)

exDM69 2y ago

> Out-of-the-box, Lua runs through an interpreter - not a compiler. The first thing the "interpreter" does is run the Lua code through `luac`, the Lua compiler and then feed the bytecode to the Lua VM. This may be transparent to the user, but it's still there. Almost all interpreted languages have a compiler and a bytecode interpreter as distinct steps. > Is loading and parsing a .PNG resource that appears in your executive bundle somewhere, also considered "Scripting" in your opinion? Of course it's not, an image file does not contain any code that could be interpreted or executed in a host environment. There is a distinction here on how code is executed, whether by compiling, interpreting, JITting or a combination of the above. But if you draw the line for "scripting" at consuming textual source code, that would exclude most use of Lua in gaming engines, as it is typically distributed as bytecode and may not even link luac compiler to the host executable. Yet it's commonly called "scripting" in the industry. And conversely, if the code in this article included a mechanism to compile the source to risc-v would it meet your definition of "scripting"? There isn't a clear definition of what's scripting and what's not, but drawing the line at consuming textual source code is not, in my opinion, a useful distinction (it's certainly a distinction) because it hardly matches what's the state of the art out there.

boffinAudio 2y ago
For decades, compiling has referred to the process of turning human-readable language into native machine code. That is what is happening here. Interpreting has, also for decades, referred to the run-time interpretation of human-readable language into some mechanism which either a) immediately results in native execution at runtime, or b) produces errors reported to the developer for fixing, precluding the generation of immutable machine code being executed natively by the CPU (except of course with JIT, which is a form of compilation at runtime, since the interim bytecode is translated into real machine code instructions...) Scripting has, for decades - not just in the gaming industry, which is a subset of computing enterprises - meant "loading a human readable text file containing a programming language and directly executing it if it passes validation - reporting errors, otherwise, preventing further execution. This is why we have BASH scripts and Python scripts and Lua scripts - they are not machine code, they are interpreted and either eventually produce machine code, or run through a state device pretending to be a non-native machine. fwsgonzo is literally compiling C code into machine code to be executed on a pretend CPU (in the form of an embedded libriscv). The code is not interpreted (from one language to another) - it is executed in a virtual machine environment. Bytecode is, on the other hand, always interpreted from one form to another prior to execution (except JIT, it is only interpreted once, compiled, and then exists as machine code always) fwsgonzo's binary blob is executed by a 'fake' CPU, it is not interpreted. If he had the execution environment set up, he could literally run that very same machine code on real RISCV hardware, unchanged, without translation or interpretation. It is not, therefore, interpreted in any sense other than by his embedded RISCV emulator - and thus it is not scripting! This is the point where the distinction is important. >The first thing the "interpreter" does is run the Lua code through `luac`, The first thing the Lua interpreter does is validate the inbound, human-readable script to make sure the program is correct, syntactically and otherwise - giving errors to the developer if errors are encountered. Meaning, at runtime, errors can be caught and fixed by the human developer. Only then, once it has been validated, is it processed into an interim representation as bytecode. Sure, Lua can be told to produce bytecode in order to speed up runtime loading and execution, but its still an interpreted bytecode. It hasn't been compiled into a native form for direct execution - it still requires a secondary native program (a VM host) in order to provide any functionality. From the Lua 5.1 manual, Section 2.4.1: ... Chunks can also be pre-compiled into binary form; see program luac for details. Programs in source and compiled forms are interchangeable; Lua automatically detects the file type and acts accordingly. Lua does have a compiler for the case where you want to create Lua bytecode (.luac), as a loadable-at-runtime resource to be further interpreted and also, the bytecode is subsequently interpreted according to the needs of the target (native) environment on which it is being executed, whether through a JIT or otherwise. That same bytecode can be executed on vastly different native CPU architectures - through interpretation. The difference is, prior to Lua's internal compilation of the script into bytecode, validation of the programs correctness is performed, preventing the developer from producing incorrect bytecode if there are errors, and this is the process of scripting - distinct from compilation - because it is a more direct human/computer interaction regarding the correctness of the script language as it is being used by the developer. >There isn't a clear definition of what's scripting and what's not, but drawing the line at consuming textual source code is not, in my opinion, a useful distinction (it's certainly a distinction) because it hardly matches what's the state of the art out there. I disagree with you, there very definitely IS a clear definition of what is scripting and what is not - one must not call it 'scripting' unless runtime loading, validation (or not), interpreting, and subsequent execution (whether on an interim bytecode or otherwise) is happening - and that is not what is happening here in fwsgonzo's project. He is compiling binary blobs, and his eventual execution environment has nothing to do with the validation of the scripted code. The C compiler he uses in a prior step does, however. Scripting is a development workflow where compilation/linking/building is not required - results are immediate, the program reports validity of the loaded script, or otherwise, and executes directly upon command by the developer. Additionally, fwsgonzo has also, already confirmed that this distinction is correct (see elsewhere in the thread) - that in fact interpreting of a script is NOT happening in any sense other than the RISCV VM is interpreting pre-validated machine code (not bytecode!) in the form of RISCV instructions. That machine code is not a human-readable script, and it is not bytecode! It is machine code, produced by a compiler. This distinction is important because that same machine code could, theoretically, be executed directly, unmodified, on a RISCV machine - that is not the case with Lua bytecode, ever, which always requires interpretation by a VM. fwsgonzo's project is entirely based around pre-compilation of native binary resources, not interpretation. A human-readable language is not being interpreted by his runtime. There is no opportunity (at runtime) with his project to have the runtime report on errors in the human-readable script - this has all been done with a prior compilation step. >but drawing the line at consuming textual source code is not, in my opinion, a useful distinction (it's certainly a distinction) because it hardly matches what's the state of the art out there. I disagree with you. I think you are taking an aberration of the gaming industry, which is rife with such things, and applying it to computing as a whole - and this is where the protest over your position lays. It is absolutely important to make this useful distinction, because in case a) INTERPRETATION: an interim representation is being generated after validating program correctness/conformity to a language spec, and in case b) COMPILATION: no such validation is occurring once the compiler produces runnable machine code. > Yet it's commonly called "scripting" in the industry. I have 40 years of experience with this subject, including working on game engines and realtime (scientific) use of Lua in high-performance environments, and the phrase "scripting" has always referred to the process of writing code that is immediately interpreted by some program before resulting in actual runtime execution if it is found to be valid code. The interpretation may not even work as desired, but the host environment/program will still continue to be executed in a runtime context. The fact that compilers "interpret" code before producing an interim representation that is then used to produce static machine code, is entirely why the distinction must be made: because there is a point where code can be incorrect and thus not runnable! In the game world, "scripting" means 'writing some human-readable script and having it interpreted at runtime, without involving a native compilation/linking/building/packaging step" - and as this is not what is happening (key word: without) in fwsgonzos' project, it is incorrect use of the phrase. He is compiling resources which can be loaded at runtime, bypassing the native loading/linking phase, and it is therefore more appropriate to refer to what he is doing as "runtime resource management" which, incidentally, feeds into a VM. This is not scripting, although scripting often necessitates the same activity - errors in the code are not reported at runtime, by the process doing something with the resource. Please just use the right term and stop trying to justify an incorrect usage - don't re-define these terms based on a misunderstanding of the mechanics involved. There are literally decades of examples which are entirely counter to your misunderstanding out there, in the state of the art. (tl;dr - Is /bin/bash a compiler? Is /bin/python a compiler? Why isn't gcc exclusively referred to as an interpreter? Is clang an interpreter? Is ld a scripting environment?)

indigoabstract 2y ago

> This particular aspect is the only feature that the authors' libriscv usage and the Lua VM have in common - but that doesn't make it scripting. Maybe I'm nitpicking, but to me scripting is just programming/extending a certain fixed piece of software to execute instructions not already programmed into it without having to modify that software itself. Usually using an API. Not unlike computer programming, which aims to make a piece of hardware execute some instructions without needing to alter the said hardware. The main difference is conceptual (the platform targeted: hardware or software). If the script happens to be run by a VM emulating a real processor because your software includes such a thing, I think the distinction becomes purely conceptual. Now, the technology is obviously super cool, but what I don't quite understand yet is what is the best use case for this? Is it really game scripting? Or compiling C++ on the fly? It's not exactly a simple drop in replacement for a Lua interpreter.

It doesn't need emulation of hardware per se, but most scripting environments are already virtual machines of some variety or another - and RISC-V just happens

It doesn't need emulation of hardware per se, but most scripting environments are already virtual machines of some variety or another - and RISC-V just happens