4 ms·

Translating a language into a different language is a popular thing to do these days, but still not a very easy one. I feel it's like peeling an infinite onion

by ventana 3mo ago

Translating a language into a different language is a popular thing to do these days, but still not a very easy one. I feel it's like peeling an infinite onion of misery. First, you write a parser of your source language, figure out the translation of the instructions, and emit the code in the target language, and you're very happy when your translated Hello world compiles. Then, a user (like me) tries writing something like package main func main() { register := 42 println(register) } well, oops. /tmp/solod_build3904763637/main.c: In function 'main': /tmp/solod_build3904763637/main.c:6:21: error: expected identifier or '(' before '=' token 6 | so_int register = 42; | ^ /tmp/solod_build3904763637/main.c:7:29: error: expected expression before 'register' 7 | so_println("%" PRIdINT, register); | ^~~~~~~~ (exit status 1) OK, now you grab the list of reserved words of the target language, which is not always an easy thing to do, and rename type names and variables as needed. The next bad thing is when you step on your own toes and see that the new names you invented, like `so_int` or `so_println`, will inevitably pollute the global namespace. We'll either cross our fingers and hope that no one will create a variable named `so_int`, or we'll need to add all our new kind-of-reserved words to our already big list of exceptions. I'm sure there are multiple levels of complexity beyond this. Not trying to say that seeing a bunch of new translators from language A to language B is bad: not at all! It really seems that this is one of the popular usages of the agents, and a rewarding one. But doing it without hidden bugs is kinda hard.

kfreds 3mo ago
That seems like a brittle approach to transpilation. A transpiler should translate all of the language's semantics, including resolving identifiers as symbols, and then choose legal names for them in the target language. Also, there is so much more to a language than its surface syntax. I've never designed a transpiler (I'm not a programmer), but surely the correct approach is to transform the source code into its type-checked AST, and then translate to the target language from there?
- ventana 3mo ago
  > A transpiler should translate all of the language's semantics, including resolving identifiers as symbols, and then choose legal names for them in the target language. Yes, it should; but it also makes things much more difficult. This specific transpiler indeed builds an AST, but does not care about identifiers, just emitting them as is [1] (I hope I found the correct place in the code). [1]: https://github.com/solod-dev/solod/blob/8485bc867ae0f0269d754de063aa43cc3b0b4c1a/internal/clang/expr.go#L394-L413 https://github.com/solod-dev/solod/blob/8485bc867ae0f0269d75...
- jamiejquinn 3mo ago
  I'm really sorry to be harsh but if you don't use programming languages and have never written a transpiler, why should anyone read your comment? You critique the approach as brittle and yet the approach you propose doesn't solve the thorny problem gp described of a growing ball of reserved keywords.
  - dwattttt 3mo ago
    You may have missed this in their comment: > resolving identifiers as symbols, and then choose legal names for them in the target language The responsibility for avoiding collision with the target language is in the transpiler; it should mangle identifiers appropriately to ensure you can't accidentally hit a keyword in the target language.
  - vanderZwan 3mo ago
    They technically didn't say that they don't use programming languages, they said they're not a programmer. There's plenty of other people who use programming languages regularly, but not necessarily because they're into them.
  - lilbigdoot 3mo ago
    I've written multiple transpilers and their comment is how I do it. Safely mapping identifiers is one of those many edge cases you discover as you go along
  - leoedin 3mo ago
aw1621107 3mo ago
> The next bad thing is when you step on your own toes and see that the new names you invented, like `so_int` or `so_println`, will inevitably pollute the global namespace. We'll either cross our fingers and hope that no one will create a variable named `so_int`, or we'll need to add all our new kind-of-reserved words to our already big list of exceptions. To be fair, it's not like it's a completely foreign concept; consider C's categories of reserved identifiers (e.g., no double underscores or underscore followed by a capital letter) as well as the identifier ugilification that needs to be done in the C/C++ stdlibs to avoid collisions with user code. In that vein, the transpiler could pick some prefix which users are highly unlikely to use and document that said prefix is off limits (e.g., "You can't use identifiers starting with `_solod_id_`") or maybe it could use C's reserved IDs (idk if such a transpiler would count as an "implementation" as far as the C standard is concerned).
lou1306 3mo ago
Oof that's not a good look. You can trivially avoid this by just prefixing all variable names with a salt. By the way this (or a variant thereof) is called "avoiding unwanted name capture" if you want to sound all sophisticated :)
- bruce343434 3mo ago
  C++ compilers do this too, thats why so many to-be-linked symbols start with "Z"
  - ventana 3mo ago
    Not only for the conflicts, but mostly to encode the parameter types into the name for method overloading, because the linker has no idea about it.
- ventana 3mo ago
  I think maybe the idea of all these transpilers is to keep the generated code readable and as close to the original code as possible. Mangling names into unreadable garbage is not difficult, but that would also make the the transpiler act more like an obfuscator, which is probably what the authors want to avoid. Protobuf has similar problem: whatever names you use for your proto messages should be translated to the target language as is, and with multiple target languages, chances are that you'll hit a keyword in one of them; each language plugin should figure out how to resolve this. $ cat test.proto syntax = "proto3"; package test; message TestMessage { int32 register = 1; int32 foo = 2; } $ protoc --cpp_out=. test.proto $ grep 'register\|foo' test.pb.cc | head -n 2 : register__{0}, foo_{0}, (I may understand `register__`, but why `foo_`? no idea)
sylware 3mo ago
A good test is to try to port significant from c++ to plain and simple C using "AI". Maybe it can retain some of the semantics for the original code. There is a more brutal option: AI could help write a c++ to plain and simple C transpiler... then we would lose the semantics of the original program, but free ourself from the very few toxic and real-life c++ compilers out there.
- Chu4eeno 3mo ago
  > A good test is to try to port significant from c++ to plain and simple C using "AI". Maybe it can retain some of the semantics for the original code. So updating cfront for modern C++.
  - potus_kushner 3mo ago
    it's already happening: https://github.com/rolandpj1968/sea-front https://github.com/rolandpj1968/sea-front
    - sylware 3mo ago
      Sanity and common sense are starting to fight back?
- deleted 3mo ago
  [deleted]
- win311fwg 3mo ago
  > but free ourself from the very few toxic and real-life c++ compilers out there. Funny because the canonical C++ compiler for the first decade of the language's life did compile to C. People eventually moved on to other compilers to free themselves from having to deal with the toxicity of C output. What's old is new again.
  - sylware 3mo ago
    You are an odd one: we all know that c++ syntax complexity is beyond salvation : it has reach such an absurd and grotesque level, we are in mental pathology realm: Rube Goldberg Machine accute syndrome. And it is not even a matter of argumentation, unless being of accutely bad faith. It seems you are failing to see that this is this very syntax complexity which makes most of this toxicity. And having a modern c++ transplier to plain and simple C, would re-open the door to alternative and 'real-life' small/medium compilers, not like those vendor/developer-locked very few options we have today. Somebody did it for microsoft rust. Would just be a good idea for nowadays c++.
benj111 3mo ago
Add prefix 'noso_' to user variables. So even if someone creates 'so_int' it becomes 'noso_so_int'

by ventana 3mo ago

kfreds 3mo ago

That seems like a brittle approach to transpilation. A transpiler should translate all of the language's semantics, including resolving identifiers as symbols, and then choose legal names for them in the target language. Also, there is so much more to a language than its surface syntax. I've never designed a transpiler (I'm not a programmer), but surely the correct approach is to transform the source code into its type-checked AST, and then translate to the target language from there?

ventana 3mo ago
> A transpiler should translate all of the language's semantics, including resolving identifiers as symbols, and then choose legal names for them in the target language. Yes, it should; but it also makes things much more difficult. This specific transpiler indeed builds an AST, but does not care about identifiers, just emitting them as is [1] (I hope I found the correct place in the code). [1]: https://github.com/solod-dev/solod/blob/8485bc867ae0f0269d754de063aa43cc3b0b4c1a/internal/clang/expr.go#L394-L413 https://github.com/solod-dev/solod/blob/8485bc867ae0f0269d75...
jamiejquinn 3mo ago
I'm really sorry to be harsh but if you don't use programming languages and have never written a transpiler, why should anyone read your comment? You critique the approach as brittle and yet the approach you propose doesn't solve the thorny problem gp described of a growing ball of reserved keywords.
- dwattttt 3mo ago
  You may have missed this in their comment: > resolving identifiers as symbols, and then choose legal names for them in the target language The responsibility for avoiding collision with the target language is in the transpiler; it should mangle identifiers appropriately to ensure you can't accidentally hit a keyword in the target language.
- vanderZwan 3mo ago
  They technically didn't say that they don't use programming languages, they said they're not a programmer. There's plenty of other people who use programming languages regularly, but not necessarily because they're into them.
- lilbigdoot 3mo ago
  I've written multiple transpilers and their comment is how I do it. Safely mapping identifiers is one of those many edge cases you discover as you go along
- leoedin 3mo ago

aw1621107 3mo ago

> The next bad thing is when you step on your own toes and see that the new names you invented, like `so_int` or `so_println`, will inevitably pollute the global namespace. We'll either cross our fingers and hope that no one will create a variable named `so_int`, or we'll need to add all our new kind-of-reserved words to our already big list of exceptions. To be fair, it's not like it's a completely foreign concept; consider C's categories of reserved identifiers (e.g., no double underscores or underscore followed by a capital letter) as well as the identifier ugilification that needs to be done in the C/C++ stdlibs to avoid collisions with user code. In that vein, the transpiler could pick some prefix which users are highly unlikely to use and document that said prefix is off limits (e.g., "You can't use identifiers starting with `_solod_id_`") or maybe it could use C's reserved IDs (idk if such a transpiler would count as an "implementation" as far as the C standard is concerned).

lou1306 3mo ago

Oof that's not a good look. You can trivially avoid this by just prefixing all variable names with a salt. By the way this (or a variant thereof) is called "avoiding unwanted name capture" if you want to sound all sophisticated :)

bruce343434 3mo ago
C++ compilers do this too, thats why so many to-be-linked symbols start with "Z"
- ventana 3mo ago
  Not only for the conflicts, but mostly to encode the parameter types into the name for method overloading, because the linker has no idea about it.
ventana 3mo ago
I think maybe the idea of all these transpilers is to keep the generated code readable and as close to the original code as possible. Mangling names into unreadable garbage is not difficult, but that would also make the the transpiler act more like an obfuscator, which is probably what the authors want to avoid. Protobuf has similar problem: whatever names you use for your proto messages should be translated to the target language as is, and with multiple target languages, chances are that you'll hit a keyword in one of them; each language plugin should figure out how to resolve this. $ cat test.proto syntax = "proto3"; package test; message TestMessage { int32 register = 1; int32 foo = 2; } $ protoc --cpp_out=. test.proto $ grep 'register\|foo' test.pb.cc | head -n 2 : register__{0}, foo_{0}, (I may understand `register__`, but why `foo_`? no idea)

sylware 3mo ago

A good test is to try to port significant from c++ to plain and simple C using "AI". Maybe it can retain some of the semantics for the original code. There is a more brutal option: AI could help write a c++ to plain and simple C transpiler... then we would lose the semantics of the original program, but free ourself from the very few toxic and real-life c++ compilers out there.

Chu4eeno 3mo ago
> A good test is to try to port significant from c++ to plain and simple C using "AI". Maybe it can retain some of the semantics for the original code. So updating cfront for modern C++.
- potus_kushner 3mo ago
  it's already happening: https://github.com/rolandpj1968/sea-front https://github.com/rolandpj1968/sea-front
  - sylware 3mo ago
    Sanity and common sense are starting to fight back?
deleted 3mo ago
[deleted]
win311fwg 3mo ago
> but free ourself from the very few toxic and real-life c++ compilers out there. Funny because the canonical C++ compiler for the first decade of the language's life did compile to C. People eventually moved on to other compilers to free themselves from having to deal with the toxicity of C output. What's old is new again.
- sylware 3mo ago
  You are an odd one: we all know that c++ syntax complexity is beyond salvation : it has reach such an absurd and grotesque level, we are in mental pathology realm: Rube Goldberg Machine accute syndrome. And it is not even a matter of argumentation, unless being of accutely bad faith. It seems you are failing to see that this is this very syntax complexity which makes most of this toxicity. And having a modern c++ transplier to plain and simple C, would re-open the door to alternative and 'real-life' small/medium compilers, not like those vendor/developer-locked very few options we have today. Somebody did it for microsoft rust. Would just be a good idea for nowadays c++.

benj111 3mo ago

Add prefix 'noso_' to user variables. So even if someone creates 'so_int' it becomes 'noso_so_int'