6 ms·

This lays groundwork for opt-in safety per file, but safety should affect API design. Once you get done doing this to every file and refactoring every public AP

This lays groundwork for opt-in safety per file, but safety should affect API design. Once you get done doing this to every file and refactoring every public API, have you really spent less effort than would have gone into a rewrite? If you only get halfway there, is the new version going to blow up any less often?

pjmlp 4y ago
Depends if the goal is to improve the safety of existing C++ code, or fully rewrite into something else. Until the likes of LLVM, GCC, CUDA, V8 and co get rewritten into something else, improving existing C++ codebases is still an issue.
fidgewidge 4y ago
Kotlin managed it in some cases with clever forwards compatible type system tricks, so it's not impossible
- pjmlp 4y ago
  Except it does so by compiling into the same bytecode as Java, javap will just output Java like code from class files, and some stuff like co-routines cannot be called from existing Java code without wrappers that setup the runtime semantics as expected by Kotlin libraries.
  - fidgewidge 4y ago
    Yes, but so? When they can implement a feature in a forwards compatible way they do, when they can't, they don't and it's caveated as such. That's why nullable types and other Kotlin specific type stuff are encoded using annotations. It's the same strategy here. Carbon is (was?) also intended to compile to the C++ ABI and so does Circle, that's why it says it doesn't run on Windows (commercially a huge error. many of the biggest C++ codebases that could benefit are running on Windows like game engines, and Windows doesn't impose a specific C++ ABI anyway). But there's no way to take existing code files and switch them to the new language in Carbon, at least not yet. Kotlin was carefully designed at every step of the way such that every Java program can be expressed in Kotlin without change, even if that meant compromising on some things. Seems like the Carbon guys are being sucked up by the lure of safetyism and just want to make their own version of Rust that happens to compile to the Itanium C++ ABI.
    - pjmlp 4y ago
      It is only forward compatible, is stuck in decisions when Java 8 was current, there is no roadmap to keep the language in sync wiht the underlying progress of the platform as Kotlin tries to be everywhere. Loom, value types, SIMD, Panama,... are all features that start to be an issue to expose as Kotlin features. It is working on Android, because Google is pushing it as Java 8 => Kotlin, not as Java XYZ <=> Kotlin.
      - fidgewidge 4y ago
        Loom doesn't need to pose any issues for Kotlin. If you target only the JVM, you just use virtual threads directly and ignore the coroutines stuff. If you target other runtimes you use them. If you use a KMP library, just surround stuff marked async with runBlocking {}. SIMD/Panama stuff is just new APIs. No problem there. Kotlin maybe even wins because Panama is super verbose. JVM value types seems never to arrive. Stuck in dev hell for years maybe. If/when it ever does see the light of day, Kotlin has value types and so it can just be compiled to a JVM value type to get the benefits. You'd need to opt in to an ABI break but no big deal. So the binary compatibility part hardly seems an issue even looking forward a long way into the future. It's probably harder for C++ because the language there seems to evolve quicker than Java!
        pjmlp 4y ago
        What Kotlin calls value types aren't the real deal, just syntax sugar for primitive types with method with a single field. KMP taken to the extreme, is better for Kotlin just to do its own thing with Kotlin/Native, except it is so behind that JetBrains is using Rust instead for Fleet services. Java is expected to have language level support for some of those features, will Kotlin catch up?
    - seanbax 4y ago
      I'd like to target Windows. Windows does impose a specific C++ ABI, but it's completely undocumented. It's really just whatever Visual C++ does. C++ ABI is very complex, especially vtable layout and RTTI and EH. I am looking to get Microsoft's assistance on targeting Windows. It's a priority, but they don't want to help yet.
Yoric 4y ago
I agree that safety should affect API design, which means that there is at least one major piece missing after the rewrite. Either the ability to switch to an even stricter type system (e.g. Rust) or a ~zero cost binding generator to a stricter language (again, e.g. Rust). However, before reaching that point, piecewise refactoring is something of considerable value. There are huge C++ codebases lying around that I believe would benefit immensely from a rewrite in a safer language, even if it's not as safe as Rust (or Scala, or F#, etc.) Firefox and Chromium have already paid the entry price for piecewise refactoring into Rust, so they're probably not the best candidates. But there are many other examples that undoubtedly are, e.g. LLVM.
- mjburgess 4y ago
  Scala and F# are safer than Rust since they're on a VM.
  - Yoric 4y ago
    Well, you can compile Rust to WASM. But yes, point taken, out of the box, Scala and F# have a safer execution mode.
  - tialaramex 4y ago
    As far as I understand, both these languages are subject to data races whereas (safe) Rust is not. Now, Scala is a JVM language and so presumably you get Java's rules, you aren't Sequentially Consistent but the damage is constrained - however human programmers still can't successfully reason about this state in non-trivial programs.
    - pjmlp 4y ago
      There are plenty of other races that Rust doesn't provide any help, and those are much more relevant in distributed computing workloads.
  - thesuperbigfrog 4y ago
    >> Scala and F# are safer than Rust since they're on a VM. Do you have any evidence to back up your claim? Both the JVM (Scala's VM) and the CLR (F#'s VM) have a history of vulnerabilities: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=java+hotspot https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=java+hotspo... https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=dotnet+clr https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=dotnet+clr Programming languages running on VM is not a safety guarantee.
gmueckl 4y ago
Your first question ignores the crux of the matter: from scratch rewrites of software that is actually worth rewriting are always failures in the making. While the rewrite is ongoing, the original must undergo continued maintenance and this (a) doubles the workload of the team and (b) presents an ever moving target for the rewrite. This means that the effort that goes into piecemeal improvement is the minimal effort possible that can succeed. Successful piecemeal rewrites will result in gradual improvement, so if you don't see that gradual improvement over time, something is very wrong with the project.

pjmlp 4y ago

Depends if the goal is to improve the safety of existing C++ code, or fully rewrite into something else. Until the likes of LLVM, GCC, CUDA, V8 and co get rewritten into something else, improving existing C++ codebases is still an issue.

fidgewidge 4y ago

Kotlin managed it in some cases with clever forwards compatible type system tricks, so it's not impossible

pjmlp 4y ago
Except it does so by compiling into the same bytecode as Java, javap will just output Java like code from class files, and some stuff like co-routines cannot be called from existing Java code without wrappers that setup the runtime semantics as expected by Kotlin libraries.
- fidgewidge 4y ago
  Yes, but so? When they can implement a feature in a forwards compatible way they do, when they can't, they don't and it's caveated as such. That's why nullable types and other Kotlin specific type stuff are encoded using annotations. It's the same strategy here. Carbon is (was?) also intended to compile to the C++ ABI and so does Circle, that's why it says it doesn't run on Windows (commercially a huge error. many of the biggest C++ codebases that could benefit are running on Windows like game engines, and Windows doesn't impose a specific C++ ABI anyway). But there's no way to take existing code files and switch them to the new language in Carbon, at least not yet. Kotlin was carefully designed at every step of the way such that every Java program can be expressed in Kotlin without change, even if that meant compromising on some things. Seems like the Carbon guys are being sucked up by the lure of safetyism and just want to make their own version of Rust that happens to compile to the Itanium C++ ABI.
  - pjmlp 4y ago
    It is only forward compatible, is stuck in decisions when Java 8 was current, there is no roadmap to keep the language in sync wiht the underlying progress of the platform as Kotlin tries to be everywhere. Loom, value types, SIMD, Panama,... are all features that start to be an issue to expose as Kotlin features. It is working on Android, because Google is pushing it as Java 8 => Kotlin, not as Java XYZ <=> Kotlin.
    - fidgewidge 4y ago
      Loom doesn't need to pose any issues for Kotlin. If you target only the JVM, you just use virtual threads directly and ignore the coroutines stuff. If you target other runtimes you use them. If you use a KMP library, just surround stuff marked async with runBlocking {}. SIMD/Panama stuff is just new APIs. No problem there. Kotlin maybe even wins because Panama is super verbose. JVM value types seems never to arrive. Stuck in dev hell for years maybe. If/when it ever does see the light of day, Kotlin has value types and so it can just be compiled to a JVM value type to get the benefits. You'd need to opt in to an ABI break but no big deal. So the binary compatibility part hardly seems an issue even looking forward a long way into the future. It's probably harder for C++ because the language there seems to evolve quicker than Java!
      - pjmlp 4y ago
        What Kotlin calls value types aren't the real deal, just syntax sugar for primitive types with method with a single field. KMP taken to the extreme, is better for Kotlin just to do its own thing with Kotlin/Native, except it is so behind that JetBrains is using Rust instead for Fleet services. Java is expected to have language level support for some of those features, will Kotlin catch up?
  - seanbax 4y ago
    I'd like to target Windows. Windows does impose a specific C++ ABI, but it's completely undocumented. It's really just whatever Visual C++ does. C++ ABI is very complex, especially vtable layout and RTTI and EH. I am looking to get Microsoft's assistance on targeting Windows. It's a priority, but they don't want to help yet.

Yoric 4y ago

I agree that safety should affect API design, which means that there is at least one major piece missing after the rewrite. Either the ability to switch to an even stricter type system (e.g. Rust) or a ~zero cost binding generator to a stricter language (again, e.g. Rust). However, before reaching that point, piecewise refactoring is something of considerable value. There are huge C++ codebases lying around that I believe would benefit immensely from a rewrite in a safer language, even if it's not as safe as Rust (or Scala, or F#, etc.) Firefox and Chromium have already paid the entry price for piecewise refactoring into Rust, so they're probably not the best candidates. But there are many other examples that undoubtedly are, e.g. LLVM.

mjburgess 4y ago
Scala and F# are safer than Rust since they're on a VM.
- Yoric 4y ago
  Well, you can compile Rust to WASM. But yes, point taken, out of the box, Scala and F# have a safer execution mode.
- tialaramex 4y ago
  As far as I understand, both these languages are subject to data races whereas (safe) Rust is not. Now, Scala is a JVM language and so presumably you get Java's rules, you aren't Sequentially Consistent but the damage is constrained - however human programmers still can't successfully reason about this state in non-trivial programs.
  - pjmlp 4y ago
    There are plenty of other races that Rust doesn't provide any help, and those are much more relevant in distributed computing workloads.
- thesuperbigfrog 4y ago
  >> Scala and F# are safer than Rust since they're on a VM. Do you have any evidence to back up your claim? Both the JVM (Scala's VM) and the CLR (F#'s VM) have a history of vulnerabilities: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=java+hotspot https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=java+hotspo... https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=dotnet+clr https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=dotnet+clr Programming languages running on VM is not a safety guarantee.

gmueckl 4y ago

Your first question ignores the crux of the matter: from scratch rewrites of software that is actually worth rewriting are always failures in the making. While the rewrite is ongoing, the original must undergo continued maintenance and this (a) doubles the workload of the team and (b) presents an ever moving target for the rewrite. This means that the effort that goes into piecemeal improvement is the minimal effort possible that can succeed. Successful piecemeal rewrites will result in gradual improvement, so if you don't see that gradual improvement over time, something is very wrong with the project.