3 ms·

I'm genuinely curious on how well this is working, is there an independent Java test suite that covers major Java 5/6 features that can verify that the JOPA com

I'm genuinely curious on how well this is working, is there an independent Java test suite that covers major Java 5/6 features that can verify that the JOPA compiler works per the spec? I.e. I see that Claude has wrote a few tests in it's commits, but it would be wonderful if there's a non-Clauded independent test suite (probably from other Java implementations?) that tracks progress. I do feel that that is pretty much needed to claim that Claude is adding features to match the Java spec.

pshirshov 11mo ago
Well, it's complicated. The original jdk compliance tests are notoriously hard to deal with. Currently I parse nearly 100% of positive testcases from JDK 7 test suite (in one of Java 7 branches) but I only have several dozens of true end to end tests (build .java with jopa, validate classfile with javap, run classfile with javac). So, I can't tell how good it actually is but it definitely handles reasonably complex source files with generics (something the original compiler was unable to do). The actual goal of the project is to be able to build at least ANT to simplify clean bootstrap of OpenJDK.
- AtlasBarfed 11mo ago
  That is perilously close to the usual: "AI DID EVERYTHING IN A DAY" "How do you know it works?" "... it just looks like it does" Like when I ask AIs to port sed to java, and it writes test cases ... running sed on a CLI and doesn't implement the full lang spec no matter how much prompting I give it.
  - pshirshov 11mo ago
    Well, at least the emitted bytecode validates with javap and a lot of stuff definitely runs on real jvm.
    - th0ma5 11mo ago
      I think the criticisms are too often dismissed as moving the goalposts or ignorant of potential, but short of recreating the active open bugs in Java, you've created a different thing whose differences have to be managed and it is unclear how helpful that may be despite the working implementations of subsets.
      - pshirshov 11mo ago
        If I (or someone else) can use it as a start point in bootstrap process - that's fine with me. This is not supposed to be a top-tier compiler. Essentially, it needs to be able to build ANT.

pshirshov 11mo ago

Well, it's complicated. The original jdk compliance tests are notoriously hard to deal with. Currently I parse nearly 100% of positive testcases from JDK 7 test suite (in one of Java 7 branches) but I only have several dozens of true end to end tests (build .java with jopa, validate classfile with javap, run classfile with javac). So, I can't tell how good it actually is but it definitely handles reasonably complex source files with generics (something the original compiler was unable to do). The actual goal of the project is to be able to build at least ANT to simplify clean bootstrap of OpenJDK.

AtlasBarfed 11mo ago
That is perilously close to the usual: "AI DID EVERYTHING IN A DAY" "How do you know it works?" "... it just looks like it does" Like when I ask AIs to port sed to java, and it writes test cases ... running sed on a CLI and doesn't implement the full lang spec no matter how much prompting I give it.
- pshirshov 11mo ago
  Well, at least the emitted bytecode validates with javap and a lot of stuff definitely runs on real jvm.
  - th0ma5 11mo ago
    I think the criticisms are too often dismissed as moving the goalposts or ignorant of potential, but short of recreating the active open bugs in Java, you've created a different thing whose differences have to be managed and it is unclear how helpful that may be despite the working implementations of subsets.
    - pshirshov 11mo ago
      If I (or someone else) can use it as a start point in bootstrap process - that's fine with me. This is not supposed to be a top-tier compiler. Essentially, it needs to be able to build ANT.