6 ms·

It's sort of amazing that the diatribe here is against Oracle for wanting to remove an undocumented internal API because of typesafety, and not against the open

It's sort of amazing that the diatribe here is against Oracle for wanting to remove an undocumented internal API because of typesafety, and not against the open source projects that use a specifically unsafe, undocumented internal API. I mean, nothing against these projects -- I understand that type safety can carry a performance penalty -- but the anger of this article is amazing.

valarauca1 11y ago
The issues is 2 fold for people. 1) People use Apache Storm and Cassandra daily. These need Unsafe to function at current performance changing its functionality may change these tools. This can be considered existiential threat to their job. 2) The one big selling point of the JVM is that JVM 1.0 code will run in JVM 8.0 without issue, and often much faster. With this change that isn't necessarily true, and potientally when your .jar was compile will weigh on which JVM is used or HOW that jvm is started.
- geofft 11y ago
  1. Have the maintainers of Storm, Cassandra, etc. said whether they architecturally need Unsafe in its current form, and whether the replacement APIs under development will fulfill their needs? There's a lot of software out there that uses MD5 or DES right now. Saying that the software needs MD5 or DES is a different claim entirely.
  - valarauca1 11y ago
    >Have the maintainers of Storm, Cassandra, etc. said whether they architecturally need Unsafe in its current form [...] ? As there is no alternative to unsafe serialization in terms of performance. It goes without saying. >There's a lot of software out there that uses MD5 or DES right now. Saying that the software needs MD5 or DES is a different claim entirely. I agree, but you are putting the trees before the forest. Saying implementation X is flawed is one thing. Saying We are removing implementation X and offering no alternatives is another. While there are plan to offer an alternative, they are simply plans not an immidate alternative, thus a problem. When DES/MD5 were removed it was clear that they were already inferior implementations, and superior alternatives existed. That isn't true for java.
    - geofft 11y ago
      > As there is no alternative to unsafe serialization in terms of performance. It goes without saying. I don't think that's true, though perhaps I don't understand what you mean by "unsafe serialization". First, I suspect one can use JNI and get at least as good performance. There are other tradeoffs, like the ease of packaging, but in order to evaluate those we need a clear problem statement. Second, premature optimization is the root of all evil, and we should be looking at realistic benchmarks to determine that there is a performance gain. There are many documented examples of software safety checks adding negligible performance overhead because they get branch-predicted away. Alternatively, new hardware support like Intel's MPX adds bounds-checking at the instruction level, so if we're talking about raw writes to a bounds-checked buffer (which is a vast improvement over raw writes to anywhere), this can probably be implemented efficiently on new hardware. On old hardware, the JVM could implement these as unchecked writes, which preserves the exact same performance, but the API would now have information to add safety where it is performant. Finally, sun.misc.Unsafe covers a lot of ground. If we can restrict it to the specific uses that these projects make of it, that's still an improvement. I don't think these libraries use the whole thing, do they? Very few things in science go without saying, and software performance and correctness is science.
      - valarauca1 11y ago
        Seeing as you've admitted you don't understand what I'm talking about. Here is some reading material: http://www.javacodegeeks.com/2010/07/java-best-practices-high-performance.html http://www.javacodegeeks.com/2010/07/java-best-practices-hig...
        geofft 11y ago
        Thanks, I definitely understand this topic imperfectly -- I am familiar with high-performance serialization, but I'm not familiar with Java or best practices with doing this sort of thing in Java. So it's definitely helpful for you to be clear in what you're discussing and what Unsafe is currently used for. Is the specific Unsafe use here to serialize a Java object into a byte array? The linked article doesn't describe using Unsafe, so I'm still not sure what functions are at issue. This is what I was referring to about a bounds-checked buffer in my parent comment: it seems like you could state that accesses to an area of memory are unsafe and unchecked, except to check that the reads/writes are within the buffer. This preserves safety for the JVM as a whole, but gets you native performance within the buffer. Intel MPX should be able to implement this efficiently, and the standard techniques in other languages for efficient bounds-checked memory access should all apply. Am I completely off-base here?
- vbezhenar 11y ago
  About #2: there are many internal changes even between minor releases and if you relying on those internals, your code is NOT portable.
  - eropple 11y ago
    sun.misc.Unsafe effectively became a non-internal API a long time ago. Yeah, it's flagged internal, but it doesn't get broken without a damned good reason (I can't remember the last time it meaningfully broke, even).
- syjer 11y ago
  1. it's quite likely that the Unsafe classes will be accessible with a flag, so for this kind of projects it's not really a big problem, as they provide the shell scripts for launching the jvm with all the necessary parameters.
  - deleted 11y ago
    [deleted]
    - kasey_junk 11y ago
      If you are using Unsafe in your code you have already decided that you know what jvm you are deploying to.
      - gizmo385 11y ago
        That's not necessarily true. What about for projects which are used across a wide array of platforms such as Spring or Gson? Those frameworks are meant to be deployed independently of platform.
    - jlarocco 11y ago
      > The problem is for more then a decade companies were told this problem would never exist. That's exactly opposite of the truth. The Sun and Oracle documentation has always said (back to at least the Java 1.1 days) that internal APIs could change or disappear at any time, and explicitly said not to use classes in sun.* unless you wanted to fix your code with every new JDK release.
- mbreese 11y ago
  On #2 - Java 1 code using the public API (java.*) will run in Java 8. Using internal APIs was never considered forward compatible.

valarauca1 11y ago

The issues is 2 fold for people. 1) People use Apache Storm and Cassandra daily. These need Unsafe to function at current performance changing its functionality may change these tools. This can be considered existiential threat to their job. 2) The one big selling point of the JVM is that JVM 1.0 code will run in JVM 8.0 without issue, and often much faster. With this change that isn't necessarily true, and potientally when your .jar was compile will weigh on which JVM is used or HOW that jvm is started.

geofft 11y ago
1. Have the maintainers of Storm, Cassandra, etc. said whether they architecturally need Unsafe in its current form, and whether the replacement APIs under development will fulfill their needs? There's a lot of software out there that uses MD5 or DES right now. Saying that the software needs MD5 or DES is a different claim entirely.
- valarauca1 11y ago
  >Have the maintainers of Storm, Cassandra, etc. said whether they architecturally need Unsafe in its current form [...] ? As there is no alternative to unsafe serialization in terms of performance. It goes without saying. >There's a lot of software out there that uses MD5 or DES right now. Saying that the software needs MD5 or DES is a different claim entirely. I agree, but you are putting the trees before the forest. Saying implementation X is flawed is one thing. Saying We are removing implementation X and offering no alternatives is another. While there are plan to offer an alternative, they are simply plans not an immidate alternative, thus a problem. When DES/MD5 were removed it was clear that they were already inferior implementations, and superior alternatives existed. That isn't true for java.
  - geofft 11y ago
    > As there is no alternative to unsafe serialization in terms of performance. It goes without saying. I don't think that's true, though perhaps I don't understand what you mean by "unsafe serialization". First, I suspect one can use JNI and get at least as good performance. There are other tradeoffs, like the ease of packaging, but in order to evaluate those we need a clear problem statement. Second, premature optimization is the root of all evil, and we should be looking at realistic benchmarks to determine that there is a performance gain. There are many documented examples of software safety checks adding negligible performance overhead because they get branch-predicted away. Alternatively, new hardware support like Intel's MPX adds bounds-checking at the instruction level, so if we're talking about raw writes to a bounds-checked buffer (which is a vast improvement over raw writes to anywhere), this can probably be implemented efficiently on new hardware. On old hardware, the JVM could implement these as unchecked writes, which preserves the exact same performance, but the API would now have information to add safety where it is performant. Finally, sun.misc.Unsafe covers a lot of ground. If we can restrict it to the specific uses that these projects make of it, that's still an improvement. I don't think these libraries use the whole thing, do they? Very few things in science go without saying, and software performance and correctness is science.
    - valarauca1 11y ago
      Seeing as you've admitted you don't understand what I'm talking about. Here is some reading material: http://www.javacodegeeks.com/2010/07/java-best-practices-high-performance.html http://www.javacodegeeks.com/2010/07/java-best-practices-hig...
      - geofft 11y ago
        Thanks, I definitely understand this topic imperfectly -- I am familiar with high-performance serialization, but I'm not familiar with Java or best practices with doing this sort of thing in Java. So it's definitely helpful for you to be clear in what you're discussing and what Unsafe is currently used for. Is the specific Unsafe use here to serialize a Java object into a byte array? The linked article doesn't describe using Unsafe, so I'm still not sure what functions are at issue. This is what I was referring to about a bounds-checked buffer in my parent comment: it seems like you could state that accesses to an area of memory are unsafe and unchecked, except to check that the reads/writes are within the buffer. This preserves safety for the JVM as a whole, but gets you native performance within the buffer. Intel MPX should be able to implement this efficiently, and the standard techniques in other languages for efficient bounds-checked memory access should all apply. Am I completely off-base here?
vbezhenar 11y ago
About #2: there are many internal changes even between minor releases and if you relying on those internals, your code is NOT portable.
- eropple 11y ago
  sun.misc.Unsafe effectively became a non-internal API a long time ago. Yeah, it's flagged internal, but it doesn't get broken without a damned good reason (I can't remember the last time it meaningfully broke, even).
syjer 11y ago
1. it's quite likely that the Unsafe classes will be accessible with a flag, so for this kind of projects it's not really a big problem, as they provide the shell scripts for launching the jvm with all the necessary parameters.
- deleted 11y ago
  [deleted]
  - kasey_junk 11y ago
    If you are using Unsafe in your code you have already decided that you know what jvm you are deploying to.
    - gizmo385 11y ago
      That's not necessarily true. What about for projects which are used across a wide array of platforms such as Spring or Gson? Those frameworks are meant to be deployed independently of platform.
  - jlarocco 11y ago
    > The problem is for more then a decade companies were told this problem would never exist. That's exactly opposite of the truth. The Sun and Oracle documentation has always said (back to at least the Java 1.1 days) that internal APIs could change or disappear at any time, and explicitly said not to use classes in sun.* unless you wanted to fix your code with every new JDK release.
mbreese 11y ago
On #2 - Java 1 code using the public API (java.*) will run in Java 8. Using internal APIs was never considered forward compatible.