14 ms·

Removal of Unsafe in Java 9 – A disaster in the making

92 points by pdeva1 11y ago

ante_annum 11y ago
So the explanation is: Let me be blunt -- sun.misc.Unsafe must die in a fire. It is -- wait for it -- Unsafe. It must go. Ignore any kind of theoretical rope and start the path to righteousness And what the author takes away is: This engineer hates the Unsafe class for no reason at all. The reason the engineer hates it is that it's unsafe. Disagree with it, sure. But it's a reason, and it's a good reason (imho). I understand that it's commonly used by apps I use, but it's hard to keep reading after this kind of knee-jerk.
- snuxoll 11y ago
  In addition, it's in the sun.misc package, if it's not in java.* or javax.* it is ripe for pruning, you'd think people would have learned by now to stop using internal classes since pretty much every JRE/JDK release known to mankind has changed them or removed them.
  - ante_annum 11y ago
    It's sort of amazing that the diatribe here is against Oracle for wanting to remove an undocumented internal API because of typesafety, and not against the open source projects that use a specifically unsafe, undocumented internal API. I mean, nothing against these projects -- I understand that type safety can carry a performance penalty -- but the anger of this article is amazing.
    - valarauca1 11y ago
      The issues is 2 fold for people. 1) People use Apache Storm and Cassandra daily. These need Unsafe to function at current performance changing its functionality may change these tools. This can be considered existiential threat to their job. 2) The one big selling point of the JVM is that JVM 1.0 code will run in JVM 8.0 without issue, and often much faster. With this change that isn't necessarily true, and potientally when your .jar was compile will weigh on which JVM is used or HOW that jvm is started.
      - geofft 11y ago
        1. Have the maintainers of Storm, Cassandra, etc. said whether they architecturally need Unsafe in its current form, and whether the replacement APIs under development will fulfill their needs? There's a lot of software out there that uses MD5 or DES right now. Saying that the software needs MD5 or DES is a different claim entirely.
        valarauca1 11y ago
        >Have the maintainers of Storm, Cassandra, etc. said whether they architecturally need Unsafe in its current form [...] ? As there is no alternative to unsafe serialization in terms of performance. It goes without saying. >There's a lot of software out there that uses MD5 or DES right now. Saying that the software needs MD5 or DES is a different claim entirely. I agree, but you are putting the trees before the forest. Saying implementation X is flawed is one thing. Saying We are removing implementation X and offering no alternatives is another. While there are plan to offer an alternative, they are simply plans not an immidate alternative, thus a problem. When DES/MD5 were removed it was clear that they were already inferior implementations, and superior alternatives existed. That isn't true for java.
        geofft 11y ago
        > As there is no alternative to unsafe serialization in terms of performance. It goes without saying. I don't think that's true, though perhaps I don't understand what you mean by "unsafe serialization". First, I suspect one can use JNI and get at least as good performance. There are other tradeoffs, like the ease of packaging, but in order to evaluate those we need a clear problem statement. Second, premature optimization is the root of all evil, and we should be looking at realistic benchmarks to determine that there is a performance gain. There are many documented examples of software safety checks adding negligible performance overhead because they get branch-predicted away. Alternatively, new hardware support like Intel's MPX adds bounds-checking at the instruction level, so if we're talking about raw writes to a bounds-checked buffer (which is a vast improvement over raw writes to anywhere), this can probably be implemented efficiently on new hardware. On old hardware, the JVM could implement these as unchecked writes, which preserves the exact same performance, but the API would now have information to add safety where it is performant. Finally, sun.misc.Unsafe covers a lot of ground. If we can restrict it to the specific uses that these projects make of it, that's still an improvement. I don't think these libraries use the whole thing, do they? Very few things in science go without saying, and software performance and correctness is science.
  - kasey_junk 11y ago
    The problem with that, is unlike most of the other internal classes, there is zero other way to do the things people are doing with unsafe without it. It's not a matter of expedience, it's a matter of necessity.
    - dlubarov 11y ago
      Yes, removing Unsafe would make things like PowerMock and Robolectric impossible to (re)implement. So what? Those libraries never should have been possible, and we can live without them. Languages need to be selective about what features they support, and Java was never intended to support things like mocking static methods, or creating instances without invoking any constructors.
      - kasey_junk 11y ago
        Without the current unsafe package most performance sensitive applications (whether throughput, latency, or pause sensitive) will have no recourse to meet their current performance levels. That will leave people to choose between staying on Java 8, worse (and often unacceptable) performance, and migrating to a different language. At this point, there is a very good chance if your library/application is known for performance and exists on the JVM it uses unsafe and without a migration plan all of us who care about performance will likely have to move off the language.
        dlubarov 11y ago
        With some effort, I'm sure the folks behind Cassandra, Hadoop etc. could achieve acceptable performance with allocateDirect(). It doesn't support explicit freeing, but you can always reuse buffers and write your code to allocate slices within them.
      - twic 11y ago
        We can live without Robolectric? How do we write tests for Android code without it?
        11y ago
- taeric 11y ago
  That is a pretty crappy explanation, on the engineer's side. To be clear, any explanation that is not immediately followed with examples to how current valid uses of Unsafe can be accomplished without it, are simply witch hunts by an engineer that hasn't had to meet the same requirements.
  - danudey 11y ago
    It's a) unsafe, and b) an internal, undocumented API. When you start importing from sun.misc.* you've made the conscious (or ignorant) decision to write non-portable code which may well break due to any JVM release. It sucks that these people are going to have to update their code (or that end users won't be able to upgrade to Java 9 without adding special command-line flags), but it's not due to fickle Oracle developers, it's due to technical debt that's finally catching up to them.
    - kasey_junk 11y ago
      You quite simply have NO option to do on the JVM most of the things people are using unsafe for in Java 9/10 without a migration plan from Oracle. What Oracle is close to "hey all you performance sensitive folks, we aren't going to support unsafe anymore, can you tell us what you are using it for and how we can migrate to that explicitly?" It just so happens that one of the leading proposals is "leave in java 9 but make it explicit to turn on". This is probably the right way to do this (in my opinion) but it does have the downside of making neither camp happy (it leaves unsafe so the purists are mad, and it causes headaches for people using unsafe).
      - MrBuddyCasino 11y ago
        Whats wrong with: - deprecate in Java 9 - "make it explicit to turn on" in 10, offer alternative official APIs as fallback - remove entirely when most people have switched
        kasey_junk 11y ago
        Nothing at all. That's what I assume will happen at the end of the day. But people saying that we can just turn it off in 9 because you shouldn't be using internal api's don't seem to get that there is no external solution.
- jevinskie 11y ago
  I think the author's point about being able to make safe Unsafe replacements and keeping Unsafe around (for the time being) to enable better compatibility was thoughtful.
carsongross 11y ago
Once again, The Platonic Engineer gazes hatefully at the gronky, ugly and awkward things that make things work in real life, rub his hands together and says... "Burn it to the ground."
- xg15 11y ago
  Or you could actually stop for a second and find out why your current model needs so many ugly and awkward workarounds and try to adjust it accordingly - which seems to be what Oracle is trying to do in the long term.
  - taeric 11y ago
    This is an odd one, though. I can not remember seeing/hearing any real issues with Unsafe. For the most part, people don't use it. There are a few that do, to ridiculous benefit. As things stand right now, without it, you can not get anything close to the performance people are seeing with it. So, what many of us are seeing here is someone is upset with basically the shape of an API. They have no evidence that I have seen of trouble caused by it. Only a desire that such things not exist. This feels almost puritanical in how it is being exercised. Which is just plain silly for a technical debate.
    - carsongross 11y ago
      puritanical I think the correct term is platonic: each of us has an Inner Platonic Engineer who cannot abide the ugliness of real world systems and insist on The Great Rewrite. A lot of progress relies on that engineer. However, in large, functioning and mature software systems he becomes the enemy.
      - taeric 11y ago
        I question just how much progress relies on that engineer. Be honest here, there are no identified problems that are being solved here. Merely an undesirable in the system that is very effective at its job. Progress, on other hand, advances something in the way of the problem space. Not merely rearranges something in the solution space.
ihuman 11y ago
What is "Unsafe?" Is it unsafe, as its name implies? If it is unsafe, why to so many programs use it?
- dikaiosune 11y ago
  It's part of the OpenJDK/Oracle VM implementation, not an official part of the Java standard library. It allows for all sorts of fun stuff like native compare and swap operations (used for concurrent collections iirc), object allocation without calling the constructor, accessing blocks of memory directly without object references, and other things. The reason it's called unsafe is because it allows developers to bypass a lot of the safety guarantees that java is supposed to provide as a memory managed language. But it's useful for getting more performance out of the VM. There's currently an effort underway to push through a variety of Java enhancement proposals that would add a lot of the unsafe functionality to the official standard library. This would enshrine the unsafe functionality as official, remove access to some of the more dangerous and less useful parts of unsafe, and would also mean that the functionality would be properly documented, which it is currently not.
  - Daneel_ 11y ago
    Thanks for the thorough explanation - you summarised the issue well, as well as explaining the impact and flow on effects. I have to say, I agree with the engineer.. I'm cringing just knowing this sort of practice is seen as acceptable, and it reinforces my distaste for Java. If you have to resort to insecure practices to achieve performance, there's something wrong with the framework.
    - alextgordon 11y ago
      Reading this thread as C programmer has me scratching my head. Of course you have to resort to low-level functionality to achieve performance. That's how computers work. We can't all live in fuzzy wuzzy land. Surely Unsafe is safer than writing the whole thing in C and using JNI?
      - 11y ago
pron 11y ago
Chill. There is no disaster in the making. Oracle is very much aware of the issue, and is actually already taking the very same steps suggested by the author. It has assembled a public working group that is coming up with a spec for a public "Unsafe" API[1] (that is safer than Unsafe). In the meantime, use of Unsafe in Java 9 will be possible with a command-line flag in order to allow the working group more time to create the new API and for Unsafe consumers to migrate, and of course to allow existing libraries to run on Java 9. [1]: https://docs.google.com/document/d/1GDm_cAxYInmoHMor-AkStzWvwE9pw6tnz_CebJQxuUE/edit https://docs.google.com/document/d/1GDm_cAxYInmoHMor-AkStzWv...
- jryan49 11y ago
  The point is if you have to pass a flag to allow the use of Unsafe, you're breaking "works-out-of-the-box" backwards compatibility.
  - pron 11y ago
    "Works-out-of-the-box" backwards compatibility -- without flags -- has never been guaranteed (even the default GC is about to change), let alone for non-public classes. Whenever you compile a class that uses Unsafe you get a warning (that you can't disable!) that says "this class may go away at any time". Still the reality is recognized, and you'll still be able to use Unsafe -- despite all warnings -- and all you need to do is add a flag. I think that's very reasonable.
    - jryan49 11y ago
      I agree in most cases that it's perfectly reasonable. But from a pragmatic position, I think this needs to be treated as a special case because of all the people that are using it. I still like the idea of having a flag that opts people in to using internal classes. It's like an agreement almost that they know what they're doing is not supported across versions.
    - snuxoll 11y ago
      Thankfully, Project Jigsaw (if it lands) in JDK9 should remove the problem of internal classes entirely — they simply won't be available to use.
ZitchDog 11y ago
Entire companies exist around functionality provided in Unsafe. I don't see this going well if they remove it without providing access to the functionality it gives. Java has plenty of problems, but backward compatibility over the years has been simply amazing. This would chip away at one of the few legitimate reasons to use Java in 2015.
- xxpor 11y ago
  sun.* has ALWAYS been exempt from that guarantee though.
  - LoSboccacc 11y ago
    yeah they went the fast and cheap route instead of doing the right thing, and they are now going to be bitten by it, and they can go cry a river for all that I care. those are the sloppy engineers that are holding back evolution of software at large, from little things like obscure functions to huge operating systems like http://blogs.msdn.com/b/oldnewthing/archive/2005/07/28/444390.aspx http://blogs.msdn.com/b/oldnewthing/archive/2005/07/28/44439...
    - Someone1234 11y ago
      You forgot to explain what the "right thing" that they should be doing is? Even according to most articles on the topic there is very to any pleasant alternatives that are cross platform e.g. https://dzone.com/articles/understanding-sunmiscunsafe https://dzone.com/articles/understanding-sunmiscunsafe
      - LoSboccacc 11y ago
        jni, and don't tell me now that c isn't portable.
        Someone1234 11y ago
        C isn't portable. You cannot write it once and run it everywhere, each platform needs a bunch of macros and flags. Plus you'll now either be delivering it in binary form (which may or may not work due to bitness, OS incompatibilities, and so on) or you'll be delivering it as code (which may or may not work due to compiler incompatibilities, build system components available, and missing libraries). C is only portable in the sense that "every platform ever made has a C compiler." But if you've ever written a C application which is just meant to run on Windows, OS X, and Linux you'll know that it is a painful experience (double that if they're meant to compile it themselves).
razorsese 11y ago
Literraly meme language
geofft 11y ago
The quote stops short, and for someone like me who doesn't follow Java very much, it feels like it's misquoting via omission: "Let me be blunt -- sun.misc.Unsafe must die in a fire. It is -- wait for it -- Unsafe. It must go. Ignore any kind of theoretical rope and start the path to righteousness _/now/_. It is still years until the end of public updates to JDK 8, so we have /years /to work this out properly. But sticking our heads in the collective sands and hoping for trivial work arounds to Unsafe is not going to work. If you're using Unsafe, this is the year to explain where the API is broken and get it straight.... "Please help us kill Unsafe, kill Unsafe dead, kill Unsafe right, and do so as quickly as possible to the ultimate benefit of everyone." The quoted engineer understands that people use it, claims that there's no immediate risk (since Java 8 will stay around), and very very strongly implies that there is a plan to move from Unsafe to something better than Unsafe providing equivalent functionality in a stable way, and asks for help with that. Is this correct?
- jryan49 11y ago
  Yes you are.
- BhavdeepSethi 11y ago
  Here is the link to the full quote. http://mail.openjdk.java.net/pipermail/openjfx-dev/2015-April/017028.html http://mail.openjdk.java.net/pipermail/openjfx-dev/2015-Apri...
- pdeva1 11y ago
  The post directly links to the full quote. If the intention was to misquote that would not be the case. > there is a plan to move from Unsafe to something better than Unsafe It is this plan and the implications of it that the post is discussing
  - gclaramunt 11y ago
    Actually I don't see the post discussing anything, is just feels like "Unsafe is going away, the sky will fall!"
  - 11y ago
xg15 11y ago
> Oracle plans to remove sun.misc.Unsafe in Java 9. This is an absolute disaster in the making and has the potential to completely destroy the entire ecosystem around Java. You know your language has a problem if the entire ecosystem threatens to fall apart as soon as you actually try to enforce the language's conceptual model and try to make use of the guarantees it provides.
- bitmapbrother 11y ago
  With the caveat that you need to believe in the hyperbole of a chicken running around with its head cut off.
sbov 11y ago
This seems a bit hyperbolic. Especially the comparison of Python 2 to Python 3. Its not like you could run Python 2 and Python 3 under the same interpreter by passing a flag.
karavelov 11y ago
What about shipping sun.misc.Unsafe as 3rd party package? Is there anything in it that can't be done as through JNI?
- snuxoll 11y ago
  No, you can do everything in Unsafe via JNI, but that comes with all the disadvantages of JNI (including .dll's and .so's is not so easy for a fat-jar deployment).
- vardump 11y ago
  It'd be unusably slow. "sun.misc.Unsafe" compiles in the instruction stream. JNI implementation would need to go through a complicated calling mechanism. I think it'd be up to 100x slower.
  - karavelov 11y ago
    At least in OpenJDK it is in fact implemented as a JNI package, so it is already going through the JNI calling convention: http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/9b8c96f96a0f/src/share/classes/sun/misc/Unsafe.java http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/9b8c96f96a0f/s...
    - twic 11y ago
      That code shows that it's implemented as native methods, but not necessarily as JNI - these could be VM intrinsics, like Object::getClass and so on. This is the native code for Unsafe: https://github.com/openjdk-mirror/jdk7u-hotspot/blob/master/src/share/vm/prims/unsafe.cpp https://github.com/openjdk-mirror/jdk7u-hotspot/blob/master/... It's certainly JNI-ish. Whether it is actually called via the JNI mechanism, i can't tell.
    - vardump 11y ago
      I think these are a strong hint they're not truly JNI: Line 61: * <p> Most methods in this class are very low-level, and correspond to a * small number of hardware instructions (on typical machines). Compilers * are encouraged to optimize these methods accordingly. Line 90: /// peek and poke operations /// (compilers should optimize these to memory ops)
exabrial 11y ago
Document it, make it public. It's very much needed!
jart 11y ago
I just read an article that says in order to even use Unsafe, you have to use reflection to unwind the stack and go searching for a reference to the Unsafe instance, because the Java authors worked so hard to make it private. So it seems to me that they'd be well within their rights to remove the class. Normally when you go to that level of epic hackery, you can't expect for official support or backwards compatibility. So the author of this article using scare tactics to keep that feature seems rather disingenuous.
- orf 11y ago
  Could you provide a link to the article? I'm not saying you're wrong (I'm not a Java guy) but that sounds convoluted.
  - twic 11y ago
    This is how you get access to Unsafe: Field theUnsafe = Unsafe.class.getDeclaredField("theUnsafe"); theUnsafe.setAccessible(true); Unsafe unsafe = (Unsafe) theUnsafe.get(null); Reflection, but no stack unwinding.
kkmickos 11y ago
Many people seem to be upset with Oracle, but I think they should be upset with the vendors who ship applications/libraries that depend on the "sun.*" packages. Oracle (then Sun) have since _1998_ strongly discouraged the use of them: https://web.archive.org/web/19980215011039/http://java.sun.com/products/jdk/faq/faq-sun-packages.html https://web.archive.org/web/19980215011039/http://java.sun.c...
- Someone1234 11y ago
  Fine, but why isn't the argument: "This then needs to be standardised in a Java.* package and correctly documented." Unsafe is useful, and is evident by how popular it is. Heck even Sun/Oracle must agree because they created it for their own internal usage. So why not just go ahead and standardise it.
vbezhenar 11y ago
Why libraries need Unsafe, after all? What problems does it solve for them?
- kasey_junk 11y ago
  It's almost entirely about performance. Unsafe allows you access to concurrent intrinsics as well as memory layout functions. It's true that it is widely used, but the people that use it also understood the ramifications of using it. If you were trying to make jvm agnostic code you didn't use it. If you were trying to make code that could run on a variety of jvm versions, you didn't use it. A lot of what people use it for is being rolled into documented/standard libraries and this is just the migration pains coming out.
  - vbezhenar 11y ago
    Why those concurrent intrinsics are not available with "safe" code? They are not compatible with Java Memory Model? They are not implemented at some platforms? What's so unsafe about concurrent primitives? There is API in java.util.concurrent.atomic, for example. I understand that Unsafe is used for direct access to memory, but why this direct access is needed? May be there are other ways to solve those problems.
    - benjaminjackman 11y ago
      AFAIK there aren't other ways to get the performance required to be competitive in the financial space. People that are using these libraries don't want to use undocumented, Unsafe code. However, their hands are forced given the races they are competing in. They could switch to C++ but that's not really a great option for firms that have java code bases going back 10+ years. Plus, personally, and for a lot of others, the jvm plus a dash of sun.misc.Unsafe, is a lot better than going over to C++. More concretely, Here are two sets of trading libraries that use unsafe for direct memory access: https://github.com/real-logic/simple-binary-encoding https://github.com/real-logic/simple-binary-encoding https://github.com/OpenHFT https://github.com/OpenHFT (various libraries in here use it, once you drink the Unsafe Kool-aid it's hard to turn back from the performance benefits you obtain). If Oracle just rips off the Unsafe band-aid without a replacement, the business decision for these firms is still sound. They will just stay on 8 and start making migration plans (likely to the C++ bandwagon). However, It sounds like Oracle plans to give a sane alternative which is good and what everyone using Unsafe would prefer (along with deprecating it the meantime to give them some time to adapt). One real world example: The CME (Chicago Mercantile Exchange) reworked the entire way they distributed market data (v3 of their Market Data Protocol) to mirror the cutting edge in performance oriented serialization libraries (e.g. Cap'n Proto). They also commissioned an Open Source library for parsing that data (real-logic SBE), it uses unsafe to approach the performance of C++ code.
clamprecht 11y ago
At least he didn't name the article "Unsafe considered harmful."
bowyakka 11y ago
Ok moving past the rhetoric of don't use internal classes, there is a value to sun.misc.Unsafe. It has been well known by Oracle for some time that it has to go away right. Realise that those of us that do use it, are fully aware that it is a hack, and that we should be doing cleaner things. However know that those cleaner things _dont exist_; and that you, Dear application developer might not quite realise some of the shadow puppetry in making your favourite library work, please lay off the bile. As others have said, this blog post stops short, the full quote is part of a very long and ongoing process to rid ourselves of Unsafe, replacing its majors usages with new API's. This is a process Oracle started last year after the previous sun.reflect.Reflection.getCalleeClass mess. Yes, it no one should have been using an internal class. This method did not go away; it was made only available to the JVM internals, locked out from end users. The upshot of this was that, with no sane alternative, suddenly logging became between 2x and 100x more expensive where class names become involved. The JDK team somewhat back peddled on this change, and we are stuck here in a strange limbo land. Was it right of Oracle to do this? Absolutely, everyone who goes down the internal package route is fully aware that these are open-state secrets, sure we all know them, but we should not be talking about them. Support for these damn things requires version to version checking, hideous reflection hacks and being prepared to read a lot of JVM source code for when bugs occur. In light of all of this, and being the general level of awesome the JDK folks are they started a very direct community outreach to remove safely Unsafe. Last year they publicly asked people on the mailing list, and privately via email to do a survey (http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-January/024650.html http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-Ja...) on usages of Unsafe. I know this because I got an email from Paul Sandoz asking me to comment on Unsafe abusages. Out of previous thinking from the JDK folks, as well as from the aforementioned survey, we have a bunch of JEP's for improving things regarding Unsafe. Where things have gone slightly sideways is that we have, currently no unified working group to deal with these changes. What this document is, is a proposal to make a working group, focused on getting the right changes into the JVM. In the same ways as Project Jigsaw is for modularising the JVM (and is indirectly responsible for removing Unsafe), or the MLVM project handled making the VM better support other languages. Unsafe is dead, long live Unsafe! I fully expect that we will achieve the right changes in during Java 9 to allow those of us that need to break the rules, to do it in a supported fashion. I am _looking_ forward to the day when I can do setMemory or getAndSetInt via a supported API, and not one where I have to know the words Unsafe.theUnsafe. The above is a storm in a teacup, its open-source democracy at work.
sreque 11y ago
There's a lot of backlash here against the author with comments like: "You used a sun.* package? You deserve to die in a fire as well!" These comments look like they are coming from people who have never used Unsafe. First, Unsafe provides some immensely useful features to the Java ecosystem. Second, there is no alternative to Unsafe on the JVM. That includes JNI (native code integrated within the JVM). So far, Oracle has stated their intent to remove Unsafe from Java 9. To all the users of Unsafe, Oracle is basically saying, "tell us your use cases for Unsafe, and if we decide they are valid we may try to create an alternative for you. If you're lucky, we may even ship that alternative in Java 9 or 10! Have a nice day!" Many people, myself included, find this unacceptable. For instance, it looks like several groups are currently collaborating on a document discussing uses cases for Unsafe and whether alternative features satisfying those use cases are expected to appear in Java 9: https://docs.google.com/document/d/1GDm_cAxYInmoHMor-AkStzWvwE9pw6tnz_CebJQxuUE/edit?pli=1 https://docs.google.com/document/d/1GDm_cAxYInmoHMor-AkStzWv... As you can see, the majority of the cells in the column "Expected in Java 9" flat out say "no". There is a good reason people are up in arms about the removal of this API. Personally, if I had to choose between Oracle keeping Unsafe and Oracle implementing Java 8 Lambdas, I would have picked Unsafe in a heartbeat. At the end of the day, Lambdas are mostly syntactic sugar, especially in their current implementation. Unsafe, on the other hand, actually empowers you to do something more.
- geofft 11y ago
  That link is a very useful contribution to this discussion (and possibly the first really useful comment in this thread), thanks. Everyone's entitled to their opinion, informed or not, but if you want an informed opinion, the question needs to be what the current uses of Unsafe are, why they're used, and whether there are plans for a migration path. It sort of seems like the major use case (judging from those numbers) is reading and writing raw byte buffers, possibly mapped from other objects/files, with native performance. Is there a potential sensible design for a bounds-checked raw data API ("unsafe but only within these pages") that maintains the JVM's usual safety guarantees for all other data, and can be implemented in a performant way?
  - pdeva1 11y ago
jkot 11y ago
Compression in my library is 3x faster with unsafe. I think it is lesser evil then native code integration.
skrowl 11y ago
Still using Java in 2015 – A disaster in the making
- bitmapbrother 11y ago
  What would a MS fanboy suggest as an alternative?
- goback2reddit 11y ago
  Go back to reddit.
hyperpape 11y ago
Why on Earth is it a big deal that you need to pass a flag to the JVM at startup? I understand that backwards compatibility is valued, but if I went to my CTO and said "we're gonna drop Java 9 in production, and we're not gonna test beforehand"...the reaction would not be pretty.
rebootthesystem 11y ago
This is related and unrelated and maybe even meta to some degree: There are a couple of companies that are guilty of using their free software update mechanism to try and trick people into installing crap they don't want (and might even mess-up their systems): Sun and Adobe. I almost happened to me last night as I updated Flash on a test system and some bullshit anti-virus crap-ware was going to be installed because I was moving fast and neglected to un-check the check-box on the site. I was able to stop the install process and start over. No harm done. These companies need public shaming to stop using this bullshit method to make a few more bucks. Are they so hard-up that they need to trick users to install crap to make money? I sure hope not. Sun, Adobe, please STOP trying to trick users into installing crap-ware they were not looking for in the first place.
- kasey_junk 11y ago
  You understand of course that Sun has been out of business for a very long time right?
  - rebootthesystem 11y ago
    Sorry, brain fart, I meant Oracle
- throwaway2048 11y ago
  java hasnt been bundled with crapware since oracle bought it
  - rebootthesystem 11y ago
    Some installations I've seen have crapware pre-selected. I haven't looked at it lately, maybe they changed this.
scubaguy 11y ago
I read this article yesterday and I really dislike how it is worded. The "this engineer" has a name, Donald Smith, this is what he said: "If you're using Unsafe, this is the year to explain where the API is broken and get it straight.... Please help us kill Unsafe, kill Unsafe dead, kill Unsafe right, and do so as quickly as possible to the ultimate benefit of everyone." It reads like a request to have a discussion on Unsafe, to talk about it and find solutions. Unfortunately, I don't see anyone pointing out why Unsafe should be kept around on that thread (http://mail.openjdk.java.net/pipermail/openjfx-dev/2015-April/017028.html http://mail.openjdk.java.net/pipermail/openjfx-dev/2015-Apri...). And this quote from Oracle shows an effort to do things right: http://mail.openjdk.java.net/pipermail/discuss/2013-October/003162.html http://mail.openjdk.java.net/pipermail/discuss/2013-October/... "This is definitely something we plan to propose for SE 9. I expect to see a JEP from the current maintainers of sun.misc.Unsafe..." This (http://blog.codefx.org/java/dev/how-java-9-and-project-jigsaw-may-break-your-code/ http://blog.codefx.org/java/dev/how-java-9-and-project-jigsa...) seems like a more balanced article on the situation. "So far we focused on the problematic aspects of Project Jigsaw. But that should not divert from the exciting and – I think – very positive nature of the planned changes. After reading the documents, I am impressed with the scope and potential of this upcoming Java release. While it is likely not as groundbreaking for individual developers as Java 8, it is even more so for everyone involved in building and deploying – especially of large monolithic projects." near future. (This is far too small to require a whole JSR.)"

92 points by pdeva1 11y ago

So the explanation is: Let me be blunt -- sun.misc.Unsafe must die in a fire. It is -- wait for it -- Unsafe. It must go. Ignore any kind of theoretical rope and start the path to righteousness And what the author takes away is: This engineer hates the Unsafe class for no reason at all. The reason the engineer hates it is that it's unsafe. Disagree with it, sure. But it's a reason, and it's a good reason (imho). I understand that it's commonly used by apps I use, but it's hard to keep reading after this kind of knee-jerk.

snuxoll 11y ago
In addition, it's in the sun.misc package, if it's not in java.* or javax.* it is ripe for pruning, you'd think people would have learned by now to stop using internal classes since pretty much every JRE/JDK release known to mankind has changed them or removed them.
- ante_annum 11y ago
  It's sort of amazing that the diatribe here is against Oracle for wanting to remove an undocumented internal API because of typesafety, and not against the open source projects that use a specifically unsafe, undocumented internal API. I mean, nothing against these projects -- I understand that type safety can carry a performance penalty -- but the anger of this article is amazing.
  - valarauca1 11y ago
    The issues is 2 fold for people. 1) People use Apache Storm and Cassandra daily. These need Unsafe to function at current performance changing its functionality may change these tools. This can be considered existiential threat to their job. 2) The one big selling point of the JVM is that JVM 1.0 code will run in JVM 8.0 without issue, and often much faster. With this change that isn't necessarily true, and potientally when your .jar was compile will weigh on which JVM is used or HOW that jvm is started.
    - geofft 11y ago
      1. Have the maintainers of Storm, Cassandra, etc. said whether they architecturally need Unsafe in its current form, and whether the replacement APIs under development will fulfill their needs? There's a lot of software out there that uses MD5 or DES right now. Saying that the software needs MD5 or DES is a different claim entirely.
      - valarauca1 11y ago
        >Have the maintainers of Storm, Cassandra, etc. said whether they architecturally need Unsafe in its current form [...] ? As there is no alternative to unsafe serialization in terms of performance. It goes without saying. >There's a lot of software out there that uses MD5 or DES right now. Saying that the software needs MD5 or DES is a different claim entirely. I agree, but you are putting the trees before the forest. Saying implementation X is flawed is one thing. Saying We are removing implementation X and offering no alternatives is another. While there are plan to offer an alternative, they are simply plans not an immidate alternative, thus a problem. When DES/MD5 were removed it was clear that they were already inferior implementations, and superior alternatives existed. That isn't true for java.
        geofft 11y ago
        > As there is no alternative to unsafe serialization in terms of performance. It goes without saying. I don't think that's true, though perhaps I don't understand what you mean by "unsafe serialization". First, I suspect one can use JNI and get at least as good performance. There are other tradeoffs, like the ease of packaging, but in order to evaluate those we need a clear problem statement. Second, premature optimization is the root of all evil, and we should be looking at realistic benchmarks to determine that there is a performance gain. There are many documented examples of software safety checks adding negligible performance overhead because they get branch-predicted away. Alternatively, new hardware support like Intel's MPX adds bounds-checking at the instruction level, so if we're talking about raw writes to a bounds-checked buffer (which is a vast improvement over raw writes to anywhere), this can probably be implemented efficiently on new hardware. On old hardware, the JVM could implement these as unchecked writes, which preserves the exact same performance, but the API would now have information to add safety where it is performant. Finally, sun.misc.Unsafe covers a lot of ground. If we can restrict it to the specific uses that these projects make of it, that's still an improvement. I don't think these libraries use the whole thing, do they? Very few things in science go without saying, and software performance and correctness is science.
- kasey_junk 11y ago
  The problem with that, is unlike most of the other internal classes, there is zero other way to do the things people are doing with unsafe without it. It's not a matter of expedience, it's a matter of necessity.
  - dlubarov 11y ago
    Yes, removing Unsafe would make things like PowerMock and Robolectric impossible to (re)implement. So what? Those libraries never should have been possible, and we can live without them. Languages need to be selective about what features they support, and Java was never intended to support things like mocking static methods, or creating instances without invoking any constructors.
    - kasey_junk 11y ago
      Without the current unsafe package most performance sensitive applications (whether throughput, latency, or pause sensitive) will have no recourse to meet their current performance levels. That will leave people to choose between staying on Java 8, worse (and often unacceptable) performance, and migrating to a different language. At this point, there is a very good chance if your library/application is known for performance and exists on the JVM it uses unsafe and without a migration plan all of us who care about performance will likely have to move off the language.
      - dlubarov 11y ago
        With some effort, I'm sure the folks behind Cassandra, Hadoop etc. could achieve acceptable performance with allocateDirect(). It doesn't support explicit freeing, but you can always reuse buffers and write your code to allocate slices within them.
    - twic 11y ago
      We can live without Robolectric? How do we write tests for Android code without it?
      - 11y ago
taeric 11y ago
That is a pretty crappy explanation, on the engineer's side. To be clear, any explanation that is not immediately followed with examples to how current valid uses of Unsafe can be accomplished without it, are simply witch hunts by an engineer that hasn't had to meet the same requirements.
- danudey 11y ago
  It's a) unsafe, and b) an internal, undocumented API. When you start importing from sun.misc.* you've made the conscious (or ignorant) decision to write non-portable code which may well break due to any JVM release. It sucks that these people are going to have to update their code (or that end users won't be able to upgrade to Java 9 without adding special command-line flags), but it's not due to fickle Oracle developers, it's due to technical debt that's finally catching up to them.
  - kasey_junk 11y ago
    You quite simply have NO option to do on the JVM most of the things people are using unsafe for in Java 9/10 without a migration plan from Oracle. What Oracle is close to "hey all you performance sensitive folks, we aren't going to support unsafe anymore, can you tell us what you are using it for and how we can migrate to that explicitly?" It just so happens that one of the leading proposals is "leave in java 9 but make it explicit to turn on". This is probably the right way to do this (in my opinion) but it does have the downside of making neither camp happy (it leaves unsafe so the purists are mad, and it causes headaches for people using unsafe).
    - MrBuddyCasino 11y ago
      Whats wrong with: - deprecate in Java 9 - "make it explicit to turn on" in 10, offer alternative official APIs as fallback - remove entirely when most people have switched
      - kasey_junk 11y ago
        Nothing at all. That's what I assume will happen at the end of the day. But people saying that we can just turn it off in 9 because you shouldn't be using internal api's don't seem to get that there is no external solution.
jevinskie 11y ago
I think the author's point about being able to make safe Unsafe replacements and keeping Unsafe around (for the time being) to enable better compatibility was thoughtful.

carsongross 11y ago

Once again, The Platonic Engineer gazes hatefully at the gronky, ugly and awkward things that make things work in real life, rub his hands together and says... "Burn it to the ground."

xg15 11y ago
Or you could actually stop for a second and find out why your current model needs so many ugly and awkward workarounds and try to adjust it accordingly - which seems to be what Oracle is trying to do in the long term.
- taeric 11y ago
  This is an odd one, though. I can not remember seeing/hearing any real issues with Unsafe. For the most part, people don't use it. There are a few that do, to ridiculous benefit. As things stand right now, without it, you can not get anything close to the performance people are seeing with it. So, what many of us are seeing here is someone is upset with basically the shape of an API. They have no evidence that I have seen of trouble caused by it. Only a desire that such things not exist. This feels almost puritanical in how it is being exercised. Which is just plain silly for a technical debate.
  - carsongross 11y ago
    puritanical I think the correct term is platonic: each of us has an Inner Platonic Engineer who cannot abide the ugliness of real world systems and insist on The Great Rewrite. A lot of progress relies on that engineer. However, in large, functioning and mature software systems he becomes the enemy.
    - taeric 11y ago
      I question just how much progress relies on that engineer. Be honest here, there are no identified problems that are being solved here. Merely an undesirable in the system that is very effective at its job. Progress, on other hand, advances something in the way of the problem space. Not merely rearranges something in the solution space.

ihuman 11y ago

What is "Unsafe?" Is it unsafe, as its name implies? If it is unsafe, why to so many programs use it?

dikaiosune 11y ago
It's part of the OpenJDK/Oracle VM implementation, not an official part of the Java standard library. It allows for all sorts of fun stuff like native compare and swap operations (used for concurrent collections iirc), object allocation without calling the constructor, accessing blocks of memory directly without object references, and other things. The reason it's called unsafe is because it allows developers to bypass a lot of the safety guarantees that java is supposed to provide as a memory managed language. But it's useful for getting more performance out of the VM. There's currently an effort underway to push through a variety of Java enhancement proposals that would add a lot of the unsafe functionality to the official standard library. This would enshrine the unsafe functionality as official, remove access to some of the more dangerous and less useful parts of unsafe, and would also mean that the functionality would be properly documented, which it is currently not.
- Daneel_ 11y ago
  Thanks for the thorough explanation - you summarised the issue well, as well as explaining the impact and flow on effects. I have to say, I agree with the engineer.. I'm cringing just knowing this sort of practice is seen as acceptable, and it reinforces my distaste for Java. If you have to resort to insecure practices to achieve performance, there's something wrong with the framework.
  - alextgordon 11y ago
    Reading this thread as C programmer has me scratching my head. Of course you have to resort to low-level functionality to achieve performance. That's how computers work. We can't all live in fuzzy wuzzy land. Surely Unsafe is safer than writing the whole thing in C and using JNI?
    - 11y ago

pron 11y ago

Chill. There is no disaster in the making. Oracle is very much aware of the issue, and is actually already taking the very same steps suggested by the author. It has assembled a public working group that is coming up with a spec for a public "Unsafe" API[1] (that is safer than Unsafe). In the meantime, use of Unsafe in Java 9 will be possible with a command-line flag in order to allow the working group more time to create the new API and for Unsafe consumers to migrate, and of course to allow existing libraries to run on Java 9. [1]: https://docs.google.com/document/d/1GDm_cAxYInmoHMor-AkStzWvwE9pw6tnz_CebJQxuUE/edit https://docs.google.com/document/d/1GDm_cAxYInmoHMor-AkStzWv...

jryan49 11y ago
The point is if you have to pass a flag to allow the use of Unsafe, you're breaking "works-out-of-the-box" backwards compatibility.
- pron 11y ago
  "Works-out-of-the-box" backwards compatibility -- without flags -- has never been guaranteed (even the default GC is about to change), let alone for non-public classes. Whenever you compile a class that uses Unsafe you get a warning (that you can't disable!) that says "this class may go away at any time". Still the reality is recognized, and you'll still be able to use Unsafe -- despite all warnings -- and all you need to do is add a flag. I think that's very reasonable.
  - jryan49 11y ago
    I agree in most cases that it's perfectly reasonable. But from a pragmatic position, I think this needs to be treated as a special case because of all the people that are using it. I still like the idea of having a flag that opts people in to using internal classes. It's like an agreement almost that they know what they're doing is not supported across versions.
  - snuxoll 11y ago
    Thankfully, Project Jigsaw (if it lands) in JDK9 should remove the problem of internal classes entirely — they simply won't be available to use.

ZitchDog 11y ago

Entire companies exist around functionality provided in Unsafe. I don't see this going well if they remove it without providing access to the functionality it gives. Java has plenty of problems, but backward compatibility over the years has been simply amazing. This would chip away at one of the few legitimate reasons to use Java in 2015.

xxpor 11y ago
sun.* has ALWAYS been exempt from that guarantee though.
- LoSboccacc 11y ago
  yeah they went the fast and cheap route instead of doing the right thing, and they are now going to be bitten by it, and they can go cry a river for all that I care. those are the sloppy engineers that are holding back evolution of software at large, from little things like obscure functions to huge operating systems like http://blogs.msdn.com/b/oldnewthing/archive/2005/07/28/444390.aspx http://blogs.msdn.com/b/oldnewthing/archive/2005/07/28/44439...
  - Someone1234 11y ago
    You forgot to explain what the "right thing" that they should be doing is? Even according to most articles on the topic there is very to any pleasant alternatives that are cross platform e.g. https://dzone.com/articles/understanding-sunmiscunsafe https://dzone.com/articles/understanding-sunmiscunsafe
    - LoSboccacc 11y ago
      jni, and don't tell me now that c isn't portable.
      - Someone1234 11y ago
        C isn't portable. You cannot write it once and run it everywhere, each platform needs a bunch of macros and flags. Plus you'll now either be delivering it in binary form (which may or may not work due to bitness, OS incompatibilities, and so on) or you'll be delivering it as code (which may or may not work due to compiler incompatibilities, build system components available, and missing libraries). C is only portable in the sense that "every platform ever made has a C compiler." But if you've ever written a C application which is just meant to run on Windows, OS X, and Linux you'll know that it is a painful experience (double that if they're meant to compile it themselves).

razorsese 11y ago

Literraly meme language

geofft 11y ago

The quote stops short, and for someone like me who doesn't follow Java very much, it feels like it's misquoting via omission: "Let me be blunt -- sun.misc.Unsafe must die in a fire. It is -- wait for it -- Unsafe. It must go. Ignore any kind of theoretical rope and start the path to righteousness _/now/_. It is still years until the end of public updates to JDK 8, so we have /years /to work this out properly. But sticking our heads in the collective sands and hoping for trivial work arounds to Unsafe is not going to work. If you're using Unsafe, this is the year to explain where the API is broken and get it straight.... "Please help us kill Unsafe, kill Unsafe dead, kill Unsafe right, and do so as quickly as possible to the ultimate benefit of everyone." The quoted engineer understands that people use it, claims that there's no immediate risk (since Java 8 will stay around), and very very strongly implies that there is a plan to move from Unsafe to something better than Unsafe providing equivalent functionality in a stable way, and asks for help with that. Is this correct?

jryan49 11y ago
Yes you are.
BhavdeepSethi 11y ago
Here is the link to the full quote. http://mail.openjdk.java.net/pipermail/openjfx-dev/2015-April/017028.html http://mail.openjdk.java.net/pipermail/openjfx-dev/2015-Apri...
pdeva1 11y ago
The post directly links to the full quote. If the intention was to misquote that would not be the case. > there is a plan to move from Unsafe to something better than Unsafe It is this plan and the implications of it that the post is discussing
- gclaramunt 11y ago
  Actually I don't see the post discussing anything, is just feels like "Unsafe is going away, the sky will fall!"
- 11y ago

xg15 11y ago

> Oracle plans to remove sun.misc.Unsafe in Java 9. This is an absolute disaster in the making and has the potential to completely destroy the entire ecosystem around Java. You know your language has a problem if the entire ecosystem threatens to fall apart as soon as you actually try to enforce the language's conceptual model and try to make use of the guarantees it provides.

bitmapbrother 11y ago
With the caveat that you need to believe in the hyperbole of a chicken running around with its head cut off.

sbov 11y ago

This seems a bit hyperbolic. Especially the comparison of Python 2 to Python 3. Its not like you could run Python 2 and Python 3 under the same interpreter by passing a flag.

karavelov 11y ago

What about shipping sun.misc.Unsafe as 3rd party package? Is there anything in it that can't be done as through JNI?

snuxoll 11y ago
No, you can do everything in Unsafe via JNI, but that comes with all the disadvantages of JNI (including .dll's and .so's is not so easy for a fat-jar deployment).
vardump 11y ago
It'd be unusably slow. "sun.misc.Unsafe" compiles in the instruction stream. JNI implementation would need to go through a complicated calling mechanism. I think it'd be up to 100x slower.
- karavelov 11y ago
  At least in OpenJDK it is in fact implemented as a JNI package, so it is already going through the JNI calling convention: http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/9b8c96f96a0f/src/share/classes/sun/misc/Unsafe.java http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/9b8c96f96a0f/s...
  - twic 11y ago
    That code shows that it's implemented as native methods, but not necessarily as JNI - these could be VM intrinsics, like Object::getClass and so on. This is the native code for Unsafe: https://github.com/openjdk-mirror/jdk7u-hotspot/blob/master/src/share/vm/prims/unsafe.cpp https://github.com/openjdk-mirror/jdk7u-hotspot/blob/master/... It's certainly JNI-ish. Whether it is actually called via the JNI mechanism, i can't tell.
  - vardump 11y ago
    I think these are a strong hint they're not truly JNI: Line 61: * <p> Most methods in this class are very low-level, and correspond to a * small number of hardware instructions (on typical machines). Compilers * are encouraged to optimize these methods accordingly. Line 90: /// peek and poke operations /// (compilers should optimize these to memory ops)

exabrial 11y ago

Document it, make it public. It's very much needed!

jart 11y ago

I just read an article that says in order to even use Unsafe, you have to use reflection to unwind the stack and go searching for a reference to the Unsafe instance, because the Java authors worked so hard to make it private. So it seems to me that they'd be well within their rights to remove the class. Normally when you go to that level of epic hackery, you can't expect for official support or backwards compatibility. So the author of this article using scare tactics to keep that feature seems rather disingenuous.

orf 11y ago
Could you provide a link to the article? I'm not saying you're wrong (I'm not a Java guy) but that sounds convoluted.
- twic 11y ago
  This is how you get access to Unsafe: Field theUnsafe = Unsafe.class.getDeclaredField("theUnsafe"); theUnsafe.setAccessible(true); Unsafe unsafe = (Unsafe) theUnsafe.get(null); Reflection, but no stack unwinding.

kkmickos 11y ago

Many people seem to be upset with Oracle, but I think they should be upset with the vendors who ship applications/libraries that depend on the "sun.*" packages. Oracle (then Sun) have since _1998_ strongly discouraged the use of them: https://web.archive.org/web/19980215011039/http://java.sun.com/products/jdk/faq/faq-sun-packages.html https://web.archive.org/web/19980215011039/http://java.sun.c...

Someone1234 11y ago
Fine, but why isn't the argument: "This then needs to be standardised in a Java.* package and correctly documented." Unsafe is useful, and is evident by how popular it is. Heck even Sun/Oracle must agree because they created it for their own internal usage. So why not just go ahead and standardise it.

vbezhenar 11y ago

Why libraries need Unsafe, after all? What problems does it solve for them?

kasey_junk 11y ago
It's almost entirely about performance. Unsafe allows you access to concurrent intrinsics as well as memory layout functions. It's true that it is widely used, but the people that use it also understood the ramifications of using it. If you were trying to make jvm agnostic code you didn't use it. If you were trying to make code that could run on a variety of jvm versions, you didn't use it. A lot of what people use it for is being rolled into documented/standard libraries and this is just the migration pains coming out.
- vbezhenar 11y ago
  Why those concurrent intrinsics are not available with "safe" code? They are not compatible with Java Memory Model? They are not implemented at some platforms? What's so unsafe about concurrent primitives? There is API in java.util.concurrent.atomic, for example. I understand that Unsafe is used for direct access to memory, but why this direct access is needed? May be there are other ways to solve those problems.
  - benjaminjackman 11y ago
    AFAIK there aren't other ways to get the performance required to be competitive in the financial space. People that are using these libraries don't want to use undocumented, Unsafe code. However, their hands are forced given the races they are competing in. They could switch to C++ but that's not really a great option for firms that have java code bases going back 10+ years. Plus, personally, and for a lot of others, the jvm plus a dash of sun.misc.Unsafe, is a lot better than going over to C++. More concretely, Here are two sets of trading libraries that use unsafe for direct memory access: https://github.com/real-logic/simple-binary-encoding https://github.com/real-logic/simple-binary-encoding https://github.com/OpenHFT https://github.com/OpenHFT (various libraries in here use it, once you drink the Unsafe Kool-aid it's hard to turn back from the performance benefits you obtain). If Oracle just rips off the Unsafe band-aid without a replacement, the business decision for these firms is still sound. They will just stay on 8 and start making migration plans (likely to the C++ bandwagon). However, It sounds like Oracle plans to give a sane alternative which is good and what everyone using Unsafe would prefer (along with deprecating it the meantime to give them some time to adapt). One real world example: The CME (Chicago Mercantile Exchange) reworked the entire way they distributed market data (v3 of their Market Data Protocol) to mirror the cutting edge in performance oriented serialization libraries (e.g. Cap'n Proto). They also commissioned an Open Source library for parsing that data (real-logic SBE), it uses unsafe to approach the performance of C++ code.

clamprecht 11y ago

At least he didn't name the article "Unsafe considered harmful."

bowyakka 11y ago

Ok moving past the rhetoric of don't use internal classes, there is a value to sun.misc.Unsafe. It has been well known by Oracle for some time that it has to go away right. Realise that those of us that do use it, are fully aware that it is a hack, and that we should be doing cleaner things. However know that those cleaner things _dont exist_; and that you, Dear application developer might not quite realise some of the shadow puppetry in making your favourite library work, please lay off the bile. As others have said, this blog post stops short, the full quote is part of a very long and ongoing process to rid ourselves of Unsafe, replacing its majors usages with new API's. This is a process Oracle started last year after the previous sun.reflect.Reflection.getCalleeClass mess. Yes, it no one should have been using an internal class. This method did not go away; it was made only available to the JVM internals, locked out from end users. The upshot of this was that, with no sane alternative, suddenly logging became between 2x and 100x more expensive where class names become involved. The JDK team somewhat back peddled on this change, and we are stuck here in a strange limbo land. Was it right of Oracle to do this? Absolutely, everyone who goes down the internal package route is fully aware that these are open-state secrets, sure we all know them, but we should not be talking about them. Support for these damn things requires version to version checking, hideous reflection hacks and being prepared to read a lot of JVM source code for when bugs occur. In light of all of this, and being the general level of awesome the JDK folks are they started a very direct community outreach to remove safely Unsafe. Last year they publicly asked people on the mailing list, and privately via email to do a survey (http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-January/024650.html http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-Ja...) on usages of Unsafe. I know this because I got an email from Paul Sandoz asking me to comment on Unsafe abusages. Out of previous thinking from the JDK folks, as well as from the aforementioned survey, we have a bunch of JEP's for improving things regarding Unsafe. Where things have gone slightly sideways is that we have, currently no unified working group to deal with these changes. What this document is, is a proposal to make a working group, focused on getting the right changes into the JVM. In the same ways as Project Jigsaw is for modularising the JVM (and is indirectly responsible for removing Unsafe), or the MLVM project handled making the VM better support other languages. Unsafe is dead, long live Unsafe! I fully expect that we will achieve the right changes in during Java 9 to allow those of us that need to break the rules, to do it in a supported fashion. I am _looking_ forward to the day when I can do setMemory or getAndSetInt via a supported API, and not one where I have to know the words Unsafe.theUnsafe. The above is a storm in a teacup, its open-source democracy at work.

sreque 11y ago

There's a lot of backlash here against the author with comments like: "You used a sun.* package? You deserve to die in a fire as well!" These comments look like they are coming from people who have never used Unsafe. First, Unsafe provides some immensely useful features to the Java ecosystem. Second, there is no alternative to Unsafe on the JVM. That includes JNI (native code integrated within the JVM). So far, Oracle has stated their intent to remove Unsafe from Java 9. To all the users of Unsafe, Oracle is basically saying, "tell us your use cases for Unsafe, and if we decide they are valid we may try to create an alternative for you. If you're lucky, we may even ship that alternative in Java 9 or 10! Have a nice day!" Many people, myself included, find this unacceptable. For instance, it looks like several groups are currently collaborating on a document discussing uses cases for Unsafe and whether alternative features satisfying those use cases are expected to appear in Java 9: https://docs.google.com/document/d/1GDm_cAxYInmoHMor-AkStzWvwE9pw6tnz_CebJQxuUE/edit?pli=1 https://docs.google.com/document/d/1GDm_cAxYInmoHMor-AkStzWv... As you can see, the majority of the cells in the column "Expected in Java 9" flat out say "no". There is a good reason people are up in arms about the removal of this API. Personally, if I had to choose between Oracle keeping Unsafe and Oracle implementing Java 8 Lambdas, I would have picked Unsafe in a heartbeat. At the end of the day, Lambdas are mostly syntactic sugar, especially in their current implementation. Unsafe, on the other hand, actually empowers you to do something more.

geofft 11y ago
That link is a very useful contribution to this discussion (and possibly the first really useful comment in this thread), thanks. Everyone's entitled to their opinion, informed or not, but if you want an informed opinion, the question needs to be what the current uses of Unsafe are, why they're used, and whether there are plans for a migration path. It sort of seems like the major use case (judging from those numbers) is reading and writing raw byte buffers, possibly mapped from other objects/files, with native performance. Is there a potential sensible design for a bounds-checked raw data API ("unsafe but only within these pages") that maintains the JVM's usual safety guarantees for all other data, and can be implemented in a performant way?
- pdeva1 11y ago

jkot 11y ago

Compression in my library is 3x faster with unsafe. I think it is lesser evil then native code integration.

skrowl 11y ago

Still using Java in 2015 – A disaster in the making

bitmapbrother 11y ago
What would a MS fanboy suggest as an alternative?
goback2reddit 11y ago
Go back to reddit.

hyperpape 11y ago

Why on Earth is it a big deal that you need to pass a flag to the JVM at startup? I understand that backwards compatibility is valued, but if I went to my CTO and said "we're gonna drop Java 9 in production, and we're not gonna test beforehand"...the reaction would not be pretty.

rebootthesystem 11y ago

This is related and unrelated and maybe even meta to some degree: There are a couple of companies that are guilty of using their free software update mechanism to try and trick people into installing crap they don't want (and might even mess-up their systems): Sun and Adobe. I almost happened to me last night as I updated Flash on a test system and some bullshit anti-virus crap-ware was going to be installed because I was moving fast and neglected to un-check the check-box on the site. I was able to stop the install process and start over. No harm done. These companies need public shaming to stop using this bullshit method to make a few more bucks. Are they so hard-up that they need to trick users to install crap to make money? I sure hope not. Sun, Adobe, please STOP trying to trick users into installing crap-ware they were not looking for in the first place.

kasey_junk 11y ago
You understand of course that Sun has been out of business for a very long time right?
- rebootthesystem 11y ago
  Sorry, brain fart, I meant Oracle
throwaway2048 11y ago
java hasnt been bundled with crapware since oracle bought it
- rebootthesystem 11y ago
  Some installations I've seen have crapware pre-selected. I haven't looked at it lately, maybe they changed this.

scubaguy 11y ago

I read this article yesterday and I really dislike how it is worded. The "this engineer" has a name, Donald Smith, this is what he said: "If you're using Unsafe, this is the year to explain where the API is broken and get it straight.... Please help us kill Unsafe, kill Unsafe dead, kill Unsafe right, and do so as quickly as possible to the ultimate benefit of everyone." It reads like a request to have a discussion on Unsafe, to talk about it and find solutions. Unfortunately, I don't see anyone pointing out why Unsafe should be kept around on that thread (http://mail.openjdk.java.net/pipermail/openjfx-dev/2015-April/017028.html http://mail.openjdk.java.net/pipermail/openjfx-dev/2015-Apri...). And this quote from Oracle shows an effort to do things right: http://mail.openjdk.java.net/pipermail/discuss/2013-October/003162.html http://mail.openjdk.java.net/pipermail/discuss/2013-October/... "This is definitely something we plan to propose for SE 9. I expect to see a JEP from the current maintainers of sun.misc.Unsafe..." This (http://blog.codefx.org/java/dev/how-java-9-and-project-jigsaw-may-break-your-code/ http://blog.codefx.org/java/dev/how-java-9-and-project-jigsa...) seems like a more balanced article on the situation. "So far we focused on the problematic aspects of Project Jigsaw. But that should not divert from the exciting and – I think – very positive nature of the planned changes. After reading the documents, I am impressed with the scope and potential of this upcoming Java release. While it is likely not as groundbreaking for individual developers as Java 8, it is even more so for everyone involved in building and deploying – especially of large monolithic projects." near future. (This is far too small to require a whole JSR.)"