6 ms·

There are so many great alternatives available. It's a no-brainer. I applaud. [edit] so why the downvote? We have YAML, JSON, protobuf, Thrift, Avro. Yup, thes

There are so many great alternatives available. It's a no-brainer. I applaud. [edit] so why the downvote? We have YAML, JSON, protobuf, Thrift, Avro. Yup, these serialize "contents" rather than "structure + contents" but one gets interop with other technologies for free. Every tech mentioned above is so simple to use that removing Java serialization is a no-brainer.

spockz 8y ago
Don’t forget Kryo[1]. Also Avrò messages can contain the schema. [1]: https://github.com/EsotericSoftware/kryo https://github.com/EsotericSoftware/kryo
- rad_gruchalski 8y ago
  Yes! But Kryo docs explicitly mentions some things may have to be handled via Java serialization.
kibibu 8y ago
I suggest you look at the myriad security problems caused by Ruby's handling of YAML and JSON. It's not necessarily the serialization format at fault, but rather developer assumptions.
- willvarfar 8y ago
  (Here's a post describing some of the problems back in the day with Ruby's YAML and JSON: https://williamedwardscoder.tumblr.com/post/43394068341/rubys-principle-of-too-much-power https://williamedwardscoder.tumblr.com/post/43394068341/ruby... YAML is of course a format that is designed to instantiate objects as described in the source, rather than as checked by the destination, so I wouldn't want the world to adopt YAML instead of Java serialization.)
  - imtringued 8y ago
    Yaml is primarily used as hierarchical config format because JSON and XML are too verbose for a human editor.
    - simonh 8y ago
      Exactly. The people loading up a YAML library and using it that way may have no idea its a potential remote code execution vector into their application.
      - philipov 8y ago
        Luckily, for those that do know what they're doing, it's simple enough to use the yaml.safe_load function that disables support for arbitrary object instantiation.
        willvarfar 8y ago
        I believe that the safe_load function only got added because of the spate of exploits that the blog post was talking about. Its good that its been added, although it would have been better if it'd been there from the beginning and the safe version was default and you had to invoke 'unsafe_load' if you wanted complex object instantiation, to hopefully encourage even novices to think twice before doing it with tainted input.
  - the_new_guy_29 8y ago
    Read through it, but i wont blame language creators. Its a lot easier to point out that something is broken than to build something better.
    - coldtea 8y ago
      And it's a lot easier to build a new broken thing, that to fix something broken. Which is what the creators of YAML did.
  - jwilk 8y ago
    Archived copy that can be read without JS enabled: https://archive.is/2JGiJ https://archive.is/2JGiJ
  - jwilk 8y ago
    > Only it then turns out that the Ruby JSON parser … yes, you’ve guessed it … the Ruby JSON parser can instantiate complex objects too. Huh. Where can I read more about this?
- jwilk 8y ago
  Regarding YAML, it's not just Ruby: * Python's yaml.load() hapilly executes arbitrary code. * Some Perl YAML libraries deserialize objects by default: http://blogs.perl.org/users/tinita/2018/02/safely-load-untrusted-yaml-in-perl.html http://blogs.perl.org/users/tinita/2018/02/safely-load-untru...
  - philipov 8y ago
    Python's YAML libraries come with a safe_load function that disables !! commands, so it can be used without loading arbitrary objects.
    - ArneBab 8y ago
      And they warn quite clearly about that. Though in hindsight, having load and unsafe_load would have been safer.
      - jwilk 8y ago
        > And they warn quite clearly about that. Do they? https://github.com/yaml/pyyaml https://github.com/yaml/pyyaml doesn't even mention safe_load(). And this is the docstring for yaml.load(): Parse the first YAML document in a stream and produce the corresponding Python object. This doesn't sound very scary, and certainly doesn't imply possible code execution.
MichaelMoser123 8y ago
That would also kill RMI (at least JRMP), and that too would be a good thing, RMI doesn't even pass through a router (unless you do RMI-IIOP - and that is very different from regular RMI - JRMP) For java backwards compatibility used to be very important, so this is big news for the platform. I think this remoting the bytecode with serialisation madness was once upon a time very important part of RMI/serialisation - back in the thin client java days this was supposed to be the way to distribute code across a network link, security was not the very first priority in the nineties (beats me why they made JRMP non routable)
- MichaelMoser123 8y ago
  Does not pass through firewall/NAT to be more specific
- fs111 8y ago
  JMX uses RMI underneath, so that would have to be fixed first before they can remove RMI
  - parasubvert 8y ago
    JMX protocols are pluggable, so this likely will be doable. The main issue will be updating all the ancient servers running Java < v7.
- parasubvert 8y ago
  If you mean that JRMP leaks internal IP addresses in the protocol itself for callbacks and thus can't be NATted, that can be fixed with a couple properties to force the use of outer-IP DNS: java.rmi.server.hostname=myhostname.com java.rmi.server.useLocalHostname=true You also can tunnel JRMP through HTTP - there was a CGI script called java-rmi dating back to the late 1990's that I think was still distributed through Java 8 (!), and also an RMI Servlet Handler which was a bit more robust/performant. Spring also still has the RmiServiceExporter and HttpInvokerServiceExporter. I remember building Java applets and servers that did fixed income quotes & bond trading systems via streamed encrypted serialized Java objects circa 1999-2000. What a security nightmare, but no one knew better. I feel old.
kazra 8y ago
Alternatives or not, you open that 'risky' door at the moment you have to read the payload to know what type of data it is. Once it's read, it's already too late. Java serialization has not many ways around it, you have to trust the sources. XML, JSON, ... used naively with reflection (e.g. new XStream().fromXML(...)) exposes the exact same issues. Custom homemade parsers are also very likely somehow vulnerable. The deserialization attack is a way to make the deserializer exploits vulnerable classes that exist in your classpath. It's not generating or executing malicious code by itself. The standard classes should be ok, or at least fixed quickly. Things like commons-collection will likely never be fixed. It might be considered as a feature from some point of view. Check this out: https://github.com/frohoff/ysoserial https://github.com/frohoff/ysoserial My 2 cents: - Secure your sources - Know you format, do not rely on reflection to parse text or binary data - Watch out with your classpath, but you can never know what new vulnerability will pop next weeks
eicnix 8y ago
While using a general data transport format instead of Java object serialization is great for most applications it cannot replace all usecases. JVM based distributed grid computing services use Java object serialization to distribute queries throughout the grid. This allows users to write their queries as arbitrary Java code which will be executed on every node without having to deploy the code on every node.
- dpark 8y ago
  How does that work? You describe a situation in which code that does not exist on a given node is executed on that node nonetheless.
  - eicnix 8y ago
    You write a query that is wrapped inside a serializable object. The object with your code inside is serialized using Java object serialization and distributed to every node. Each node is now able to execute your query based on Java code. https://ignite.apache.org/features/computegrid.html https://ignite.apache.org/features/computegrid.html
    - peeters 8y ago
      This isn't a native feature of Java serialization. Class implementations are not included in the serialized data. > How do closures get shipped around? > Every closure is an object of a particular class. When the closure is being sent it gets serialised to a binary form, send over the wire to a remote node and deserialised there. The remote node should have the closure's class in its classpath or enable peerClassLoading in order to load the class from the sender side. https://apacheignite.readme.io/docs/faq https://apacheignite.readme.io/docs/faq

spockz 8y ago

Don’t forget Kryo[1]. Also Avrò messages can contain the schema. [1]: https://github.com/EsotericSoftware/kryo https://github.com/EsotericSoftware/kryo

rad_gruchalski 8y ago
Yes! But Kryo docs explicitly mentions some things may have to be handled via Java serialization.

kibibu 8y ago

I suggest you look at the myriad security problems caused by Ruby's handling of YAML and JSON. It's not necessarily the serialization format at fault, but rather developer assumptions.

willvarfar 8y ago
(Here's a post describing some of the problems back in the day with Ruby's YAML and JSON: https://williamedwardscoder.tumblr.com/post/43394068341/rubys-principle-of-too-much-power https://williamedwardscoder.tumblr.com/post/43394068341/ruby... YAML is of course a format that is designed to instantiate objects as described in the source, rather than as checked by the destination, so I wouldn't want the world to adopt YAML instead of Java serialization.)
- imtringued 8y ago
  Yaml is primarily used as hierarchical config format because JSON and XML are too verbose for a human editor.
  - simonh 8y ago
    Exactly. The people loading up a YAML library and using it that way may have no idea its a potential remote code execution vector into their application.
    - philipov 8y ago
      Luckily, for those that do know what they're doing, it's simple enough to use the yaml.safe_load function that disables support for arbitrary object instantiation.
      - willvarfar 8y ago
        I believe that the safe_load function only got added because of the spate of exploits that the blog post was talking about. Its good that its been added, although it would have been better if it'd been there from the beginning and the safe version was default and you had to invoke 'unsafe_load' if you wanted complex object instantiation, to hopefully encourage even novices to think twice before doing it with tainted input.
- the_new_guy_29 8y ago
  Read through it, but i wont blame language creators. Its a lot easier to point out that something is broken than to build something better.
  - coldtea 8y ago
    And it's a lot easier to build a new broken thing, that to fix something broken. Which is what the creators of YAML did.
- jwilk 8y ago
  Archived copy that can be read without JS enabled: https://archive.is/2JGiJ https://archive.is/2JGiJ
- jwilk 8y ago
  > Only it then turns out that the Ruby JSON parser … yes, you’ve guessed it … the Ruby JSON parser can instantiate complex objects too. Huh. Where can I read more about this?
jwilk 8y ago
Regarding YAML, it's not just Ruby: * Python's yaml.load() hapilly executes arbitrary code. * Some Perl YAML libraries deserialize objects by default: http://blogs.perl.org/users/tinita/2018/02/safely-load-untrusted-yaml-in-perl.html http://blogs.perl.org/users/tinita/2018/02/safely-load-untru...
- philipov 8y ago
  Python's YAML libraries come with a safe_load function that disables !! commands, so it can be used without loading arbitrary objects.
  - ArneBab 8y ago
    And they warn quite clearly about that. Though in hindsight, having load and unsafe_load would have been safer.
    - jwilk 8y ago
      > And they warn quite clearly about that. Do they? https://github.com/yaml/pyyaml https://github.com/yaml/pyyaml doesn't even mention safe_load(). And this is the docstring for yaml.load(): Parse the first YAML document in a stream and produce the corresponding Python object. This doesn't sound very scary, and certainly doesn't imply possible code execution.

MichaelMoser123 8y ago

That would also kill RMI (at least JRMP), and that too would be a good thing, RMI doesn't even pass through a router (unless you do RMI-IIOP - and that is very different from regular RMI - JRMP) For java backwards compatibility used to be very important, so this is big news for the platform. I think this remoting the bytecode with serialisation madness was once upon a time very important part of RMI/serialisation - back in the thin client java days this was supposed to be the way to distribute code across a network link, security was not the very first priority in the nineties (beats me why they made JRMP non routable)

MichaelMoser123 8y ago
Does not pass through firewall/NAT to be more specific
fs111 8y ago
JMX uses RMI underneath, so that would have to be fixed first before they can remove RMI
- parasubvert 8y ago
  JMX protocols are pluggable, so this likely will be doable. The main issue will be updating all the ancient servers running Java < v7.
parasubvert 8y ago
If you mean that JRMP leaks internal IP addresses in the protocol itself for callbacks and thus can't be NATted, that can be fixed with a couple properties to force the use of outer-IP DNS: java.rmi.server.hostname=myhostname.com java.rmi.server.useLocalHostname=true You also can tunnel JRMP through HTTP - there was a CGI script called java-rmi dating back to the late 1990's that I think was still distributed through Java 8 (!), and also an RMI Servlet Handler which was a bit more robust/performant. Spring also still has the RmiServiceExporter and HttpInvokerServiceExporter. I remember building Java applets and servers that did fixed income quotes & bond trading systems via streamed encrypted serialized Java objects circa 1999-2000. What a security nightmare, but no one knew better. I feel old.

kazra 8y ago

Alternatives or not, you open that 'risky' door at the moment you have to read the payload to know what type of data it is. Once it's read, it's already too late. Java serialization has not many ways around it, you have to trust the sources. XML, JSON, ... used naively with reflection (e.g. new XStream().fromXML(...)) exposes the exact same issues. Custom homemade parsers are also very likely somehow vulnerable. The deserialization attack is a way to make the deserializer exploits vulnerable classes that exist in your classpath. It's not generating or executing malicious code by itself. The standard classes should be ok, or at least fixed quickly. Things like commons-collection will likely never be fixed. It might be considered as a feature from some point of view. Check this out: https://github.com/frohoff/ysoserial https://github.com/frohoff/ysoserial My 2 cents: - Secure your sources - Know you format, do not rely on reflection to parse text or binary data - Watch out with your classpath, but you can never know what new vulnerability will pop next weeks

eicnix 8y ago

While using a general data transport format instead of Java object serialization is great for most applications it cannot replace all usecases. JVM based distributed grid computing services use Java object serialization to distribute queries throughout the grid. This allows users to write their queries as arbitrary Java code which will be executed on every node without having to deploy the code on every node.

dpark 8y ago
How does that work? You describe a situation in which code that does not exist on a given node is executed on that node nonetheless.
- eicnix 8y ago
  You write a query that is wrapped inside a serializable object. The object with your code inside is serialized using Java object serialization and distributed to every node. Each node is now able to execute your query based on Java code. https://ignite.apache.org/features/computegrid.html https://ignite.apache.org/features/computegrid.html
  - peeters 8y ago
    This isn't a native feature of Java serialization. Class implementations are not included in the serialized data. > How do closures get shipped around? > Every closure is an object of a particular class. When the closure is being sent it gets serialised to a binary form, send over the wire to a remote node and deserialised there. The remote node should have the closure's class in its classpath or enable peerClassLoading in order to load the class from the sender side. https://apacheignite.readme.io/docs/faq https://apacheignite.readme.io/docs/faq