4 ms·

What is the "deserialisation pandemic"? It doesn't have obvious web search results, and I'm struggling to imagine what about deserialisation what be common betw

What is the "deserialisation pandemic"? It doesn't have obvious web search results, and I'm struggling to imagine what about deserialisation what be common between Java and Python (except that, in both cases, I'd surely just use protobuf if I wanted binary serialisation).

kibwen 2y ago
See https://en.wikipedia.org/wiki/Log4Shell https://en.wikipedia.org/wiki/Log4Shell , but also historically the mess that is pickling/unpickling in Python (see the big scary warning at the top of https://docs.python.org/3/library/pickle.html#pickle-python-object-serialization https://docs.python.org/3/library/pickle.html#pickle-python-... ), and more broadly any dynamic language that exposes `eval` in any capacity.
- tptacek 2y ago
  For many years, these were the most widespread serverside RCE vulnerabilities; Rails YAML might be the best-known, but there were a bunch of different variants in Java serialization, and a whole cottage subfield of vulnerability research deriving different sequences of objects/methods to bounce deserializations through. It was a huge problem, and my perception is that it sort of bled into SSRF (now the scariest vulnerability you're likely to have serverside) via XML deserialization.
  - sn9 2y ago
    You said that Go and Rust managed to avoid these issues. Is there anywhere I can read about how they avoided it? And why other popular modern languages can't?
plorkyeran 2y ago
In the early 2000/2010s there was a popular idea that it'd be neat to have (de)serialization functionality that could perfectly roundtrip your language's native objects, without requiring that the objects be whatever the language uses as plain old data storage. In the happy case it worked super well and basically every language sufficiently dynamic to support it got a library which let you take some in memory objects, write them to disk, then restore them exactly as they were at some later time. This had the obvious-in-retrospect major problem that it meant that your deserialization was functionally equivalent to eval(), and if an attacker could ever control what you deserialized they could execute arbitrary code. Many programmers did not realize this and just plain called deserialization functions on untrusted data, and even when people did become aware that was bad it still turned lots of minor bugs into RCE bugs. It was often a long and painful migration away from insecure deserialization methods because of how darn convenient they were, so it continued to be a problem long after it was well understood that things like pickle were a bad idea.

kibwen 2y ago

See https://en.wikipedia.org/wiki/Log4Shell https://en.wikipedia.org/wiki/Log4Shell , but also historically the mess that is pickling/unpickling in Python (see the big scary warning at the top of https://docs.python.org/3/library/pickle.html#pickle-python-object-serialization https://docs.python.org/3/library/pickle.html#pickle-python-... ), and more broadly any dynamic language that exposes `eval` in any capacity.

tptacek 2y ago
For many years, these were the most widespread serverside RCE vulnerabilities; Rails YAML might be the best-known, but there were a bunch of different variants in Java serialization, and a whole cottage subfield of vulnerability research deriving different sequences of objects/methods to bounce deserializations through. It was a huge problem, and my perception is that it sort of bled into SSRF (now the scariest vulnerability you're likely to have serverside) via XML deserialization.
- sn9 2y ago
  You said that Go and Rust managed to avoid these issues. Is there anywhere I can read about how they avoided it? And why other popular modern languages can't?

plorkyeran 2y ago

In the early 2000/2010s there was a popular idea that it'd be neat to have (de)serialization functionality that could perfectly roundtrip your language's native objects, without requiring that the objects be whatever the language uses as plain old data storage. In the happy case it worked super well and basically every language sufficiently dynamic to support it got a library which let you take some in memory objects, write them to disk, then restore them exactly as they were at some later time. This had the obvious-in-retrospect major problem that it meant that your deserialization was functionally equivalent to eval(), and if an attacker could ever control what you deserialized they could execute arbitrary code. Many programmers did not realize this and just plain called deserialization functions on untrusted data, and even when people did become aware that was bad it still turned lots of minor bugs into RCE bugs. It was often a long and painful migration away from insecure deserialization methods because of how darn convenient they were, so it continued to be a problem long after it was well understood that things like pickle were a bad idea.