5 ms·

Is your criticism about the fact that pickle is language-specific or about it being a binary format?

philsnow 10y ago
(not the person you're replying to, but) my criticism of python pickle is that it allows the deserialized thing to refer to objects outside of a restricted environment (in this case, subprocess.Popen). JSON deserialization can't refer to console.log or any other object. Unless you're storing an HMAC alongside your pickled blob, and verifying that HMAC on the deserialization side, you should not trust pickle for anything. (Even then, it's still potentially dangerous, because in a large project some other dev who doesn't understand the issue could come along and write a new thing to deserialize the same pickles and not check the HMAC).
- sitkack 10y ago
  If you want a "secure pickle" you should corrupt it on creation so that it needs a to uncorrupted to be read. preferably with the HMAC.
  - ashitlerferad 10y ago
    No, just switch from pickle to json.
    - sitkack 10y ago
      But how would you get arbitrary code execution? Pickle does solve a lot of problems, but it creates so many that I think it should be removed from the battery pack. You can't make HMAC integrity checks optional, and you make the mandatory by breaking the payload.
- IanCal 10y ago
  > Unless you're storing an HMAC alongside your pickled blob, and verifying that HMAC on the deserialization side, you should not trust pickle for anything. I dislike such strong statements. When I use pickle, it's often a dump of where a computation is up to, or a simple cache. How can that be exploited? Someone would need to be meddling on my filesystem, in which case I'm already screwed. What risks am I opening myself up to? Why should I not trust it for my use case?
  - philsnow 10y ago
    It depends on the kind of groups/organizations you write code in. In some, I think such strong statements are necessary. You can mitigate the dangers of pickle by educating junior devs or people new to python, by having a strong culture of code reviews with experienced reviewers, by having a style guide that explicitly prohibits pickle except in exceptional cases. ... Or you can just not use it in the first place and make it trivial for newcomers to your codebase to use safer serialization methods.
- vertex-four 10y ago
  Well, specifically, never use pickle for communication. It's perfectly reasonable as an on-disk storage format for some things.

(not the person you're replying to, but) my criticism of python pickle is that it allows the deserialized thing to refer to objects outside of a restricted environment (in this case, subprocess.Popen). JSON deserialization can't refer to console.log or any other object. Unless you're storing an HMAC alongside your pickled blob, and verifying that HMAC on the deserialization side, you should not trust pickle for anything. (Even then, it's still potentially dangerous, because in a large project some other dev who doesn't understand the issue could come along and write a new thing to deserialize the same pickles and not check the HMAC).

sitkack 10y ago
If you want a "secure pickle" you should corrupt it on creation so that it needs a to uncorrupted to be read. preferably with the HMAC.
- ashitlerferad 10y ago
  No, just switch from pickle to json.
  - sitkack 10y ago
    But how would you get arbitrary code execution? Pickle does solve a lot of problems, but it creates so many that I think it should be removed from the battery pack. You can't make HMAC integrity checks optional, and you make the mandatory by breaking the payload.
IanCal 10y ago
> Unless you're storing an HMAC alongside your pickled blob, and verifying that HMAC on the deserialization side, you should not trust pickle for anything. I dislike such strong statements. When I use pickle, it's often a dump of where a computation is up to, or a simple cache. How can that be exploited? Someone would need to be meddling on my filesystem, in which case I'm already screwed. What risks am I opening myself up to? Why should I not trust it for my use case?
- philsnow 10y ago
  It depends on the kind of groups/organizations you write code in. In some, I think such strong statements are necessary. You can mitigate the dangers of pickle by educating junior devs or people new to python, by having a strong culture of code reviews with experienced reviewers, by having a style guide that explicitly prohibits pickle except in exceptional cases. ... Or you can just not use it in the first place and make it trivial for newcomers to your codebase to use safer serialization methods.
vertex-four 10y ago
Well, specifically, never use pickle for communication. It's perfectly reasonable as an on-disk storage format for some things.