4 ms·

Does anyone know if pandas is affected? I serialize/deserialize dataframes which pandas uses parquet under the hood.

by ustad 2y ago

natebc 2y ago
https://www.endorlabs.com/learn/critical-rce-vulnerability-in-apache-parquet-cve-2025-30065---advisory-and-analysis https://www.endorlabs.com/learn/critical-rce-vulnerability-i... > Any application or service using Apache Parquet Java library versions 1.15.0 or earlier is believed to be vulnerable (our own data indicates that this was introduced in version 1.8.0; however, current guidance is to review all historical versions). This includes systems that read or import Parquet files using popular big-data frameworks (e.g. Hadoop, Spark, Flink) or custom applications that incorporate the Parquet Java code. If you are unsure whether your software stack uses Parquet, check with your vendors or developers – many data analytics and storage solutions include this library. Seems safe to assume yes, pandas is probably affected by using this library.
- nindalf 2y ago
  The paragraph you pasted in states that only applications importing the Java library are vulnerable. Isn’t pandas implemented in Python/C? How would it have been importing the Java library?
  - deleted 2y ago
    [deleted]
  - natebc 2y ago
    I'm sorry. I made a mistake.
- 3eb7988a1663 2y ago
  That does not follow for me. Pandas does not utilize Java/JVM.
  - deleted 2y ago
    [deleted]
  - natebc 2y ago
    I'm sorry. I made a mistake.
- deleted 2y ago
  [deleted]
minimaxir 2y ago
Pandas doesn't use the parquet python package under the hood: https://pandas.pydata.org/docs/reference/api/pandas.read_parquet.html https://pandas.pydata.org/docs/reference/api/pandas.read_par... > Parquet library to use. If ‘auto’, then the option io.parquet.engine is used. The default io.parquet.engine behavior is to try ‘pyarrow’, falling back to ‘fastparquet’ if ‘pyarrow’ is unavailable. Those should be unaffected.
- westurner 2y ago
  Python pickles have the same issue but it is a design decision per the docs. Python docs > library > pickle: https://docs.python.org/3/library/pickle.html https://docs.python.org/3/library/pickle.html Re: a hypothetical pickle parser protocol that doesn't eval code at parse time; "skipcode pickle protocol 6: "AI Supply Chain Attack: How Malicious Pickle Files Backdoor Models" .. "Insecurity and Python Pickles" : https://news.ycombinator.com/item?id=43426963 https://news.ycombinator.com/item?id=43426963
  - deleted 2y ago
    [deleted]
  - echoangle 2y ago
    But python pickle is only supposed to be used with trusted input, so it’s not a vulnerability.

Does anyone know if pandas is affected? I serialize/deserialize dataframes which pandas uses parquet under the hood.

by ustad 2y ago

Does anyone know if pandas is affected? I serialize/deserialize dataframes which pandas uses parquet under the hood.

https://www.endorlabs.com/learn/critical-rce-vulnerability-in-apache-parquet-cve-2025-30065---advisory-and-analysis https://www.endorlabs.com/learn/critical-rce-vulnerability-i... > Any application or service using Apache Parquet Java library versions 1.15.0 or earlier is believed to be vulnerable (our own data indicates that this was introduced in version 1.8.0; however, current guidance is to review all historical versions). This includes systems that read or import Parquet files using popular big-data frameworks (e.g. Hadoop, Spark, Flink) or custom applications that incorporate the Parquet Java code. If you are unsure whether your software stack uses Parquet, check with your vendors or developers – many data analytics and storage solutions include this library. Seems safe to assume yes, pandas is probably affected by using this library.

nindalf 2y ago
The paragraph you pasted in states that only applications importing the Java library are vulnerable. Isn’t pandas implemented in Python/C? How would it have been importing the Java library?
- deleted 2y ago
  [deleted]
- natebc 2y ago
  I'm sorry. I made a mistake.
3eb7988a1663 2y ago
That does not follow for me. Pandas does not utilize Java/JVM.
- deleted 2y ago
  [deleted]
- natebc 2y ago
  I'm sorry. I made a mistake.
deleted 2y ago
[deleted]

minimaxir 2y ago

Pandas doesn't use the parquet python package under the hood: https://pandas.pydata.org/docs/reference/api/pandas.read_parquet.html https://pandas.pydata.org/docs/reference/api/pandas.read_par... > Parquet library to use. If ‘auto’, then the option io.parquet.engine is used. The default io.parquet.engine behavior is to try ‘pyarrow’, falling back to ‘fastparquet’ if ‘pyarrow’ is unavailable. Those should be unaffected.

westurner 2y ago
Python pickles have the same issue but it is a design decision per the docs. Python docs > library > pickle: https://docs.python.org/3/library/pickle.html https://docs.python.org/3/library/pickle.html Re: a hypothetical pickle parser protocol that doesn't eval code at parse time; "skipcode pickle protocol 6: "AI Supply Chain Attack: How Malicious Pickle Files Backdoor Models" .. "Insecurity and Python Pickles" : https://news.ycombinator.com/item?id=43426963 https://news.ycombinator.com/item?id=43426963
- deleted 2y ago
  [deleted]
- echoangle 2y ago
  But python pickle is only supposed to be used with trusted input, so it’s not a vulnerability.