3 ms·

When did vulnerability reports get so vague? Looks like a classic serialization bug https://github.com/apache/parquet-java/compare/apache-parquet-1.15.0...apac

by g-mork 2y ago

When did vulnerability reports get so vague? Looks like a classic serialization bug https://github.com/apache/parquet-java/compare/apache-parquet-1.15.0...apache-parquet-1.15.1#diff-843b7d69e1ab11efea5f9cbb3cdef97018962430525040a2a5c3eed0fb5848a3R273 https://github.com/apache/parquet-java/compare/apache-parque...

amluto 2y ago
Better link: https://github.com/apache/parquet-java/pull/3169 https://github.com/apache/parquet-java/pull/3169 If by “classic” you mean “using a language-dependent deserialization mechanism that is wildly unsafe”, I suppose. The surprising part is that Parquet is a fairly modern format with a real schema that is nominally language-independent. How on Earth did Java class names end up in the file format? Why is the parser willing to parse them at all? At most (at least by default), the parser should treat them as predefined strings that have semantics completely independent of any actual Java class.
- bri3d 2y ago
  This seems to come from parquet-avro, which looks to attempt to embed Avro in Parquet files and in the course of doing so, does silly Java reflection gymnastics. I don’t think “normal” parquet is affected.
  - amluto 2y ago
    The documentation for all of this is atrocious. But if avro-in-parquet is a weird optional feature, it should be off by default! Parquet’s metadata is primarily in Thrift, not Avro, and it seems to me that no Avro should be involved in decoding Parquet files unless explicitly requested.
    - bri3d 2y ago
      To the sibling comment’s point, I suppose it’s not weird in the Java ecosystem. The parquet-java project has a design where it deserializes Parquet fields into Java representations grabbed from _other_ projects rather than either having some kind of canonical self-representation in memory or acting as just an abstract codec. So, one of the most common things to do is apparently to use the “Avro” flavored serdes to get generic records in memory (note that the actual Avro serialization format is not involved with doing that; parquet-java just uses the classes from Avro as the in memory representations and deserializes Parquet into them). The whole approach seems a bit goofy; I’d expect the library to work as some kind of abstracted codec interface (requiring the in-memory representations to host Parquet, rather than the other way around - like how pandas hosts fastparquet in Python land) or provide a canonical object representation. Instead, it’s this in between where it has a grab bag of converters that transform Parquet to and from random object types pulled from elsewhere in the Java ecosystem.
      - amluto 2y ago
        I’d still like to see a clear explanation of where one can stick a Java class name in a Parquet file such that it ends up interpreted by the Avro codec. And I’m curious why it was fixed by making a list of allowed class names instead of disabling the entire mechanism.
  - tikhonj 2y ago
    Last time I tried to use the official Apache Parquet Java library, parsing "normal" Parquet files depended on parquet-avro because the library used Avro's GenericRecord class to represent rows from Parquet files with arbitrary schemas. So this problem would presumably affect any kind of Parquet parsing, even if there is absolutely no Avro actually involved. (Yes, this doesn't make sense; the official Parquet Java library had some of the worst code design I've had the misfortune to depend on.)
    - twoodfin 2y ago
      Indeed, given the massive interest Parquet has generated over the past 5 years, and its critical role in modern data infrastructure, I’ve been disappointed every time I’ve dug into the open source ecosystem around it for one reason or another. I think it’s revealing and unfortunate that everyone serious about Parquet, from DuckDB to Databricks, has written their own “codec”. Some recent frustrations on this front from the DuckDB folks: https://duckdb.org/2025/01/22/parquet-encodings.html https://duckdb.org/2025/01/22/parquet-encodings.html
      - dev_l1x_be 2y ago
        Unfortunately many of the big data libraries are like that and there is no motivation to fix these things. One example is the ORC Java libraries that had 100s of unnecessary dependencies while at the same time importing the filesystem into the format itself.
    - jeeeb 2y ago
      The Apache Arrow libraries are a good alternative for reading parquet files in Java. They provide a column oriented interface, rather than the ugly Avro stuff in the Apache Parquet library.
unit149 2y ago
[dead]
hypeatei 2y ago
Tangential, but there was a recent sandbox escape vulnerability in both Chrome and Firefox. The bug threads are still private, almost two weeks since it was disclosed and fixed. Very strange. https://bugzilla.mozilla.org/show_bug.cgi?id=1956398 https://bugzilla.mozilla.org/show_bug.cgi?id=1956398 https://issues.chromium.org/issues/405143032 https://issues.chromium.org/issues/405143032 https://www.cve.org/CVERecord?id=CVE-2025-2783 https://www.cve.org/CVERecord?id=CVE-2025-2783
- hovav 2y ago
  Standard operating procedure for both the Chrome [https://chromium.googlesource.com/chromium/src/+/HEAD/docs/security/faq.md https://chromium.googlesource.com/chromium/src/+/HEAD/docs/s...] and Firefox [https://www.mozilla.org/en-US/about/governance/policies/security-group/bugs/ https://www.mozilla.org/en-US/about/governance/policies/secu...] bug tracking systems. But the fix itself is public in both the Chrome [https://chromium.googlesource.com/chromium/src.git/+/36dbbf38697dd1e23ef8944bb9e57f6e0b3d41ec https://chromium.googlesource.com/chromium/src.git/+/36dbbf3...] and Firefox [https://github.com/mozilla/gecko-dev/commit/ac605820636c3b964542a2c0589af04a02235d00 https://github.com/mozilla/gecko-dev/commit/ac605820636c3b96...] source repos, and it makes pretty clear what the bug is.
  - benatkin 2y ago
    Looks like this one only applied to windows. Here’s a link to the diff: https://chromium.googlesource.com/chromium/src.git/+/36dbbf38697dd1e23ef8944bb9e57f6e0b3d41ec%5E%21/ https://chromium.googlesource.com/chromium/src.git/+/36dbbf3...
- deleted 2y ago
  [deleted]

by g-mork 2y ago

amluto 2y ago

Better link: https://github.com/apache/parquet-java/pull/3169 https://github.com/apache/parquet-java/pull/3169 If by “classic” you mean “using a language-dependent deserialization mechanism that is wildly unsafe”, I suppose. The surprising part is that Parquet is a fairly modern format with a real schema that is nominally language-independent. How on Earth did Java class names end up in the file format? Why is the parser willing to parse them at all? At most (at least by default), the parser should treat them as predefined strings that have semantics completely independent of any actual Java class.

bri3d 2y ago
This seems to come from parquet-avro, which looks to attempt to embed Avro in Parquet files and in the course of doing so, does silly Java reflection gymnastics. I don’t think “normal” parquet is affected.
- amluto 2y ago
  The documentation for all of this is atrocious. But if avro-in-parquet is a weird optional feature, it should be off by default! Parquet’s metadata is primarily in Thrift, not Avro, and it seems to me that no Avro should be involved in decoding Parquet files unless explicitly requested.
  - bri3d 2y ago
    To the sibling comment’s point, I suppose it’s not weird in the Java ecosystem. The parquet-java project has a design where it deserializes Parquet fields into Java representations grabbed from _other_ projects rather than either having some kind of canonical self-representation in memory or acting as just an abstract codec. So, one of the most common things to do is apparently to use the “Avro” flavored serdes to get generic records in memory (note that the actual Avro serialization format is not involved with doing that; parquet-java just uses the classes from Avro as the in memory representations and deserializes Parquet into them). The whole approach seems a bit goofy; I’d expect the library to work as some kind of abstracted codec interface (requiring the in-memory representations to host Parquet, rather than the other way around - like how pandas hosts fastparquet in Python land) or provide a canonical object representation. Instead, it’s this in between where it has a grab bag of converters that transform Parquet to and from random object types pulled from elsewhere in the Java ecosystem.
    - amluto 2y ago
      I’d still like to see a clear explanation of where one can stick a Java class name in a Parquet file such that it ends up interpreted by the Avro codec. And I’m curious why it was fixed by making a list of allowed class names instead of disabling the entire mechanism.
- tikhonj 2y ago
  Last time I tried to use the official Apache Parquet Java library, parsing "normal" Parquet files depended on parquet-avro because the library used Avro's GenericRecord class to represent rows from Parquet files with arbitrary schemas. So this problem would presumably affect any kind of Parquet parsing, even if there is absolutely no Avro actually involved. (Yes, this doesn't make sense; the official Parquet Java library had some of the worst code design I've had the misfortune to depend on.)
  - twoodfin 2y ago
    Indeed, given the massive interest Parquet has generated over the past 5 years, and its critical role in modern data infrastructure, I’ve been disappointed every time I’ve dug into the open source ecosystem around it for one reason or another. I think it’s revealing and unfortunate that everyone serious about Parquet, from DuckDB to Databricks, has written their own “codec”. Some recent frustrations on this front from the DuckDB folks: https://duckdb.org/2025/01/22/parquet-encodings.html https://duckdb.org/2025/01/22/parquet-encodings.html
    - dev_l1x_be 2y ago
      Unfortunately many of the big data libraries are like that and there is no motivation to fix these things. One example is the ORC Java libraries that had 100s of unnecessary dependencies while at the same time importing the filesystem into the format itself.
  - jeeeb 2y ago
    The Apache Arrow libraries are a good alternative for reading parquet files in Java. They provide a column oriented interface, rather than the ugly Avro stuff in the Apache Parquet library.

unit149 2y ago

[dead]

hypeatei 2y ago

Tangential, but there was a recent sandbox escape vulnerability in both Chrome and Firefox. The bug threads are still private, almost two weeks since it was disclosed and fixed. Very strange. https://bugzilla.mozilla.org/show_bug.cgi?id=1956398 https://bugzilla.mozilla.org/show_bug.cgi?id=1956398 https://issues.chromium.org/issues/405143032 https://issues.chromium.org/issues/405143032 https://www.cve.org/CVERecord?id=CVE-2025-2783 https://www.cve.org/CVERecord?id=CVE-2025-2783

hovav 2y ago
Standard operating procedure for both the Chrome [https://chromium.googlesource.com/chromium/src/+/HEAD/docs/security/faq.md https://chromium.googlesource.com/chromium/src/+/HEAD/docs/s...] and Firefox [https://www.mozilla.org/en-US/about/governance/policies/security-group/bugs/ https://www.mozilla.org/en-US/about/governance/policies/secu...] bug tracking systems. But the fix itself is public in both the Chrome [https://chromium.googlesource.com/chromium/src.git/+/36dbbf38697dd1e23ef8944bb9e57f6e0b3d41ec https://chromium.googlesource.com/chromium/src.git/+/36dbbf3...] and Firefox [https://github.com/mozilla/gecko-dev/commit/ac605820636c3b964542a2c0589af04a02235d00 https://github.com/mozilla/gecko-dev/commit/ac605820636c3b96...] source repos, and it makes pretty clear what the bug is.
- benatkin 2y ago
  Looks like this one only applied to windows. Here’s a link to the diff: https://chromium.googlesource.com/chromium/src.git/+/36dbbf38697dd1e23ef8944bb9e57f6e0b3d41ec%5E%21/ https://chromium.googlesource.com/chromium/src.git/+/36dbbf3...
deleted 2y ago
[deleted]