4 ms·

There was a nice article [0] about bloated edge cases libraries (discussion [1]). Sometimes, it's just not the responsibility of the library. Trying to handle

There was a nice article [0] about bloated edge cases libraries (discussion [1]). Sometimes, it's just not the responsibility of the library. Trying to handle every possible errors is a quick way to complexity. [0]: https://43081j.com/2025/09/bloat-of-edge-case-libraries https://43081j.com/2025/09/bloat-of-edge-case-libraries [1]: https://news.ycombinator.com/item?id=45319399 https://news.ycombinator.com/item?id=45319399

layer8 1y ago
The problem in the present case is that the caller is not made aware of the limitation, so can’t be expected to prevent passing unsupported input, and has no way to handle the overflow case after the fact.
- skydhash 1y ago
  Do you not review libraries you add to your project? A quick scan of the issues page if it's on a forge? Or just reading through the code if it's small enough (or select functions)? Code is the ultimate specification. I don't trust the docs if the behavior is different from what it's saying (or more often fails to mention). And anything that deals with recursive structures (or looping without a clear counter and checks) is my one of the first candidate for checks. > has no way to handle the overflow case after the fact. Fork/Vendor the code and add your assertions.
  - layer8 1y ago
    Obviously I just did review it, and my conclusion was to not use that code. In the spirit of the article you linked, I’d rather write my own version.
  - jama211 1y ago
    If it has limitations they should be documented though right? especially if they’re security concerns.
  - knowitnone2 1y ago
    If you review libraries, why do you need to quick scan the issues? You would have already identified all the issues right? Right?
    - jama211 1y ago
      Hahaha well said
klysm 1y ago
Strongly disagree here because JSON can come from untrusted sources and this has security implications. It's not the same kind of problem that the bloat article discusses where you just have bad contracts on interfaces.
- leptons 1y ago
  JSON does not necessarily come from untrusted sources if you control the entire system. Not everything needs to be absolutely 100% secure so long as you control the system. If you are opening the system to the public, then sure, you should strive for security, but that isn't always necessary in projects that are not processing public input. Here's an example - I once coded a limited JSON parser in assembly language. I did not attempt to make it secure in any way. The purpose was to parse control messages sent over a serial port connection to an embedded CPU that controlled a small motor to rotate a camera and snap a photo. There was simply no way for any "untrusted" JSON to enter the system. It worked perfectly and nothing could ever be compromised by having a very simple JSON parser in the embedded device controlling the motor.
  - userbinator 1y ago
    You probably didn't control the other end, as otherwise you would've used something more sane than JSON?
    - leptons 1y ago
      I controlled both ends. There is nothing "insane" about JSON. It's used far and wide for many purposes. The system sending the JSON was based on Nodejs, so it was pretty natural to use JSON. And I did it with JSON just because I wanted to. I'd have had to invent some other protocol to do it anyway, and I didn't feel like reinventing the wheel when it was quite simple to write a basic JSON parser in assembly language, which is what I am comfortable with on the embedded system (been coding assembly for 40 years).
      - userbinator 1y ago
flykespice 1y ago
There is no easy way out when you're working with C: either you handle all possible UB cases with exhaustive checks, or you move on to another language. (TIP: choose the latter)
- jeroenhd 1y ago
  Very few programming languages default to checked increments. Most Rust or Java programmers would make the same mistake. Writing a function to do a checked addition like in other languages isn't exactly difficult, either.
  - paulddraper 1y ago
    Yes but those languages have defined overflow.
    - cozzyd 1y ago
      -fwrapv
  - deleted 1y ago
    [deleted]
  - koito17 1y ago
    > Most Rust or Java programmers would make the same mistake. Detecting these mistakes in Rust is not too difficult. In debug builds, integer overflow triggers a panic[1]. Additionally, clippy (the official linter of Rust), has a rule[2] to detect this mistake. [1] https://doc.rust-lang.org/book/ch03-02-data-types.html#integer-overflow https://doc.rust-lang.org/book/ch03-02-data-types.html#integ... [2] https://rust-lang.github.io/rust-clippy/master/index.html#arithmetic_side_effects https://rust-lang.github.io/rust-clippy/master/index.html#ar...
    - account42 1y ago
      You can also just use ubsan if you want the runtime checks for overflows in C/C++, no need to switch languages for this.
FooBarBizBazz 1y ago
This might be the right attitude for a max function written in JavaScript, where the calling code has some control over the inputs. It's the wrong attitude for a JSON parser written in C, unless you like to get owned.
- account42 1y ago
  It's entirely reasonable for a tiny JSON parser written in C to not deal with JSON files over 0x7fffffff bytes long.
paulddraper 1y ago
The amount of untrusted JSON I parse is very high. UB is bad.
RossBencina 1y ago
> Sometimes, it's just not the responsibility of the library. Sometimes. In this case, where the library is a parser that is written in C. I think it is reasonable to expect the library to handle all possible inputs. Even corner cases like this which are unlikely to be encountered in common practice. This is not "bloat" it is correctness. In C, this kind of bug is capable of being exploited. Sure, many users of this lib won't be using it in exposed cases, but sooner or later the lib will end up in some widely-used internet-facing codebase. As others have said, the fix could be as simple as bailing once the input size exceeds 1GB. Or it could be fine-grained. Either-way the fix would not "bloat" the codebase. And yes, I'm well aware of the single-file C library movement. I am a fan.

layer8 1y ago

The problem in the present case is that the caller is not made aware of the limitation, so can’t be expected to prevent passing unsupported input, and has no way to handle the overflow case after the fact.

skydhash 1y ago
Do you not review libraries you add to your project? A quick scan of the issues page if it's on a forge? Or just reading through the code if it's small enough (or select functions)? Code is the ultimate specification. I don't trust the docs if the behavior is different from what it's saying (or more often fails to mention). And anything that deals with recursive structures (or looping without a clear counter and checks) is my one of the first candidate for checks. > has no way to handle the overflow case after the fact. Fork/Vendor the code and add your assertions.
- layer8 1y ago
  Obviously I just did review it, and my conclusion was to not use that code. In the spirit of the article you linked, I’d rather write my own version.
- jama211 1y ago
  If it has limitations they should be documented though right? especially if they’re security concerns.
- knowitnone2 1y ago
  If you review libraries, why do you need to quick scan the issues? You would have already identified all the issues right? Right?
  - jama211 1y ago
    Hahaha well said

klysm 1y ago

Strongly disagree here because JSON can come from untrusted sources and this has security implications. It's not the same kind of problem that the bloat article discusses where you just have bad contracts on interfaces.

leptons 1y ago
JSON does not necessarily come from untrusted sources if you control the entire system. Not everything needs to be absolutely 100% secure so long as you control the system. If you are opening the system to the public, then sure, you should strive for security, but that isn't always necessary in projects that are not processing public input. Here's an example - I once coded a limited JSON parser in assembly language. I did not attempt to make it secure in any way. The purpose was to parse control messages sent over a serial port connection to an embedded CPU that controlled a small motor to rotate a camera and snap a photo. There was simply no way for any "untrusted" JSON to enter the system. It worked perfectly and nothing could ever be compromised by having a very simple JSON parser in the embedded device controlling the motor.
- userbinator 1y ago
  You probably didn't control the other end, as otherwise you would've used something more sane than JSON?
  - leptons 1y ago
    I controlled both ends. There is nothing "insane" about JSON. It's used far and wide for many purposes. The system sending the JSON was based on Nodejs, so it was pretty natural to use JSON. And I did it with JSON just because I wanted to. I'd have had to invent some other protocol to do it anyway, and I didn't feel like reinventing the wheel when it was quite simple to write a basic JSON parser in assembly language, which is what I am comfortable with on the embedded system (been coding assembly for 40 years).
    - userbinator 1y ago

flykespice 1y ago

There is no easy way out when you're working with C: either you handle all possible UB cases with exhaustive checks, or you move on to another language. (TIP: choose the latter)

jeroenhd 1y ago
Very few programming languages default to checked increments. Most Rust or Java programmers would make the same mistake. Writing a function to do a checked addition like in other languages isn't exactly difficult, either.
- paulddraper 1y ago
  Yes but those languages have defined overflow.
  - cozzyd 1y ago
    -fwrapv
- deleted 1y ago
  [deleted]
- koito17 1y ago
  > Most Rust or Java programmers would make the same mistake. Detecting these mistakes in Rust is not too difficult. In debug builds, integer overflow triggers a panic[1]. Additionally, clippy (the official linter of Rust), has a rule[2] to detect this mistake. [1] https://doc.rust-lang.org/book/ch03-02-data-types.html#integer-overflow https://doc.rust-lang.org/book/ch03-02-data-types.html#integ... [2] https://rust-lang.github.io/rust-clippy/master/index.html#arithmetic_side_effects https://rust-lang.github.io/rust-clippy/master/index.html#ar...
  - account42 1y ago
    You can also just use ubsan if you want the runtime checks for overflows in C/C++, no need to switch languages for this.

FooBarBizBazz 1y ago

This might be the right attitude for a max function written in JavaScript, where the calling code has some control over the inputs. It's the wrong attitude for a JSON parser written in C, unless you like to get owned.

account42 1y ago
It's entirely reasonable for a tiny JSON parser written in C to not deal with JSON files over 0x7fffffff bytes long.

paulddraper 1y ago

The amount of untrusted JSON I parse is very high. UB is bad.

RossBencina 1y ago

> Sometimes, it's just not the responsibility of the library. Sometimes. In this case, where the library is a parser that is written in C. I think it is reasonable to expect the library to handle all possible inputs. Even corner cases like this which are unlikely to be encountered in common practice. This is not "bloat" it is correctness. In C, this kind of bug is capable of being exploited. Sure, many users of this lib won't be using it in exposed cases, but sooner or later the lib will end up in some widely-used internet-facing codebase. As others have said, the fix could be as simple as bailing once the input size exceeds 1GB. Or it could be fine-grained. Either-way the fix would not "bloat" the codebase. And yes, I'm well aware of the single-file C library movement. I am a fan.