5 ms·
Is this relevant/are there situations where people nest JSON even 100x/1000x? I have not come across something like that.
by et2o 7y ago
Is this relevant/are there situations where people nest JSON even 100x/1000x? I have not come across something like that.
- shadowgovt 7y agoProcedurally-generated JSON can nest data as deep as it needs to to describe arbitrarily-complex data, but I'd usually consider it an indicator that a problem thought to be simple has overflowed the bounds of its initial assumptions when it happens.
- fireflies_ 7y agoYou probably wouldn't see it in "natural" use but it's a potential attack vector if you ever deserialize arbitrary JSON (e.g. JSON bodies in API requests). Of course you should have general limits that would catch these things anyway...
- skykooler 7y agoGeneral limits for what? If you can segfault a program with 28kb of JSON, that's far below the maximum many APIs will return, so you'd need to do some sort of pre-parsing of the JSON to determine the nesting level before parsing it...
- vbezhenar 7y agoFor example Java won't segfault. It'll throw stackoverflow exception which will be caught and served with some standard error like HTTP 500. It shouldn't affect other queries.
- tracker1 7y agoWell, recursion tests would be more for input... I usually limit all POST data to 5-10k and don't like to go over that. I also tend to use chunked uploads, that are a little slower, but serve to limit the size and time of any inbound work.
- bargle0 7y agoThe question isn't necessarily what might be found in nature, but what might be intentionally constructed in order to attack something. Without knowing more, the segfault in the C++ library is somewhat alarming.
- kibwen 7y agoOn the topic of malicious input, the serde_json example seems to be deliberately limiting recursion depth in order to head off DoS attacks: "serde_json has a recursion limit to protect from malicious clients sending a deeply recursive structure and DOS-ing a server. However, it does have an optional feature unbounded_depth that disables this protection. It still uses the stack though, so it will still eventually blow the stack instead of using all available memory" https://github.com/lovasoa/bad_json_parsers/issues/7 https://github.com/lovasoa/bad_json_parsers/issues/7 This begins to suggest that it may be downright desirable for servers to toss out deeply-nested JSON, regardless of the JSON specification.
- vbezhenar 7y agoWhy not implement parsing without recursion? It doesn't look like a particularly hard task for me. Why restrict users with artificial limitations?
- Tomte 7y agoBecause even if the stack doesn't run out, it still processes for a long time. Non-recursion doesn't help you there, you need to look at every byte, after all.
- shadowgovt 7y agoAnd it's worth noting that many JSON parsers assume the problem domain is "shipping data over a network," where the input may be malicious. If the non-recursive parser has an N^2 piece of algorithm embedded in it, its ability to consume large data structures becomes a DOS liability.
- 7y ago
- vbezhenar 7y agoIf you're representing tree data structure in a natural way, you can easily run into this limitation on a real world data.
- adrianN 7y agoI have seen JSON output with thousands of levels of nesting. It was generated by a sort of compiler tool that compiled to JSON.