5 ms·

Well, one special edge-case would be where you only need to parse some extremely tiny subset of JSON (for example: you only need to parse dictionaries whose key

• aliceryhl 6y ago
  There are fully correct JSON parsers you aren't going to beat, even if you implement a subset. [1] [1]: https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of-json-per-second/ https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of...
  • Ma8ee 6y ago
    No, I won't beat them. But if it a limited subset that I can implement with twenty lines straightforward code, that will often be cheaper. I've been on projects where they imported xml-parsers many times bigger than the rest of the whole codebase just to send a well formatted order number.
    • dogma1138 6y ago
      And what happens when your parsing needs to be expanded?
      • fernandotakai 6y ago
        taking in consideration how business work, in a few years you are going to have a full parser in your hands.
        • dogma1138 6y ago
          With all the technical debt associated with it, which is the problem basing your project on a dependency that would allow you to easily scale and add features is a huge benefit. This is like saying you should roll your own crypto because you only need to do a very limited sub set of crypto operations so why use something like NaCl or Tink.
          • samus 6y ago
            Encryption is a terrible edge case. If you are forced to half-ass encryption, you should seriously question the project requirements. Bad encryption can be worse than none at all. Things won't end well if data security is treated as a detail.
      • Ma8ee 6y ago
        Then you reevaluate the situation. YAGNI is true here too.
        • dogma1138 6y ago
          YANGI is nice but when the PM asks you why it would take two months to accept a new JSON format from a client and you’ll answer well because we didn’t want to use an industry standard fully functional and vetted JSON parser so we essentially wrote our own edge case parser we both know how that conversation will end. And YANGI doesn’t have anything against dependencies.
          • Ma8ee 6y ago
            I said, then you reevaluate the situation. When there are new requirements you do a quick estimate if you should add four new lines to the existing 20 or if it is worth to switch to an external library. 4 new lines to the 20, just add them to the core. But if this is regularly occurring that you have to add things, or requirements affecting this particular little parser that was supposed to be simple and static isn’t, then you should probably change your decision and use the library. But you do that only then. Because chances are that with your approach you are going to drag along a large generic library that you only use a tiny fraction of. And that also has costs. In particular if your immediate impulse always is to add another library instead of writing things yourself.
        • mro_name 6y ago
          That's IMHO key – solve problems once you know them, not earlier. Old idea, also core to XP.
    • abiogenesis 6y ago
      What do you mean by cheaper? By using a third party library you are writing twenty lines less code, so it's cheaper in that aspect. There are probably libraries that are faster than your twenty lines of un-optimized code, so it's cheaper as far as computing resources are considered too. The only time it could matter is when you ship the code to the client through the wire (such as a Javascript bundle).
      • Retric 6y ago
        You can’t use a library with zero lines of code. On top of this library’s always have development overhead outside of the code you write. Ex: What version number should you use? Did the latest version break something? Did the old version break something on the latest compiler? Etc etc.
      • Ma8ee 6y ago
        It’s cheaper in the sense that it is faster to write and maintain those 20 lines of code. Because someone has to evaluate the library, understand it well enough to actually call it and then make sure it stays up to date. And often there are a few lines of code to translate your data into a form that the library requires etc.
        • BigJono 6y ago
          Plus for every developer to come, one call to an external library usually also means 30 pages of documentation to trawl through if they ever want to change anything, 29.9 of which is completely irrelevant to whatever your narrow use case is. That's the real cost. The size of the code means absolutely nothing.
    • benibela 6y ago
      xml and json are different beasts A json parser can probably be implemented in an afternoon. But a conformant xml parser can take months. There are some weird things in xml, for example, this is correct xml: <?xml version="1.0"?><!DOCTYPE abc[<?abc >]]<abc><abc/>]>>?>]><?x?><a/><?x <(x)>>?> And for external entities, you need an http client in the xml parser, although it is probably better to not support that part of the standard.
      • Ma8ee 6y ago
        And we didn’t need a conformant xml parser, which I know is huge and complex.
  • tnorthcutt 6y ago
    Indeed: https://github.com/qntm/fastjson https://github.com/qntm/fastjson
  • flukus 6y ago
    And it's going to take more than an afternoon to evaluate these parsers. You have to look at the options, evaluate the API, evaluate if they're stable and supported, evaluate if they integrate well with you're project, evaluate any dependencies they might have, etc. Then you need a plan to manage these dependencies long term. If you're needs can be solved adequately by strtok then that's a far simpler and more maintainable solution that can be knocked out in an afternoon.
• Izkata 6y ago
  Your example is more apt than intended: That's not valid json, which only allows string keys. If you use a library it'll either barf now or later when they fix it, so if you're forced to work with an API like that and can't change it, a custom parser is really the only way to go.
  • JackFr 6y ago
    Well the good libraries have option flags that will allow you to handle JSON as found in the wild.
    • majewsky 6y ago
      That's not JSON, though. It's absolutely something else. Maybe a JS snippet. Maybe YAML. Definitely not JSON, though. (Some JSON libraries do have option flags, but usually it's about whether, during deserializing into a known type, unknown fields are an error or silently ignored. Or whether C-style comments are an error or considered as whitespace.)
      • JackFr 6y ago
        It may not be JSON but it’s out there in the wild. A lot. Let’s say I’m scraping a website. I can: 1) complain to the owner that “it’s not JSON”. 2) write a parser for a syntax that has no spec, (it’s not JSON, but it sure looks an awful lot like JSON with unquoted keys.) or 3) Set ALLOW_UNQUOTED_FIELD_NAMES to true in the Jackson library.
      • Izkata 6y ago
        > Maybe a JS snippet. While acceptable, also misleading: JS only does string keys, but unlike JSON it'll convert whatever it's given into strings. Not a problem most of the time, since it'll do the same conversion for both accessing and setting, but good to be aware of if you're doing something like iterating Object.keys()
        • jimmynest 6y ago
• TeMPOraL 6y ago
  I agree. > Of course, you might say, inevitably feature-creep will expand the list of things your parser needs to parse If you've done your parser correctly, you'll be able to replace its implementation with the new dependency, with little to no need for extra refactoring in the rest of the codebase.
• hinkley 6y ago
  You can also apply YAGNI to 'do we need our own custom parser'? You don't know what your requirements are. The customers haven't told you yet. If you pick a library with a straightforward interface, especially one that isn't too opinionated, you can always drop in a custom implementation later on. Frameworks, not so much (but that cuts both ways; the people who will write libraries often love writing frameworks too)