11 ms·

My rule is, don't use a dependency to implement your core business. Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to

My rule is, don't use a dependency to implement your core business. Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to supporting for its entire lifetime -- JSON parsing code? All the code you write and support should be directly tied to what you as a business decide are your fundamental value propositions. Everything else you write is just fat waiting to be cut by someone who knows how to write a business case. To be clear, this is about the lifetime support of code. It's very, very rare that code can be written once and never touched. But that long tail of support eats up time and money, and is almost always discounted in these conversations. I don't even care that Jackson JSON parsing has years of work behind it, when I can hack together a JSON parser in a day. I care that Jackson will continue to improve their offering without any further input, while that's not true of my version.

xamuel 6y ago
Well, one special edge-case would be where you only need to parse some extremely tiny subset of JSON (for example: you only need to parse dictionaries whose keys and values are positive integers, like {1:2,3:4}). Then, depending how expensive the full json parser is, it might be worth your while just writing the limited parser yourself. Of course, you might say, inevitably feature-creep will expand the list of things your parser needs to parse, but that's not a law of physics. Sometimes in certain limited, well-defined projects, it really is true that YAGNI.
- aliceryhl 6y ago
  There are fully correct JSON parsers you aren't going to beat, even if you implement a subset. [1] [1]: https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of-json-per-second/ https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of...
  - Ma8ee 6y ago
    No, I won't beat them. But if it a limited subset that I can implement with twenty lines straightforward code, that will often be cheaper. I've been on projects where they imported xml-parsers many times bigger than the rest of the whole codebase just to send a well formatted order number.
    - dogma1138 6y ago
      And what happens when your parsing needs to be expanded?
      - fernandotakai 6y ago
        taking in consideration how business work, in a few years you are going to have a full parser in your hands.
        dogma1138 6y ago
        With all the technical debt associated with it, which is the problem basing your project on a dependency that would allow you to easily scale and add features is a huge benefit. This is like saying you should roll your own crypto because you only need to do a very limited sub set of crypto operations so why use something like NaCl or Tink.
        samus 6y ago
        Encryption is a terrible edge case. If you are forced to half-ass encryption, you should seriously question the project requirements. Bad encryption can be worse than none at all. Things won't end well if data security is treated as a detail.
      - Ma8ee 6y ago
        Then you reevaluate the situation. YAGNI is true here too.
        dogma1138 6y ago
        YANGI is nice but when the PM asks you why it would take two months to accept a new JSON format from a client and you’ll answer well because we didn’t want to use an industry standard fully functional and vetted JSON parser so we essentially wrote our own edge case parser we both know how that conversation will end. And YANGI doesn’t have anything against dependencies.
  - tnorthcutt 6y ago
    Indeed: https://github.com/qntm/fastjson https://github.com/qntm/fastjson
  - flukus 6y ago
    And it's going to take more than an afternoon to evaluate these parsers. You have to look at the options, evaluate the API, evaluate if they're stable and supported, evaluate if they integrate well with you're project, evaluate any dependencies they might have, etc. Then you need a plan to manage these dependencies long term. If you're needs can be solved adequately by strtok then that's a far simpler and more maintainable solution that can be knocked out in an afternoon.
- Izkata 6y ago
  Your example is more apt than intended: That's not valid json, which only allows string keys. If you use a library it'll either barf now or later when they fix it, so if you're forced to work with an API like that and can't change it, a custom parser is really the only way to go.
  - JackFr 6y ago
    Well the good libraries have option flags that will allow you to handle JSON as found in the wild.
    - majewsky 6y ago
      That's not JSON, though. It's absolutely something else. Maybe a JS snippet. Maybe YAML. Definitely not JSON, though. (Some JSON libraries do have option flags, but usually it's about whether, during deserializing into a known type, unknown fields are an error or silently ignored. Or whether C-style comments are an error or considered as whitespace.)
      - JackFr 6y ago
        It may not be JSON but it’s out there in the wild. A lot. Let’s say I’m scraping a website. I can: 1) complain to the owner that “it’s not JSON”. 2) write a parser for a syntax that has no spec, (it’s not JSON, but it sure looks an awful lot like JSON with unquoted keys.) or 3) Set ALLOW_UNQUOTED_FIELD_NAMES to true in the Jackson library.
      - Izkata 6y ago
        > Maybe a JS snippet. While acceptable, also misleading: JS only does string keys, but unlike JSON it'll convert whatever it's given into strings. Not a problem most of the time, since it'll do the same conversion for both accessing and setting, but good to be aware of if you're doing something like iterating Object.keys()
        jimmynest 6y ago
- TeMPOraL 6y ago
  I agree. > Of course, you might say, inevitably feature-creep will expand the list of things your parser needs to parse If you've done your parser correctly, you'll be able to replace its implementation with the new dependency, with little to no need for extra refactoring in the rest of the codebase.
- hinkley 6y ago
  You can also apply YAGNI to 'do we need our own custom parser'? You don't know what your requirements are. The customers haven't told you yet. If you pick a library with a straightforward interface, especially one that isn't too opinionated, you can always drop in a custom implementation later on. Frameworks, not so much (but that cuts both ways; the people who will write libraries often love writing frameworks too)
hungry_haibt 6y ago
Great rule. I was wondering, how do you manage updating the Jackson JSON parsing package. What if you have 100 such packages and they get updated weekly with breaking changes ?
- Floegipoky 6y ago
  > What if you have 100 such packages and they get updated weekly with breaking changes? The solution to that is simple, stop using node.js ;)
  - hungry_haibt 6y ago
    Only good solution in this thread :)
- triceratops 6y ago
  Update all your dependencies periodically - monthly, quarterly, whatever. Freeze dependencies in the meanwhile.
  - munk-a 6y ago
    If you're in a larger corporater environment this can also be used to create some predictable labour needs - create a seasonal updating taskforce so that the business get a more transparent view of how much labour is being sunk into maintaining these, break it down into specific dependencies if you've got one or two that you think are particularly expensive- showing after the fact labour numbers from one season may motivate sane inhousing for next season.
- markstos 6y ago
  Only update dependencies when your code requires the new version, depends on a bug fix or it fixes a security vulnerability. Otherwise, continue using the same version. Have good test coverage to catch bugs that may originate in dependencies and subscribe to a third-party service to track vulnerabilities in your dependencies.
nayuki 6y ago
> don't use a dependency to implement your core business In logic language, you're saying "If X is your core business, don't outsource X". > Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to supporting for its entire lifetime -- JSON parsing code? All the code you write and support should be directly tied to what you as a business decide are your fundamental value propositions. Everything else you write is just fat waiting to be cut by someone who knows how to write a business case. The rest of your argument is interpreted as "If X is not your core business, don't in-house X". These two logical implication statements are not equivalents of each other, but are converses. Casual language often conflates If, Only-If, and If-And-Only-If.
- Closi 6y ago
  I think they both follow similar thinking. You should spend time implementing your core business implies that you shouldn’t spend time implementing things that aren’t in your core business, otherwise the first statement is pretty useless.
- pests 6y ago
  Maybe not equivalents but definitely two arguments for his core point.
- jaggederest 6y ago
  core business = C outsource = O Object = x x ∉ C ↔ O(x) If the symbols don't show up: if-and-only-if x is not in C, O(x)
  - nothal 6y ago
    If I wanted to learn more about rigorous, non-elementary logic, do you have a recommended resource? I've taken a course in intro level probability theory which covered it generally and another course that built on it lightly but nothing rigorous and I am wooed by how concise things become in a logical form.
    - renewiltord 6y ago
hans789 6y ago
Quite agree, every single line of code written requires lifetime support. Code adds up and reduces productivity gradually, so only write code in core business logics.
jyunkess 6y ago
That's true. Beside _lifetime support_, working on that core business feature make us _understand_ deeply about the that feature. I've seen people integrate dependency for their core business. It helped to get started fast, but will create a blockage that required understanding deeper to overcome
cellular 6y ago
But "it's a good problem to have"!
KajMagnus 6y ago
I think a JSON parser is not a good example though — takes longer than a few hours / an afternoon, to write a JSON parser, add tests, fix bugs, corner cases. More like a week, or weeks, ... ... Look, a tiny json parser — Not an afternoon project: https://github.com/rafagafe/tiny-json/blob/master/tiny-json.c https://github.com/rafagafe/tiny-json/blob/master/tiny-json.... And a question about small JSON parsers — didn't see any afternoon projects among the answers: https://stackoverflow.com/questions/6061172/smallest-less-internal-code-up-to-spec-json-parser-in-java/6062466 https://stackoverflow.com/questions/6061172/smallest-less-in... I suppose a JSON parser was just an example. Made the whole answer sound weird to me though :- ) when the blog is about afternoon-projects and then a reply is about a week(s), could be month(s), long project.
- signaru 6y ago
  There's a fair middle ground when the dependency in itself doesn't have dependencies, and is small enough with a permissive license such that the entirety of its code can be dropped in to your project. Especially for very specific functionalities. I have used such tiny xml parsers, and I'm not affected by the fact that my copy is no longer the latest version. Its not so far from copying and pasting snippets of existing code.
- jdmichal 6y ago
  > I think a thing is not a good example though — takes longer than a few hours / an afternoon, to write a thing, add tests, fix bugs, corner cases. More like a week, or weeks, ... You're making my point for me. This is exactly what I meant by the lifetime of support you're signing up for by writing lines of code. Once you write that code, you're now in the business of supporting that code. Was that a good decision for your business?
- imtringued 6y ago
  Same with CSV. It looks easy, but it isn't. I've never seen anyone who writes their own CSV parser actually implement features necessary to conform to the standard like quoting and escape sequences. The end result is software that breaks when delimiters or quotes appear in user input. Honestly, I prefer xlsx spreadsheets because of that. Nobody fools themselves into implementing the parser or serializer for the format themselves. The only tiny pitfall with them is when people create spreadsheets manually in excel and write numbers as text, but parsing strings to numbers is absolutely trivial. You have to do that with CSV anyway.
SeriousM 6y ago
So you're saying that I should implement my own ormapper just because my product is using a database? And even this is not thee case, writing everything yourself will end up in your own hands. No Bugfixes, no patches or improvements without spending man work. I've worked in such a company and it was a mess accompanied by dev leaders who's to proud of their code to allow any change.
- jdmichal 6y ago
  I'm confused by your response. Is your core business mapping objects to databases? As in, that's what you get paid for? If not, my heuristic is that you should not be writing an ORM tool.

xamuel 6y ago

Well, one special edge-case would be where you only need to parse some extremely tiny subset of JSON (for example: you only need to parse dictionaries whose keys and values are positive integers, like {1:2,3:4}). Then, depending how expensive the full json parser is, it might be worth your while just writing the limited parser yourself. Of course, you might say, inevitably feature-creep will expand the list of things your parser needs to parse, but that's not a law of physics. Sometimes in certain limited, well-defined projects, it really is true that YAGNI.

aliceryhl 6y ago
There are fully correct JSON parsers you aren't going to beat, even if you implement a subset. [1] [1]: https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of-json-per-second/ https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of...
- Ma8ee 6y ago
  No, I won't beat them. But if it a limited subset that I can implement with twenty lines straightforward code, that will often be cheaper. I've been on projects where they imported xml-parsers many times bigger than the rest of the whole codebase just to send a well formatted order number.
  - dogma1138 6y ago
    And what happens when your parsing needs to be expanded?
    - fernandotakai 6y ago
      taking in consideration how business work, in a few years you are going to have a full parser in your hands.
      - dogma1138 6y ago
        With all the technical debt associated with it, which is the problem basing your project on a dependency that would allow you to easily scale and add features is a huge benefit. This is like saying you should roll your own crypto because you only need to do a very limited sub set of crypto operations so why use something like NaCl or Tink.
        samus 6y ago
        Encryption is a terrible edge case. If you are forced to half-ass encryption, you should seriously question the project requirements. Bad encryption can be worse than none at all. Things won't end well if data security is treated as a detail.
    - Ma8ee 6y ago
      Then you reevaluate the situation. YAGNI is true here too.
      - dogma1138 6y ago
        YANGI is nice but when the PM asks you why it would take two months to accept a new JSON format from a client and you’ll answer well because we didn’t want to use an industry standard fully functional and vetted JSON parser so we essentially wrote our own edge case parser we both know how that conversation will end. And YANGI doesn’t have anything against dependencies.
- tnorthcutt 6y ago
  Indeed: https://github.com/qntm/fastjson https://github.com/qntm/fastjson
- flukus 6y ago
  And it's going to take more than an afternoon to evaluate these parsers. You have to look at the options, evaluate the API, evaluate if they're stable and supported, evaluate if they integrate well with you're project, evaluate any dependencies they might have, etc. Then you need a plan to manage these dependencies long term. If you're needs can be solved adequately by strtok then that's a far simpler and more maintainable solution that can be knocked out in an afternoon.
Izkata 6y ago
Your example is more apt than intended: That's not valid json, which only allows string keys. If you use a library it'll either barf now or later when they fix it, so if you're forced to work with an API like that and can't change it, a custom parser is really the only way to go.
- JackFr 6y ago
  Well the good libraries have option flags that will allow you to handle JSON as found in the wild.
  - majewsky 6y ago
    That's not JSON, though. It's absolutely something else. Maybe a JS snippet. Maybe YAML. Definitely not JSON, though. (Some JSON libraries do have option flags, but usually it's about whether, during deserializing into a known type, unknown fields are an error or silently ignored. Or whether C-style comments are an error or considered as whitespace.)
    - JackFr 6y ago
      It may not be JSON but it’s out there in the wild. A lot. Let’s say I’m scraping a website. I can: 1) complain to the owner that “it’s not JSON”. 2) write a parser for a syntax that has no spec, (it’s not JSON, but it sure looks an awful lot like JSON with unquoted keys.) or 3) Set ALLOW_UNQUOTED_FIELD_NAMES to true in the Jackson library.
    - Izkata 6y ago
      > Maybe a JS snippet. While acceptable, also misleading: JS only does string keys, but unlike JSON it'll convert whatever it's given into strings. Not a problem most of the time, since it'll do the same conversion for both accessing and setting, but good to be aware of if you're doing something like iterating Object.keys()
      - jimmynest 6y ago
TeMPOraL 6y ago
I agree. > Of course, you might say, inevitably feature-creep will expand the list of things your parser needs to parse If you've done your parser correctly, you'll be able to replace its implementation with the new dependency, with little to no need for extra refactoring in the rest of the codebase.
hinkley 6y ago
You can also apply YAGNI to 'do we need our own custom parser'? You don't know what your requirements are. The customers haven't told you yet. If you pick a library with a straightforward interface, especially one that isn't too opinionated, you can always drop in a custom implementation later on. Frameworks, not so much (but that cuts both ways; the people who will write libraries often love writing frameworks too)

hungry_haibt 6y ago

Great rule. I was wondering, how do you manage updating the Jackson JSON parsing package. What if you have 100 such packages and they get updated weekly with breaking changes ?

Floegipoky 6y ago
> What if you have 100 such packages and they get updated weekly with breaking changes? The solution to that is simple, stop using node.js ;)
- hungry_haibt 6y ago
  Only good solution in this thread :)
triceratops 6y ago
Update all your dependencies periodically - monthly, quarterly, whatever. Freeze dependencies in the meanwhile.
- munk-a 6y ago
  If you're in a larger corporater environment this can also be used to create some predictable labour needs - create a seasonal updating taskforce so that the business get a more transparent view of how much labour is being sunk into maintaining these, break it down into specific dependencies if you've got one or two that you think are particularly expensive- showing after the fact labour numbers from one season may motivate sane inhousing for next season.
markstos 6y ago
Only update dependencies when your code requires the new version, depends on a bug fix or it fixes a security vulnerability. Otherwise, continue using the same version. Have good test coverage to catch bugs that may originate in dependencies and subscribe to a third-party service to track vulnerabilities in your dependencies.

nayuki 6y ago

> don't use a dependency to implement your core business In logic language, you're saying "If X is your core business, don't outsource X". > Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to supporting for its entire lifetime -- JSON parsing code? All the code you write and support should be directly tied to what you as a business decide are your fundamental value propositions. Everything else you write is just fat waiting to be cut by someone who knows how to write a business case. The rest of your argument is interpreted as "If X is not your core business, don't in-house X". These two logical implication statements are not equivalents of each other, but are converses. Casual language often conflates If, Only-If, and If-And-Only-If.

Closi 6y ago
I think they both follow similar thinking. You should spend time implementing your core business implies that you shouldn’t spend time implementing things that aren’t in your core business, otherwise the first statement is pretty useless.
pests 6y ago
Maybe not equivalents but definitely two arguments for his core point.
jaggederest 6y ago
core business = C outsource = O Object = x x ∉ C ↔ O(x) If the symbols don't show up: if-and-only-if x is not in C, O(x)
- nothal 6y ago
  If I wanted to learn more about rigorous, non-elementary logic, do you have a recommended resource? I've taken a course in intro level probability theory which covered it generally and another course that built on it lightly but nothing rigorous and I am wooed by how concise things become in a logical form.
  - renewiltord 6y ago

hans789 6y ago

Quite agree, every single line of code written requires lifetime support. Code adds up and reduces productivity gradually, so only write code in core business logics.

jyunkess 6y ago

That's true. Beside _lifetime support_, working on that core business feature make us _understand_ deeply about the that feature. I've seen people integrate dependency for their core business. It helped to get started fast, but will create a blockage that required understanding deeper to overcome

cellular 6y ago

But "it's a good problem to have"!

KajMagnus 6y ago

I think a JSON parser is not a good example though — takes longer than a few hours / an afternoon, to write a JSON parser, add tests, fix bugs, corner cases. More like a week, or weeks, ... ... Look, a tiny json parser — Not an afternoon project: https://github.com/rafagafe/tiny-json/blob/master/tiny-json.c https://github.com/rafagafe/tiny-json/blob/master/tiny-json.... And a question about small JSON parsers — didn't see any afternoon projects among the answers: https://stackoverflow.com/questions/6061172/smallest-less-internal-code-up-to-spec-json-parser-in-java/6062466 https://stackoverflow.com/questions/6061172/smallest-less-in... I suppose a JSON parser was just an example. Made the whole answer sound weird to me though :- ) when the blog is about afternoon-projects and then a reply is about a week(s), could be month(s), long project.

signaru 6y ago
There's a fair middle ground when the dependency in itself doesn't have dependencies, and is small enough with a permissive license such that the entirety of its code can be dropped in to your project. Especially for very specific functionalities. I have used such tiny xml parsers, and I'm not affected by the fact that my copy is no longer the latest version. Its not so far from copying and pasting snippets of existing code.
jdmichal 6y ago
> I think a thing is not a good example though — takes longer than a few hours / an afternoon, to write a thing, add tests, fix bugs, corner cases. More like a week, or weeks, ... You're making my point for me. This is exactly what I meant by the lifetime of support you're signing up for by writing lines of code. Once you write that code, you're now in the business of supporting that code. Was that a good decision for your business?
imtringued 6y ago
Same with CSV. It looks easy, but it isn't. I've never seen anyone who writes their own CSV parser actually implement features necessary to conform to the standard like quoting and escape sequences. The end result is software that breaks when delimiters or quotes appear in user input. Honestly, I prefer xlsx spreadsheets because of that. Nobody fools themselves into implementing the parser or serializer for the format themselves. The only tiny pitfall with them is when people create spreadsheets manually in excel and write numbers as text, but parsing strings to numbers is absolutely trivial. You have to do that with CSV anyway.

SeriousM 6y ago

So you're saying that I should implement my own ormapper just because my product is using a database? And even this is not thee case, writing everything yourself will end up in your own hands. No Bugfixes, no patches or improvements without spending man work. I've worked in such a company and it was a mess accompanied by dev leaders who's to proud of their code to allow any change.

jdmichal 6y ago
I'm confused by your response. Is your core business mapping objects to databases? As in, that's what you get paid for? If not, my heuristic is that you should not be writing an ORM tool.