5 ms·

"apex predator of grug is complexity complexity bad say again: complexity very bad you say now: complexity very, very bad" https://grugbrain.dev/ https://

"apex predator of grug is complexity complexity bad say again: complexity very bad you say now: complexity very, very bad" https://grugbrain.dev/ https://grugbrain.dev/ No really -- I would bet that the single greatest contribution comes from the massive growth in complexity, in all relevant areas, over the last 20+ years.

faxmeyourcode 4y ago
Every time I see this posted somewhere it's been long enough for me to forget about it, so I go through and read it again and glean something new from it.
- Michelangelo11 4y ago
  He also has a great Twitter account here: https://mobile.twitter.com/GrugBrainedDev/with_replies https://mobile.twitter.com/GrugBrainedDev/with_replies
TeMPOraL 4y ago
Some of the complexity is, unfortunately, introduced by grugs, because complexity smarter than grug and sometimes pretend is simple. Couple months back, I read an article that made a bold claim: that the old wisdom saying most software is IO-bound stopped being true some time ago; instead, most software nowadays is CPU-bound, typically on parsing JSON. JSON is something that may seem simple to grug, particularly a webdev grug. So simple that they'll use it for structuring and exchanging data everywhere. But while on the API side it seems simple, actual JSON parsers and serializers are quite complex, and the format itself expensive to parse and wasteful[0]. All that complexity gets silently embedded into everything, and the overhead at runtime is paid at nearly every step and nearly every level of software stacks, even though most of it isn't even needed. That's the failure mode of grug understanding of simplicity. Sometimes a little more effort up front yields much lower total complexity. (Also, eliminating complexity is good. Shifting complexity from developers to users is criminal.) -- [0] - It's better than XML in most scenarios, especially scenarios in which neither of them should be used in the first place.
- pixl97 4y ago
  What are you suggesting instead? A protocol change like protobuf for example? Or an architecture change (send less data in the first place)? Just wondering what a more complex but better solution would look like in this example.
  - TeMPOraL 4y ago
    Either, both, or more. It's situational. Switch to protobuf. Or switch to CSV. Or switch to SQLite database files. Or stay with JSON, but reduce the complexity of the format, even if it means you need a post-processing step on your side[0]. Change overall design to do less back-and-forth. Batch requests and replies. Send just the required amount of information, instead of having the receiver discard 95% of the reply every time. Don't convert to JSON (or other ad-hoc stringly-typed serialization format) and back from it inside your own application process, just because it feels "simpler" than using a data structure[1]. Etc. I get why people like using JSON protocols, even defaulting to it for ad-hoc ones. I do that too! It's the local optimum for protocol development[2]. JSON protocols are easy to extend and easy to debug. It's a good starting point when your protocol is in total flux. At some point, however, the protocol mostly solidifies. It should then be revisited and tightened up a bit. The main benefit is of course performance. Even if the refactor can't simplify the architecture of your project (e.g. no possibility to batch things or reduce amount of places that do communication), what grugs often miss is that performance improvements alone can reduce complexity. The canonical case here is scaling: switching from single-process to a scalable distributed solution is a massive jump in complexity. Keeping an eye on performance and removing (or avoiding) waste introduced for the sake of "simplicity" or development velocity will delay the point at which you need to switch to distributed solution. A little up-front cleverness and complexity now, plus a little spend to beef up your servers, may delay the switch forever. Computers are fast, we're just not using it, and grugs seeking simplicity by gradient descent and sleep-walking into high-complexity regions of solution space are partly to blame here. -- [0] - I still shudder when I think back to a certain charting tool in the browser, that required on the text input side to be supplied with datapoints in the format: [{x: 42, y: 100}, {x: 43, y: 64}, ...] Maybe this is because it matched the internal representation (if so, I do have some thoughts about it too). But doing it like: {x: [42, 43, ...], y: [100, 64, ...]} or: [[42, 100], [43, 64], ...] would easily cut the input size by 50% or more, meaning that much less work for the parser in the library and the serializer generating the input text. This shorter format is arguably more human-readable too! [1] - Sounds stupid, but I've seen this happen in otherwise sophisticated and somewhat performance-sensitive codebases. Beyond the waste coming from most of your data being passed around in pseudo-JSON strings, converted to various scalar and array types at the point of use and then serialized back, it leads to surprising amount of subtle bugs - especially when people manipulate the serialized format manually, because it's again "simpler" this way. [2] - Every programming language these days has good JSON support built-in or easily available, it's hierarchical without footguns (c.f. YAML), it's plaintext and has good editor support, but still relatively compact (c.f. XML), it's malleable, it's browser-native, etc.

faxmeyourcode 4y ago

Every time I see this posted somewhere it's been long enough for me to forget about it, so I go through and read it again and glean something new from it.

Michelangelo11 4y ago
He also has a great Twitter account here: https://mobile.twitter.com/GrugBrainedDev/with_replies https://mobile.twitter.com/GrugBrainedDev/with_replies

TeMPOraL 4y ago

Some of the complexity is, unfortunately, introduced by grugs, because complexity smarter than grug and sometimes pretend is simple. Couple months back, I read an article that made a bold claim: that the old wisdom saying most software is IO-bound stopped being true some time ago; instead, most software nowadays is CPU-bound, typically on parsing JSON. JSON is something that may seem simple to grug, particularly a webdev grug. So simple that they'll use it for structuring and exchanging data everywhere. But while on the API side it seems simple, actual JSON parsers and serializers are quite complex, and the format itself expensive to parse and wasteful[0]. All that complexity gets silently embedded into everything, and the overhead at runtime is paid at nearly every step and nearly every level of software stacks, even though most of it isn't even needed. That's the failure mode of grug understanding of simplicity. Sometimes a little more effort up front yields much lower total complexity. (Also, eliminating complexity is good. Shifting complexity from developers to users is criminal.) -- [0] - It's better than XML in most scenarios, especially scenarios in which neither of them should be used in the first place.

pixl97 4y ago
What are you suggesting instead? A protocol change like protobuf for example? Or an architecture change (send less data in the first place)? Just wondering what a more complex but better solution would look like in this example.
- TeMPOraL 4y ago
  Either, both, or more. It's situational. Switch to protobuf. Or switch to CSV. Or switch to SQLite database files. Or stay with JSON, but reduce the complexity of the format, even if it means you need a post-processing step on your side[0]. Change overall design to do less back-and-forth. Batch requests and replies. Send just the required amount of information, instead of having the receiver discard 95% of the reply every time. Don't convert to JSON (or other ad-hoc stringly-typed serialization format) and back from it inside your own application process, just because it feels "simpler" than using a data structure[1]. Etc. I get why people like using JSON protocols, even defaulting to it for ad-hoc ones. I do that too! It's the local optimum for protocol development[2]. JSON protocols are easy to extend and easy to debug. It's a good starting point when your protocol is in total flux. At some point, however, the protocol mostly solidifies. It should then be revisited and tightened up a bit. The main benefit is of course performance. Even if the refactor can't simplify the architecture of your project (e.g. no possibility to batch things or reduce amount of places that do communication), what grugs often miss is that performance improvements alone can reduce complexity. The canonical case here is scaling: switching from single-process to a scalable distributed solution is a massive jump in complexity. Keeping an eye on performance and removing (or avoiding) waste introduced for the sake of "simplicity" or development velocity will delay the point at which you need to switch to distributed solution. A little up-front cleverness and complexity now, plus a little spend to beef up your servers, may delay the switch forever. Computers are fast, we're just not using it, and grugs seeking simplicity by gradient descent and sleep-walking into high-complexity regions of solution space are partly to blame here. -- [0] - I still shudder when I think back to a certain charting tool in the browser, that required on the text input side to be supplied with datapoints in the format: [{x: 42, y: 100}, {x: 43, y: 64}, ...] Maybe this is because it matched the internal representation (if so, I do have some thoughts about it too). But doing it like: {x: [42, 43, ...], y: [100, 64, ...]} or: [[42, 100], [43, 64], ...] would easily cut the input size by 50% or more, meaning that much less work for the parser in the library and the serializer generating the input text. This shorter format is arguably more human-readable too! [1] - Sounds stupid, but I've seen this happen in otherwise sophisticated and somewhat performance-sensitive codebases. Beyond the waste coming from most of your data being passed around in pseudo-JSON strings, converted to various scalar and array types at the point of use and then serialized back, it leads to surprising amount of subtle bugs - especially when people manipulate the serialized format manually, because it's again "simpler" this way. [2] - Every programming language these days has good JSON support built-in or easily available, it's hierarchical without footguns (c.f. YAML), it's plaintext and has good editor support, but still relatively compact (c.f. XML), it's malleable, it's browser-native, etc.