5 ms·

> cognitive load is the base problem one solves for when scaling development organizations. Just to add, this generalizes beyond just development orgs. I've s

by cosmie 7y ago

> cognitive load is the base problem one solves for when scaling development organizations. Just to add, this generalizes beyond just development orgs. I've scaled out analytics orgs at a few companies, and one of the most difficult aspects is penetrating into the workflows of business functions. As a rule, most[1] business users tend to operate at a high level of cognitive load, with little in the way of support structures to reduce that. But effectively integrating operational analytics requires[2] contextualization. So more often than not, the analytics scaling acts as a forcing function to put in place the operational and process supports needed for that contextualization, which has the secondary effect of reducing the cognitive load of the end users. In more than one occasion, I've seen that secondary effect have a greater impact than the analytics itself. [1] A major exception to this rule are tech companies and companies with technical founders. If the core of your business is a technical system, then your business functions are created as extensions of that system and you get a lot of process/structure "for free" on the business side due to that fact. And if that's not the case but you still have technical founders, those individuals will approach business functions with a process/systems-first mentality since that's how they approach technical problems. While it has it's own issues, it does lend itself to reducing the cognitive load of those functions and enabling more efficient scaling. Business leaders will take the opposite approach, where they leave the function mostly unstructured and (ideally) create process, structure, and systems based off of the emergent needs. [2] You can certainly provide analytics deliverables without contextualization. But it won't be nearly as effective, will mostly be ignored, and will rarely actually help anything since it leaves the onus of contextualization and interpretation on the end user and therefore does nothing to reduce their cognitive load.

techsin101 7y ago
Commenting to read later. But I'd love an example
- cosmie 7y ago
  Any particular industry/function/scenario you'd like used as an example? I can think of plenty of examples, but not sure which would click the best for you.
  - polskibus 7y ago
    +1 for an example - not for the "major exception" but rather for the "rule". I'd love to find out the specifics of contextualization that you mention - how should it be done, what's the ideal process for it, to deliver a valuable analytics deployment in the end that won't get buried due to lack of use several months later.
    - cosmie 7y ago
      Smaller business teams tend to use the "shared hosting" model when it comes to staffing: they hire a person and start subscribing relevant duties/needs to that person. At some point in time, they'll get oversubscribed to the point where it noticeably impacts their latency/performance. Once that happens consistently enough, the team will (hopefully) hire a second body and overflow duties to that new person. Rinse and repeat. Think of a 1-2 person marketing team, that has to handle anything that crops up and happens to fall under the "marketing" umbrella. At some point, demand for particular workloads will reach a point to justify adding in a dedicated server or two (i.e. hiring a specialist). But, while you now have a specialized/dedicated resource to direct specific types of work to, it's still a "subscribe relevant duties" reactionary model. And the specialized resource gets utilized/subscribed to just like a general resource would, but only for specific types of work. The day-to-day for these people is to execute work in the same manner as an interpreted language - they parse the request, constraints, and their capabilities, then immediately execute that work. Very little in the way of formal or documented processes, structures, outputs, inputs, etc exist. While some processes might be "cached" if they occur frequently and consistently enough, at some point they'll be garbage collected and have to be re-interpreted next time they execute. This is the reason most business teams operate with high cognitive load - disparate workloads, assigned with an oversubscription model, that require going through an interpretation phase before every execution. Some particular workloads may be consistent or important enough to warrant a JIT compilation model. Or they'll be underutilized enough to have the capacity to proactively implement a JIT compilation model. Components of the work will be precompiled and defined and structured, then at runtime they can reduce the interpreter overhead (cognitive load) necessary for execution. Very, very little of their workload ever makes it to a state of ahead of time compilation and the corresponding reduction in runtime overhead. That's the general rule for business teams. Developers though are so expensive that they're rarely allowed to operate this way. Instead, it's (theoretically) more cost efficient to invest in JIT optimizations for their workloads in the form of BAs, PMs, DevOps, ticketing systems, formalized processes, etc. And because they create systems and structure for a living, they're more apt and able to precompile parts of their workload themselves far more efficiently and effectively than a business team can do. As for ensuring a valuable analytics deployment: there's no single answer to that, it has to be interpreted at runtime. ;) But as a generalization, focus your analytics on giving your end users the tools necessary to be better JIT compilers. And ensure you don't introduce runtime errors for them. If your analytics deliverable is relevant to 80% of process executions, and has a known 20% of the time that it's not applicable, the user will lean into it and rely on it as a runtime optimization when appropriate. If it doesn't work for 20% of executions, but that 20% isn't known ahead of time, it'll be seen as a potential risk of runtime errors and completely rejected or bypassed by the end user. And if you prematurely go past that and leverage analytics for AOT compilation (i.e. full automation), your deployment is likely to fail entirely due to adding in too much rigidity to accommodate the unstable and ill-defined state of the process itself. Or if you go to the other extreme and your analytics just provides more inputs at interpretation time with little benefit of JIT improvements, it's just more runtime overhead and will have limited adoption unless the end user is forced or has enough spare capacity to absorb the additional interpretation overhead. Feel free to reach out if you want something more contextual - my email is in my profile.
- robbintt 7y ago
  You can click the date aka "3 days ago" to get a link to the exact comment, which can be bookmarked.

by cosmie 7y ago

techsin101 7y ago

Commenting to read later. But I'd love an example

cosmie 7y ago
Any particular industry/function/scenario you'd like used as an example? I can think of plenty of examples, but not sure which would click the best for you.
- polskibus 7y ago
  +1 for an example - not for the "major exception" but rather for the "rule". I'd love to find out the specifics of contextualization that you mention - how should it be done, what's the ideal process for it, to deliver a valuable analytics deployment in the end that won't get buried due to lack of use several months later.
  - cosmie 7y ago
    Smaller business teams tend to use the "shared hosting" model when it comes to staffing: they hire a person and start subscribing relevant duties/needs to that person. At some point in time, they'll get oversubscribed to the point where it noticeably impacts their latency/performance. Once that happens consistently enough, the team will (hopefully) hire a second body and overflow duties to that new person. Rinse and repeat. Think of a 1-2 person marketing team, that has to handle anything that crops up and happens to fall under the "marketing" umbrella. At some point, demand for particular workloads will reach a point to justify adding in a dedicated server or two (i.e. hiring a specialist). But, while you now have a specialized/dedicated resource to direct specific types of work to, it's still a "subscribe relevant duties" reactionary model. And the specialized resource gets utilized/subscribed to just like a general resource would, but only for specific types of work. The day-to-day for these people is to execute work in the same manner as an interpreted language - they parse the request, constraints, and their capabilities, then immediately execute that work. Very little in the way of formal or documented processes, structures, outputs, inputs, etc exist. While some processes might be "cached" if they occur frequently and consistently enough, at some point they'll be garbage collected and have to be re-interpreted next time they execute. This is the reason most business teams operate with high cognitive load - disparate workloads, assigned with an oversubscription model, that require going through an interpretation phase before every execution. Some particular workloads may be consistent or important enough to warrant a JIT compilation model. Or they'll be underutilized enough to have the capacity to proactively implement a JIT compilation model. Components of the work will be precompiled and defined and structured, then at runtime they can reduce the interpreter overhead (cognitive load) necessary for execution. Very, very little of their workload ever makes it to a state of ahead of time compilation and the corresponding reduction in runtime overhead. That's the general rule for business teams. Developers though are so expensive that they're rarely allowed to operate this way. Instead, it's (theoretically) more cost efficient to invest in JIT optimizations for their workloads in the form of BAs, PMs, DevOps, ticketing systems, formalized processes, etc. And because they create systems and structure for a living, they're more apt and able to precompile parts of their workload themselves far more efficiently and effectively than a business team can do. As for ensuring a valuable analytics deployment: there's no single answer to that, it has to be interpreted at runtime. ;) But as a generalization, focus your analytics on giving your end users the tools necessary to be better JIT compilers. And ensure you don't introduce runtime errors for them. If your analytics deliverable is relevant to 80% of process executions, and has a known 20% of the time that it's not applicable, the user will lean into it and rely on it as a runtime optimization when appropriate. If it doesn't work for 20% of executions, but that 20% isn't known ahead of time, it'll be seen as a potential risk of runtime errors and completely rejected or bypassed by the end user. And if you prematurely go past that and leverage analytics for AOT compilation (i.e. full automation), your deployment is likely to fail entirely due to adding in too much rigidity to accommodate the unstable and ill-defined state of the process itself. Or if you go to the other extreme and your analytics just provides more inputs at interpretation time with little benefit of JIT improvements, it's just more runtime overhead and will have limited adoption unless the end user is forced or has enough spare capacity to absorb the additional interpretation overhead. Feel free to reach out if you want something more contextual - my email is in my profile.
robbintt 7y ago
You can click the date aka "3 days ago" to get a link to the exact comment, which can be bookmarked.