7 ms·

The Opposite of Documentation is Superstition (2020)

154 points by BerislavLopac 2y ago

smitty1e 2y ago
I have also used the phrase "tribal knowledge" in the documentation-vacuum context.
- m348e912 2y ago
  There could be a business to be made in capturing and disseminating tribal knowledge for organizations. Even orgs with the best documentation don't have a great mechanism to capture and make available tribal knowledge which is un-authoritative, sometimes conflicting, and constantly changing. In the past I made an effort to address this by creating a unofficial but quickly searchable document repository documenting every question I have gotten from a teammate by either answering it or pointing to where the authoritative answer could be found. What I found was the following: 1. Questions are repetitive, you can quickly build a base of tribal knowledge and point people to the answer. 2. If the document tool is quick and efficient, team members will generally check it first instead of asking. 3. Sometimes document tribal knowledge conflicts with someone else's tribal knowledge. This is a great opportunity to hash out the more correct answer and update the document. 4. Tribal knowledge gets stale fast and needs constant curating. 5. The less friction to find an answer, eg fast access and fast/accurate searching, greatly increases success. NB. GPT is a perfect front end for such an effort.
  - wjholden 2y ago
    I would have loved to train an LLM on my old technical blog. In my last job, I used a blog to write informal reports. I set pretty loose rules for the blog for my team. We would post meeting minutes, outage postmortems, scripts, tricky one-liners, general lessons learned, and so on. We used SharePoint, which is kinda meh but it works. The "Alert Me" feature in SharePoint made it easy to subscribe my teammates to the blog via email, so you got the benefits of email (push notifications) without the drawbacks (employee mailboxes are unsearchable once they leave). I believe email contributes to this tacit ("tribal") knowledge that the OP names. I think my blog went a long way to counter this effect. A major goal in our blog posts was to capture the "why" behind our decisions. Often, we fail to remember the nuances of our own decisions in hindsight. I always thought it was important to name our present priorities and known-unknowns.
  - worksonmymach 2y ago
    Atlassian Rovo is this. The trick is to get people to get their tribal knowledge down in public Slack channels or their docs system. Recording most Zooms with AI summaries may help.
  - drewcoo 2y ago
    If the businesses valued it, it would be documented knowledge. It's clearly not worth the time or money to document. If the workers values it, it would likely be documented knowledge. Undocumented knowledge can be job security.
    - m348e912 2y ago
      > If the workers values it, it would likely be documented knowledge. Undocumented knowledge can be job security. You're mostly right. It's paradoxical in some ways. Workers generally won't go through the trouble of documenting tribal knowledge. However they greatly appreciate if someone else does.
- worksonmymach 2y ago
  Hard to avoid some tribal knowledge. To be free of it you need full reference documentation and tutorials for everything you do. And to update it as you go. And to realize what might be missing from a paedagogical perspective. Not easy and probably not possible for a commercial operation. And not necessary. 80-20 rule applies.
  - smitty1e 2y ago
    And applying that 80/20 rule implies a culture of writing things down and communicating them, which doesn't scale and is highly dependent upon the general chattiness of the team in question.
nox101 2y ago
Seems to me documentation can also be superstition if it's out of date. I wonder if LLMs can generate docs at any reasonable level of usefulness and accuracy Speaking of which, I just tried using ChatGPT to create a React app with typescript and rollup. What it generated didn't work (got errors). Of course I could tell it the errors and it fixed them but it suggests they aren't yet ready to write accurate docs
- iterateoften 2y ago
  Maybe LLMs make docs irrelevant if they are just generating static pages or doc strings. Why spend the time generating something static when everyone has their own LLMs they can just ask directly? Anything you can generate as docs someone can generate themselves anyways.
  - danihh 2y ago
    We actually ran this train of thought with our startup right now. The problem? Not everybody knows what questions to ask. For non-power users it’s usually better to present digestible information and create docs that answer questions they didn’t even know they had.
  - blharr 2y ago
    I mean, LLMs can be regenerated as needed. With every major release, you might regenerate the documentation. Then, you don't need to have everyone taking time to ask the same questions, reducing the amount of times you need to call the LLM. And thereby you can use a more expensive LLM, like o1 to generate the documentation where you wouldn't want to use it normally
  - pona-a 2y ago
    The thing is, good documentation captures the difference between what you expect and what something is. LLMs, as they exist now, are mass-producers of cliché. You can sense it even in their prose or those inverted puzzle evaluations [0] , they are simply statistical models picking the most likely options without enough depth to subvert the average. When you ask them to describe a piece of code, you'll only get the rephrasing of what the code already says. - [0] https://github.com/cpldcpu/MisguidedAttention https://github.com/cpldcpu/MisguidedAttention
closed 2y ago
This is an interesting case, since the pigeon study is about what happens when the underlying process is random. But if the shape drawing process isn't random, I think the author's experience of feeling unable to articulate the rules AND gravitating to a set of behaviors is a good example of procedural memory (implicit vs explicit). Explicit rules would probably help speed things up, though!
ChrisMarshallNY 2y ago
He's really talking about the user's mental model, here. I've found that it is sometimes a good idea to reinforce mental models that don't actually reflect the operation of the software (in my case) under it. Documentation is good, but, in my experience, almost no one ever reads it. I can tell, because of all the questions that I get, that are in the docs. That's totally my fault. A really good tech writer could probably do a better job of conveying information. That said, I think that it's important to present good, discoverable UI, that doesn't punish users for exploring.
- pfix 2y ago
  Both is important. And related. Documentation needs to be discoverable. I was amazed when I tried jj (jujutsu, the "new git") and it popped me into some kind of weird textual user interface after executing `jj split` and I felt lost. I guessed, pressing `?` won't hurt and it told me just to use the mouse. The menues showed the related hot keys. But the actual documentation of the tool has room for improvement. I needed a YouTube video to get started and that's rare for me. So what I want to say is, that you need an intuitive, discoverable UI, but also a documentation that has each case (and if it's just for linking in 1st level support cases) _and_ is discoverable. And by that I mean both easy to grasp (e.g. following https://diataxis.fr/ https://diataxis.fr/) and also can actually be found. I've had cases where a tool had good documentation, but actually finding it was the hard part.
  - throwaway14356 2y ago
    our architects love building the house before making the drawings. i imagine we will probably figure it out eventually when the feature set can be strictly defined. (maybe you eventually want a bath tub and a toilet in each room? maybe not?)
    - ChrisMarshallNY 2y ago
      I wouldn't call it "love," more than "necessity." It really depends on the nature of the project, but UI design often requires a lot of "Paving the Bare Spots"[0]. It's really just too damn complex and counter-intuitive (or too intuitive) to catch in Requirements. Software allows us to iterate this incredibly quickly. Hardware design also does it, but at a much slower pace, and a much greater cost. [0] https://littlegreenviper.com/the-road-most-traveled-by/#paving https://littlegreenviper.com/the-road-most-traveled-by/#pavi...
vincent-manis 2y ago
I can't remember the number of times I have given up on software because there is no “Concepts and Facilities” document that gives me a mental model of what the program will do and how I operate it. Instead, one sees a website with a mass of unrelated documents that I'm supposed to read in some order, and divine how everything works. Maybe because I'm a child of the 20th century that I value documentation that gives me a path from understanding what and why to learning how I can use it, with a master index so I can look up exactly what the “Mung Until No Good” command does, and see which other commands relate to it. Perhaps one day generative AI will be good enough that we can feed in one of these websites, with its pages with titles like “Migrating from V4.7.2 to V4.7.3” and “Building for OS/2 Warp” (I exaggerate, but only slightly) to documentation that is useful for learning, use, and troubleshooting. I live in hope.
- a_t48 2y ago
  Thanks for the reminder to make the concepts section of my site easier to read. :)
- floating-io 2y ago
  Not just an issue for documentation, but also for marketing materials. I can't count the number of times I've looked into some app that was referenced vaguely in a comment somewhere, only to have to dig for half an hour on that thing's web site just to figure out what the thing is supposed to even do in the first place...
  - 392 2y ago
    That's a great sign that using it will be more pain than just solving the problem yourself
shizzy0 2y ago
Some of the best technical documentation I’ve ever written has straight up lies in it. Granted they were placed there in the service of distilling the most germane portions of the API I was trying to explain. But it made a huge difference once I decided my code in the library didn’t have to be my code in its documentation. I felt a little like a biographical writer who makes a composite character. But honestly I recommend trying it. Lie in your docs if it serves the greater goal of communicating what’s most important. Let the computer labor in the “truth” and its tediums.
- Noumenon72 2y ago
  Do you mean you just left out required args, try-catches for checked exceptions, and so on, so the example would be easier to read? Did you put an asterisk on, like "* illustrative code: will not compile"?
Animats 2y ago
I've called this ritual-taboo programming for decades. It happens for user interfaces and APIs when the documentation is absent, or only consists of examples. If there's no reference documentation, everything is a copy of something someone else did. Nobody understands how it really works. Now, for some interfaces, this isn't too bad. Most people don't know why US AC power plugs are polarized, or what the ground prong is for. Electricians have to, but users do not. For more complex interfaces, it means that many functions will either be misused or undiscovered. This is the source of the plaint that only 10%-20% of a product's features are used. On user interface design, the classic is "Tog on Interface", Bruce "Tog" Tognazzini, 1992. That's from the Mac UI era. A more modern take is "The Gamer's Brain", by Celia Hodent, designer of Fortnite's UX. Software internal documentation seems to suffer today from a mindset that comments are unnecessary and waste space. Especially in the Javascript era. It's worst in languages that don't have data declarations. There's no place to properly document the data. Rust has good conventions and tools for documenting data. There's a well defined place where the documentation for each structure and field goes, and reasonable tools for checking it and turning that into documentation. If you fail to do this, when you publish your crate on crates.io, the documentation pages will come up blank, which screams "loser". Rust is weak on function parameter documentation. There should have been a defined place where each formal parameter gets a comment, which then appears in the documentation of the call. Most other languages don't take such a hard line. More should.
- fiforpg 2y ago
  > ritual-taboo programming I like to think of it as "cargo-cult programming", along the lines of the well-known essay by Feynman.
  - Animats 2y ago
    Cargo cults don't work at all. Ritual-taboo systems sort of work but are suboptimal.
    - michaelcampbell 2y ago
      Only if that's how you make the distinction between the 2. Copying something hoping it will do something you want because it did do that thing elsewhere, without understanding at all how or why it works is essentially the definition of CC programming.
w10-1 2y ago
The opposite of docs is also expertise: learn through experience, then have a career moat. It’s why Microsoft and Apple get away with secret API, which also gives them some leverage. Docs tend to be good only in the post-prototype, pre-dominance phase of rapid growth. Or when the author cares about users.
throwaway14356 2y ago
the fun of that study is that the pigeon behavior they think is to get food is superstition it self.
GuB-42 2y ago
One also could say that the opposite of open source is superstition. It works with the example given in the article: OneNote is proprietary software, and it is documented, though incompletely. Had it been open source, one could see exactly how shape recognition works, bugs, quirks and all. No documentation will give that amount of detail. Sure, not everyone has the time and skills for that, but maybe someone on stackoverflow does and have answered the question with proof, that is, not a superstition.
- ozim 2y ago
  Well one note is one thing - other thing is using open source libraries by devs. No one has time to read the code in depth so it is also mostly reading tea leaves like „oh it has a lot of stars and a lot of downloads so it must be good”. Don’t get me started on left pad ;)
codr7 2y ago
Isn't this more likely an example of LLM-crappification? As in, the reason it isn't documented is that no one has a clue...
- brabel 2y ago
  In that case, it should be documented at least that the process is driven by LLM and hence, it's not known exactly how it makes a decision...but I guess that would not look good to most users so they just leave it out.
  - codr7 2y ago
    Writing that you don't have a clue why the software works or doesn't isn't very good marketing :) Welcome to the stupid singularity.
akira2501 2y ago
Technically the opposite of documentation is anecdote. Literal interpretation of anecdotes lead to superstition. Ironically this leads to them becoming documented.
ChrisMarshallNY 2y ago
Right now, I'm working on a SwiftUI project that displays charts of various statistics for our app system. The docs for SwiftUI are ... less than ideal. I'm pretty sure that it can do what I need, but the docs don't cover it at all. I'll need to keep playing with code completion (which kinda sucks, right now, because Xcode keeps making up nonexistent APIs), and looking at the actual system exported headers. I'll get it, eventually, but this is pretty crazy bad.
- ChrisMarshallNY 2y ago
  Just to follow up, I finally got it all working today. No thanks to the "documentation." Lots of trial, and lots of error. I needed to have charts that you can pinch-to-zoom.
worksonmymach 2y ago
This refers to user docs rather than your team's README.md. Programmers have the luxury of documenting the previously undocumented for their repos (assuming management allocates time). End users would need to reverse engineer but that assumes predictability. Without that you get superstition as per article.
bruce511 2y ago
"Superstition" results from any number of things. A) insufficient docs B) programmer can't be bothered to read the docs C) programmer doesn't trust the docs D) programmer had a bug once, blames the underlying library, not his code E) programmer had a bug, changes code, bug goes away, jumps to conclusion. Trust in the docs is a big thing. If the docs are wrong a lot the the programmer is skeptical even when they're right. But I've also noticed that most developers simply don't read them. "I'm too busy (fixing bugs) to read the docs." Or "my backlog is so long I don't have time to learn how to do it right, I just want it done fast " Superstition occurs with the absence of knowledge - the existence of docs is necessary, but not sufficient, in overcoming that absence.
drewcoo 2y ago
Superstition plus documentation equals religion.
marmaduke 2y ago
The pigeon experiment is a great one to learn from not just about programming or software, but about life in general. Where are you getting your next dopamine hit? Is it random? Maybe that’s where our idiosyncrasies come from.
gmd63 2y ago
Functionality doesn't exist if it's not communicated, whether through intuitive UI or documentation. Much like laws don't exist if they're not enforced.
slwvx 2y ago
"Substition is the Discworld opposite of superstition. Superstition being the belief in the things that everyone believes in, but which are not true. Substition is the belief in things which are true but most people do not believe in." https://wiki.lspace.org/Substition https://wiki.lspace.org/Substition
M95D 2y ago
I thought the opposite was called cargo cult.
rekabis 2y ago
> By the end of the experiment, six of the eight pigeons developed nonsensical "superstitions" to get food Almost every human will do this for something IRL. As semi-irrational creatures, we tend to forcibly seek out patterns where there are none. Some people become so encrusted with such behaviour that it legitimately impacts their ability to operate effectively in modern society. Not surprisingly, the religiously-afflicted tend to pack the upper end of this spectrum.

154 points by BerislavLopac 2y ago

smitty1e 2y ago

I have also used the phrase "tribal knowledge" in the documentation-vacuum context.

m348e912 2y ago
There could be a business to be made in capturing and disseminating tribal knowledge for organizations. Even orgs with the best documentation don't have a great mechanism to capture and make available tribal knowledge which is un-authoritative, sometimes conflicting, and constantly changing. In the past I made an effort to address this by creating a unofficial but quickly searchable document repository documenting every question I have gotten from a teammate by either answering it or pointing to where the authoritative answer could be found. What I found was the following: 1. Questions are repetitive, you can quickly build a base of tribal knowledge and point people to the answer. 2. If the document tool is quick and efficient, team members will generally check it first instead of asking. 3. Sometimes document tribal knowledge conflicts with someone else's tribal knowledge. This is a great opportunity to hash out the more correct answer and update the document. 4. Tribal knowledge gets stale fast and needs constant curating. 5. The less friction to find an answer, eg fast access and fast/accurate searching, greatly increases success. NB. GPT is a perfect front end for such an effort.
- wjholden 2y ago
  I would have loved to train an LLM on my old technical blog. In my last job, I used a blog to write informal reports. I set pretty loose rules for the blog for my team. We would post meeting minutes, outage postmortems, scripts, tricky one-liners, general lessons learned, and so on. We used SharePoint, which is kinda meh but it works. The "Alert Me" feature in SharePoint made it easy to subscribe my teammates to the blog via email, so you got the benefits of email (push notifications) without the drawbacks (employee mailboxes are unsearchable once they leave). I believe email contributes to this tacit ("tribal") knowledge that the OP names. I think my blog went a long way to counter this effect. A major goal in our blog posts was to capture the "why" behind our decisions. Often, we fail to remember the nuances of our own decisions in hindsight. I always thought it was important to name our present priorities and known-unknowns.
- worksonmymach 2y ago
  Atlassian Rovo is this. The trick is to get people to get their tribal knowledge down in public Slack channels or their docs system. Recording most Zooms with AI summaries may help.
- drewcoo 2y ago
  If the businesses valued it, it would be documented knowledge. It's clearly not worth the time or money to document. If the workers values it, it would likely be documented knowledge. Undocumented knowledge can be job security.
  - m348e912 2y ago
    > If the workers values it, it would likely be documented knowledge. Undocumented knowledge can be job security. You're mostly right. It's paradoxical in some ways. Workers generally won't go through the trouble of documenting tribal knowledge. However they greatly appreciate if someone else does.
worksonmymach 2y ago
Hard to avoid some tribal knowledge. To be free of it you need full reference documentation and tutorials for everything you do. And to update it as you go. And to realize what might be missing from a paedagogical perspective. Not easy and probably not possible for a commercial operation. And not necessary. 80-20 rule applies.
- smitty1e 2y ago
  And applying that 80/20 rule implies a culture of writing things down and communicating them, which doesn't scale and is highly dependent upon the general chattiness of the team in question.

nox101 2y ago

Seems to me documentation can also be superstition if it's out of date. I wonder if LLMs can generate docs at any reasonable level of usefulness and accuracy Speaking of which, I just tried using ChatGPT to create a React app with typescript and rollup. What it generated didn't work (got errors). Of course I could tell it the errors and it fixed them but it suggests they aren't yet ready to write accurate docs

iterateoften 2y ago
Maybe LLMs make docs irrelevant if they are just generating static pages or doc strings. Why spend the time generating something static when everyone has their own LLMs they can just ask directly? Anything you can generate as docs someone can generate themselves anyways.
- danihh 2y ago
  We actually ran this train of thought with our startup right now. The problem? Not everybody knows what questions to ask. For non-power users it’s usually better to present digestible information and create docs that answer questions they didn’t even know they had.
- blharr 2y ago
  I mean, LLMs can be regenerated as needed. With every major release, you might regenerate the documentation. Then, you don't need to have everyone taking time to ask the same questions, reducing the amount of times you need to call the LLM. And thereby you can use a more expensive LLM, like o1 to generate the documentation where you wouldn't want to use it normally
- pona-a 2y ago
  The thing is, good documentation captures the difference between what you expect and what something is. LLMs, as they exist now, are mass-producers of cliché. You can sense it even in their prose or those inverted puzzle evaluations [0] , they are simply statistical models picking the most likely options without enough depth to subvert the average. When you ask them to describe a piece of code, you'll only get the rephrasing of what the code already says. - [0] https://github.com/cpldcpu/MisguidedAttention https://github.com/cpldcpu/MisguidedAttention

closed 2y ago

This is an interesting case, since the pigeon study is about what happens when the underlying process is random. But if the shape drawing process isn't random, I think the author's experience of feeling unable to articulate the rules AND gravitating to a set of behaviors is a good example of procedural memory (implicit vs explicit). Explicit rules would probably help speed things up, though!

ChrisMarshallNY 2y ago

He's really talking about the user's mental model, here. I've found that it is sometimes a good idea to reinforce mental models that don't actually reflect the operation of the software (in my case) under it. Documentation is good, but, in my experience, almost no one ever reads it. I can tell, because of all the questions that I get, that are in the docs. That's totally my fault. A really good tech writer could probably do a better job of conveying information. That said, I think that it's important to present good, discoverable UI, that doesn't punish users for exploring.

pfix 2y ago
Both is important. And related. Documentation needs to be discoverable. I was amazed when I tried jj (jujutsu, the "new git") and it popped me into some kind of weird textual user interface after executing `jj split` and I felt lost. I guessed, pressing `?` won't hurt and it told me just to use the mouse. The menues showed the related hot keys. But the actual documentation of the tool has room for improvement. I needed a YouTube video to get started and that's rare for me. So what I want to say is, that you need an intuitive, discoverable UI, but also a documentation that has each case (and if it's just for linking in 1st level support cases) _and_ is discoverable. And by that I mean both easy to grasp (e.g. following https://diataxis.fr/ https://diataxis.fr/) and also can actually be found. I've had cases where a tool had good documentation, but actually finding it was the hard part.
- throwaway14356 2y ago
  our architects love building the house before making the drawings. i imagine we will probably figure it out eventually when the feature set can be strictly defined. (maybe you eventually want a bath tub and a toilet in each room? maybe not?)
  - ChrisMarshallNY 2y ago
    I wouldn't call it "love," more than "necessity." It really depends on the nature of the project, but UI design often requires a lot of "Paving the Bare Spots"[0]. It's really just too damn complex and counter-intuitive (or too intuitive) to catch in Requirements. Software allows us to iterate this incredibly quickly. Hardware design also does it, but at a much slower pace, and a much greater cost. [0] https://littlegreenviper.com/the-road-most-traveled-by/#paving https://littlegreenviper.com/the-road-most-traveled-by/#pavi...

vincent-manis 2y ago

I can't remember the number of times I have given up on software because there is no “Concepts and Facilities” document that gives me a mental model of what the program will do and how I operate it. Instead, one sees a website with a mass of unrelated documents that I'm supposed to read in some order, and divine how everything works. Maybe because I'm a child of the 20th century that I value documentation that gives me a path from understanding what and why to learning how I can use it, with a master index so I can look up exactly what the “Mung Until No Good” command does, and see which other commands relate to it. Perhaps one day generative AI will be good enough that we can feed in one of these websites, with its pages with titles like “Migrating from V4.7.2 to V4.7.3” and “Building for OS/2 Warp” (I exaggerate, but only slightly) to documentation that is useful for learning, use, and troubleshooting. I live in hope.

a_t48 2y ago
Thanks for the reminder to make the concepts section of my site easier to read. :)
floating-io 2y ago
Not just an issue for documentation, but also for marketing materials. I can't count the number of times I've looked into some app that was referenced vaguely in a comment somewhere, only to have to dig for half an hour on that thing's web site just to figure out what the thing is supposed to even do in the first place...
- 392 2y ago
  That's a great sign that using it will be more pain than just solving the problem yourself

shizzy0 2y ago

Some of the best technical documentation I’ve ever written has straight up lies in it. Granted they were placed there in the service of distilling the most germane portions of the API I was trying to explain. But it made a huge difference once I decided my code in the library didn’t have to be my code in its documentation. I felt a little like a biographical writer who makes a composite character. But honestly I recommend trying it. Lie in your docs if it serves the greater goal of communicating what’s most important. Let the computer labor in the “truth” and its tediums.

Noumenon72 2y ago
Do you mean you just left out required args, try-catches for checked exceptions, and so on, so the example would be easier to read? Did you put an asterisk on, like "* illustrative code: will not compile"?

Animats 2y ago

I've called this ritual-taboo programming for decades. It happens for user interfaces and APIs when the documentation is absent, or only consists of examples. If there's no reference documentation, everything is a copy of something someone else did. Nobody understands how it really works. Now, for some interfaces, this isn't too bad. Most people don't know why US AC power plugs are polarized, or what the ground prong is for. Electricians have to, but users do not. For more complex interfaces, it means that many functions will either be misused or undiscovered. This is the source of the plaint that only 10%-20% of a product's features are used. On user interface design, the classic is "Tog on Interface", Bruce "Tog" Tognazzini, 1992. That's from the Mac UI era. A more modern take is "The Gamer's Brain", by Celia Hodent, designer of Fortnite's UX. Software internal documentation seems to suffer today from a mindset that comments are unnecessary and waste space. Especially in the Javascript era. It's worst in languages that don't have data declarations. There's no place to properly document the data. Rust has good conventions and tools for documenting data. There's a well defined place where the documentation for each structure and field goes, and reasonable tools for checking it and turning that into documentation. If you fail to do this, when you publish your crate on crates.io, the documentation pages will come up blank, which screams "loser". Rust is weak on function parameter documentation. There should have been a defined place where each formal parameter gets a comment, which then appears in the documentation of the call. Most other languages don't take such a hard line. More should.

fiforpg 2y ago
> ritual-taboo programming I like to think of it as "cargo-cult programming", along the lines of the well-known essay by Feynman.
- Animats 2y ago
  Cargo cults don't work at all. Ritual-taboo systems sort of work but are suboptimal.
  - michaelcampbell 2y ago
    Only if that's how you make the distinction between the 2. Copying something hoping it will do something you want because it did do that thing elsewhere, without understanding at all how or why it works is essentially the definition of CC programming.

w10-1 2y ago

The opposite of docs is also expertise: learn through experience, then have a career moat. It’s why Microsoft and Apple get away with secret API, which also gives them some leverage. Docs tend to be good only in the post-prototype, pre-dominance phase of rapid growth. Or when the author cares about users.

throwaway14356 2y ago

the fun of that study is that the pigeon behavior they think is to get food is superstition it self.

GuB-42 2y ago

One also could say that the opposite of open source is superstition. It works with the example given in the article: OneNote is proprietary software, and it is documented, though incompletely. Had it been open source, one could see exactly how shape recognition works, bugs, quirks and all. No documentation will give that amount of detail. Sure, not everyone has the time and skills for that, but maybe someone on stackoverflow does and have answered the question with proof, that is, not a superstition.

ozim 2y ago
Well one note is one thing - other thing is using open source libraries by devs. No one has time to read the code in depth so it is also mostly reading tea leaves like „oh it has a lot of stars and a lot of downloads so it must be good”. Don’t get me started on left pad ;)

codr7 2y ago

Isn't this more likely an example of LLM-crappification? As in, the reason it isn't documented is that no one has a clue...

brabel 2y ago
In that case, it should be documented at least that the process is driven by LLM and hence, it's not known exactly how it makes a decision...but I guess that would not look good to most users so they just leave it out.
- codr7 2y ago
  Writing that you don't have a clue why the software works or doesn't isn't very good marketing :) Welcome to the stupid singularity.

akira2501 2y ago

Technically the opposite of documentation is anecdote. Literal interpretation of anecdotes lead to superstition. Ironically this leads to them becoming documented.

ChrisMarshallNY 2y ago

Right now, I'm working on a SwiftUI project that displays charts of various statistics for our app system. The docs for SwiftUI are ... less than ideal. I'm pretty sure that it can do what I need, but the docs don't cover it at all. I'll need to keep playing with code completion (which kinda sucks, right now, because Xcode keeps making up nonexistent APIs), and looking at the actual system exported headers. I'll get it, eventually, but this is pretty crazy bad.

ChrisMarshallNY 2y ago
Just to follow up, I finally got it all working today. No thanks to the "documentation." Lots of trial, and lots of error. I needed to have charts that you can pinch-to-zoom.

worksonmymach 2y ago

This refers to user docs rather than your team's README.md. Programmers have the luxury of documenting the previously undocumented for their repos (assuming management allocates time). End users would need to reverse engineer but that assumes predictability. Without that you get superstition as per article.

bruce511 2y ago

"Superstition" results from any number of things. A) insufficient docs B) programmer can't be bothered to read the docs C) programmer doesn't trust the docs D) programmer had a bug once, blames the underlying library, not his code E) programmer had a bug, changes code, bug goes away, jumps to conclusion. Trust in the docs is a big thing. If the docs are wrong a lot the the programmer is skeptical even when they're right. But I've also noticed that most developers simply don't read them. "I'm too busy (fixing bugs) to read the docs." Or "my backlog is so long I don't have time to learn how to do it right, I just want it done fast " Superstition occurs with the absence of knowledge - the existence of docs is necessary, but not sufficient, in overcoming that absence.

drewcoo 2y ago

Superstition plus documentation equals religion.

marmaduke 2y ago

The pigeon experiment is a great one to learn from not just about programming or software, but about life in general. Where are you getting your next dopamine hit? Is it random? Maybe that’s where our idiosyncrasies come from.

gmd63 2y ago

Functionality doesn't exist if it's not communicated, whether through intuitive UI or documentation. Much like laws don't exist if they're not enforced.

slwvx 2y ago

"Substition is the Discworld opposite of superstition. Superstition being the belief in the things that everyone believes in, but which are not true. Substition is the belief in things which are true but most people do not believe in." https://wiki.lspace.org/Substition https://wiki.lspace.org/Substition

M95D 2y ago

I thought the opposite was called cargo cult.

rekabis 2y ago

> By the end of the experiment, six of the eight pigeons developed nonsensical "superstitions" to get food Almost every human will do this for something IRL. As semi-irrational creatures, we tend to forcibly seek out patterns where there are none. Some people become so encrusted with such behaviour that it legitimately impacts their ability to operate effectively in modern society. Not surprisingly, the religiously-afflicted tend to pack the upper end of this spectrum.