10 ms·

Doug Engelbart’s design for knowledge-based organizations (1992) [pdf]

147 points by conanxin 4y ago

physicsgraph 4y ago
This seems to be a preview of an article available as PDF after you sign in. Without the main content the purpose isn't clear to me. Am I missing something?
- Jtsummers 4y ago
  https://www.dougengelbart.org/pubs/seminars/sembinder1992nov/T.pdf https://www.dougengelbart.org/pubs/seminars/sembinder1992nov... - A PDF of the actual thing, matches the two opening paragraphs and you don't need to sign-in. From 1992, for the curious.
  - dang 4y ago
    We've changed to that from https://www.customers.com/articles/doug-engelbarts-design-knowledge-based-organizationspart-1/ https://www.customers.com/articles/doug-engelbarts-design-kn... above. Thanks!
  - 082349872349872 4y ago
    > From 1992 That explains the fax-mediated comments section. I guess HTML FORMs[0,1] weren't going to hit until late[2] 93... [0] https://web.mit.edu/kolya/.f/root/net.mit.edu/sipb.mit.edu/user/tabbott/sipb-www/doc/Mosaic-2.1/overview.html https://web.mit.edu/kolya/.f/root/net.mit.edu/sipb.mit.edu/u... [1] View Source confirms still in use by HN [2] http://1997.webhistory.org/www.lists/www-talk.1993q4/0447.html http://1997.webhistory.org/www.lists/www-talk.1993q4/0447.ht...
mikewarot 4y ago
Bush's Memex and Engelbart's mother of all demos hint at something much more powerful than HTML and the premature optimization that we got as a result. In reading the pdf linked by Jtsummers, I recall how some of the sales people I supported had 20+ gigabytes of email in their inboxes... which drove me nuts (Exchange Server really didn't cope well with it back then), and now I know why, they were building a hyperlinked database (well, threaded) that they could search through. Here in HN, we do the same thing over time, building a tree structured database that preserves context and is searchable. The main limitation here is there's only one possible tree, and it's the privileged view. There's at least a billion dollars of knowledge captured here in HN. Yet, it takes Dang and a strong community to keep that one view rational, due to the corrupting forces of human nature and profit motives from spam and scams. There's got to be a better way.
- Xeoncross 4y ago
  I think Google, GPT-3, co-pilot, simple tricks like a 'wordcloud' and even regurgitation/review communities like HN, reddit and good reads show that the presentation format of the information in a given system can be transformed to many diverse representations. Books and movies have long made plots off new ways to help humans learn things and form connections by either 1) changing the presentation format or 2) downloading all the information into the human brain to let it do the sorting. For example, overlying data on a map in 3d using glasses or wiring a human brain into a computer's serial ports. The main problem moving forward is making the data available. I have a fear that in the future only governments and large companies in bed with governments will have access to this data and use it in unethical ways. Meanwhile, startups and individuals will be bared from creating 'unlicensed' presentations or accessing the underlying data itself (i.e. running a search engine)
  - mikewarot 4y ago
    >I have a fear that in the future only governments and large companies in bed with governments will have access to this data and use it in unethical ways. I'd look back to the Memex for the solution to that problem, it stored everything locally, thus links couldn't be broken. Output included all the linked items. It's possible to build a system that stores everything you've seen on the web locally, to prevent things from disappearing. I think local storage is highly undervalued, and the pendulum will shift back.
  - jimmySixDOF 4y ago
    >overlying data on a map in 3d using glasses Yes. The opportunity to increase our functional representations of 2D knowledge bases is going to be a game changer that will compliment the Cambrian Explosion currently underway with the PKM/TfT use of symantec Graph connections & AI LLMs. Some efforts ongoing towards this [1] already and imho there is so much you can do to interact with data once you open up the presentation model to spatial computing inside a 3D embodied user interface like VR today, Passthrough AR tomorrow, and full AR a little down the line. [1] https://substack.soft.space/p/softspacear-prototype04 https://substack.soft.space/p/softspacear-prototype04
  - 4y ago
Terretta 4y ago
I read this as a prophecy of Foam (https://github.com/foambubble/foam https://github.com/foambubble/foam), Obsidian.md (https://obsidian.md/ https://obsidian.md/), and the like. See this thread on Foam from 2020: https://news.ycombinator.com/item?id=23666950 https://news.ycombinator.com/item?id=23666950 Despite ”design for knowledge based organizations” being the title of the article in the PDF, this is perhaps more about approach and capabilities for organizing flow from capture to use to pruning of knowledge, more than the organizations working on knowledge. There is one aside about the organization itself: > Engelbart sees every organization as a collection of interacting knowledge domains. He has focused his research on designing support structures for knowledge collection and refinementwithin and across these knowledge domains. But then that jumps down into a given domain, and goes on to propose a knowledge management approach within that (and each) domain. This is where it gets interestingly predictive of the recent coalescence in knowledge management tools emerging after the two dark age decades of style over semantics. The first idea is a concept of bringing knowledge in, working it, and keeping it. He calls this Concurrent Development, Integration, and Application of Knowledge (CODIAK) process. To make this workable he proposes a time-relevance layered approach very similar to the (perhaps easier to action) “PARA” method adopted by many KM tool users today: https://fortelabs.com/para https://fortelabs.com/para After this, he goes into functional implications, and describes almost to a T capabilities in the latest round of knowledge management tooling “systems” such as, say, Obsidian.md, built on standards (markdown, wikilinks, front matter) and extensibility enabling journaling, querying, views, as first class citizens. When hypertext markup iterated into a page style description advertorial tool rather than linked semantic knowledge structure, that left an opening for the tools we’re seeing now. Today, Obsidian.md is close to the mark, except, of course, for approachability by casuals. For casuals, consider e.g. Craft.do (https://www.craft.do/ https://www.craft.do/). If you’re doing a comparison before jumping in, it’s worth diffing the tools against Englebart’s take. I’d argue he’s dead on.
- majormajor 4y ago
  There are similarities but I'd disagree with this: > Despite ”design for knowledge based organizations” being the title of the article in the PDF, this is perhaps more about approach and capabilities for organizing flow from capture to use to pruning of knowledge, more than the organizations working on knowledge. I would agree that without the tools, an organization wouldn't be able to collaborate effectively in the way he advocates for, but I think unlocking group productivity should very much be more the goal here than single-person productivity. From the second page: "Englebert realized that they key to dealing with increasing complexity was human collaboration." No amount of methodology will let a single person keep up with a team using a similar methodology. Another key line from me: "Englebert is not advocating that we perform artificial acts with documents by superimposing structure on them. Instead, he advocates that we capture the inherent structure in all forms of human expression in order to make them easier for people to navigate through, view in different ways, and hyperlink." As you note, there are a lot of similarities in there to Roam Research and those similar systems, especially the cross-linking/blocks type stuff vs just having a single hierarchical document structure. But don't just think of it for your own navigation and recollection, but for others to learn and get up to speed and then offer their perspectives. But I think it's a big disservice to think about it only from a single-person's POV vs a way of capturing knowledge inside a group. Think of "git" for software dev too, which is very along the lines of the logging/journaling of changes discussed in the article, but which differs from some previous iterations by being far more multi-user friendly (remember fighting over SVN locks?).
ultra_nick 4y ago
Our startup's building a browser based AI Tutor that estimates a learner's knowledge level and uses it to explain concepts in context. It's like step-by-step navigation for learning. We're still in the very early stages, so it'd be years before we get to enterprise customers. Our website isn't active yet, but you can follow our LinkedIn if you're interested in updates. We have an MVP release planned for this month. https://www.linkedin.com/company/conceptionary/ https://www.linkedin.com/company/conceptionary/
- groby_b 4y ago
  You can do better than a linkedin link. This is HN, this is an interesting topic - surely, you have interesting things to say that people here would care about. Recruitment adverts, less so.
8bitsrule 4y ago
Painfully ironic that I should be forced to read about Doug's brilliant from an effing PDF file.
mark_l_watson 4y ago
I permanently saved the PDF. Brilliant stuff. I am probably overly optimistic that some combination of HTML, linked data, GPT-3 like models for effective summarization of very long texts, better user models (their level of expertise, things they have paid attention to in the past, etc.) will get us part way to Engelbart’s vision.
benstopics 4y ago
I think the issue with this is that as individuals we attach our identity to our dialog and put it out into the world to share it, express ourselves, to provide value to the world, and also to collaborate and iterate and learn ourselves… with other individuals. The problem with deconstructing human ideas and information like this is that it ignores temporality. What is true today may not be true tomorrow, what is relevant today may not be relevant tomorrow. For the preservation of scientific or historic information this works great, and even allows copywriting of works. However, the true order of the day is leading and making decisions which ends up taking place in dialogue. Dialogue is the single-threaded brute forcing and tree traversing activity for making decisions. Within a single thread, multiple subthreads do branch off, but then they become decentralized which is not what we want. We want each subthread to be conscious of it’s uniqueness to all other threads globally and for any duplication to be catalogued or ideally merged and then hyperlinked and indexed within the grand scheme. This is difficult when the most granular we can get in terms of categorization is the factual information or simply the topic itself. Places like HN and Reddit do well in allowing you to provide an entire article or website as the context and allow that to be explored, but how would you categorize this information? The real question is how do you make this visible and accessible from a single standpoint: an individual trying to learn something or make a decision. I think search engines like Google have solved this problem well simply by using the heuristic method of merit (backlinks, page rank, keywords) and so to implement this on a miniature scale for an organization would be useful, but the problem is that it would require users within the organization to forgo going to Google and instead use the provided Intranet which initially would be less useful and again may never be as useful as Google. So the solution might be to use Google as an indexing service which currently is not possible. I think at the end of the day we are ignoring is the incentives. If people were incentivized to construct the information in this way, get some sort of reward, then it would happen naturally. But I don’t even think it is possible for a human to do that, say, store all of HN dialogues globally for example in their head and consider THAT before writing anything otherwise hyperlinking to what is already there. The main issue is that knowledge is a graph, not a tree, and dialogue is linear. So in order to consume knowledge through text as quickly and efficiently as possible is more of a graph or traveling salesman problem which is very hard to solve, maybe something that Quantum AI can help with. One thing would be to incentivize hyperlinking. Another might be to store information in images using something like a way more advanced DALL-E that allows the human mind to grasp concepts simply through looking at trippy images or videos. To somehow use spatial and visual understanding to up the number of dimensions of what knowledge we can store and download into our brains.

147 points by conanxin 4y ago

This seems to be a preview of an article available as PDF after you sign in. Without the main content the purpose isn't clear to me. Am I missing something?

Jtsummers 4y ago
https://www.dougengelbart.org/pubs/seminars/sembinder1992nov/T.pdf https://www.dougengelbart.org/pubs/seminars/sembinder1992nov... - A PDF of the actual thing, matches the two opening paragraphs and you don't need to sign-in. From 1992, for the curious.
- dang 4y ago
  We've changed to that from https://www.customers.com/articles/doug-engelbarts-design-knowledge-based-organizationspart-1/ https://www.customers.com/articles/doug-engelbarts-design-kn... above. Thanks!
- 082349872349872 4y ago
  > From 1992 That explains the fax-mediated comments section. I guess HTML FORMs[0,1] weren't going to hit until late[2] 93... [0] https://web.mit.edu/kolya/.f/root/net.mit.edu/sipb.mit.edu/user/tabbott/sipb-www/doc/Mosaic-2.1/overview.html https://web.mit.edu/kolya/.f/root/net.mit.edu/sipb.mit.edu/u... [1] View Source confirms still in use by HN [2] http://1997.webhistory.org/www.lists/www-talk.1993q4/0447.html http://1997.webhistory.org/www.lists/www-talk.1993q4/0447.ht...

mikewarot 4y ago

Bush's Memex and Engelbart's mother of all demos hint at something much more powerful than HTML and the premature optimization that we got as a result. In reading the pdf linked by Jtsummers, I recall how some of the sales people I supported had 20+ gigabytes of email in their inboxes... which drove me nuts (Exchange Server really didn't cope well with it back then), and now I know why, they were building a hyperlinked database (well, threaded) that they could search through. Here in HN, we do the same thing over time, building a tree structured database that preserves context and is searchable. The main limitation here is there's only one possible tree, and it's the privileged view. There's at least a billion dollars of knowledge captured here in HN. Yet, it takes Dang and a strong community to keep that one view rational, due to the corrupting forces of human nature and profit motives from spam and scams. There's got to be a better way.

Xeoncross 4y ago
I think Google, GPT-3, co-pilot, simple tricks like a 'wordcloud' and even regurgitation/review communities like HN, reddit and good reads show that the presentation format of the information in a given system can be transformed to many diverse representations. Books and movies have long made plots off new ways to help humans learn things and form connections by either 1) changing the presentation format or 2) downloading all the information into the human brain to let it do the sorting. For example, overlying data on a map in 3d using glasses or wiring a human brain into a computer's serial ports. The main problem moving forward is making the data available. I have a fear that in the future only governments and large companies in bed with governments will have access to this data and use it in unethical ways. Meanwhile, startups and individuals will be bared from creating 'unlicensed' presentations or accessing the underlying data itself (i.e. running a search engine)
- mikewarot 4y ago
  >I have a fear that in the future only governments and large companies in bed with governments will have access to this data and use it in unethical ways. I'd look back to the Memex for the solution to that problem, it stored everything locally, thus links couldn't be broken. Output included all the linked items. It's possible to build a system that stores everything you've seen on the web locally, to prevent things from disappearing. I think local storage is highly undervalued, and the pendulum will shift back.
- jimmySixDOF 4y ago
  >overlying data on a map in 3d using glasses Yes. The opportunity to increase our functional representations of 2D knowledge bases is going to be a game changer that will compliment the Cambrian Explosion currently underway with the PKM/TfT use of symantec Graph connections & AI LLMs. Some efforts ongoing towards this [1] already and imho there is so much you can do to interact with data once you open up the presentation model to spatial computing inside a 3D embodied user interface like VR today, Passthrough AR tomorrow, and full AR a little down the line. [1] https://substack.soft.space/p/softspacear-prototype04 https://substack.soft.space/p/softspacear-prototype04
- 4y ago

Terretta 4y ago

I read this as a prophecy of Foam (https://github.com/foambubble/foam https://github.com/foambubble/foam), Obsidian.md (https://obsidian.md/ https://obsidian.md/), and the like. See this thread on Foam from 2020: https://news.ycombinator.com/item?id=23666950 https://news.ycombinator.com/item?id=23666950 Despite ”design for knowledge based organizations” being the title of the article in the PDF, this is perhaps more about approach and capabilities for organizing flow from capture to use to pruning of knowledge, more than the organizations working on knowledge. There is one aside about the organization itself: > Engelbart sees every organization as a collection of interacting knowledge domains. He has focused his research on designing support structures for knowledge collection and refinementwithin and across these knowledge domains. But then that jumps down into a given domain, and goes on to propose a knowledge management approach within that (and each) domain. This is where it gets interestingly predictive of the recent coalescence in knowledge management tools emerging after the two dark age decades of style over semantics. The first idea is a concept of bringing knowledge in, working it, and keeping it. He calls this Concurrent Development, Integration, and Application of Knowledge (CODIAK) process. To make this workable he proposes a time-relevance layered approach very similar to the (perhaps easier to action) “PARA” method adopted by many KM tool users today: https://fortelabs.com/para https://fortelabs.com/para After this, he goes into functional implications, and describes almost to a T capabilities in the latest round of knowledge management tooling “systems” such as, say, Obsidian.md, built on standards (markdown, wikilinks, front matter) and extensibility enabling journaling, querying, views, as first class citizens. When hypertext markup iterated into a page style description advertorial tool rather than linked semantic knowledge structure, that left an opening for the tools we’re seeing now. Today, Obsidian.md is close to the mark, except, of course, for approachability by casuals. For casuals, consider e.g. Craft.do (https://www.craft.do/ https://www.craft.do/). If you’re doing a comparison before jumping in, it’s worth diffing the tools against Englebart’s take. I’d argue he’s dead on.

majormajor 4y ago
There are similarities but I'd disagree with this: > Despite ”design for knowledge based organizations” being the title of the article in the PDF, this is perhaps more about approach and capabilities for organizing flow from capture to use to pruning of knowledge, more than the organizations working on knowledge. I would agree that without the tools, an organization wouldn't be able to collaborate effectively in the way he advocates for, but I think unlocking group productivity should very much be more the goal here than single-person productivity. From the second page: "Englebert realized that they key to dealing with increasing complexity was human collaboration." No amount of methodology will let a single person keep up with a team using a similar methodology. Another key line from me: "Englebert is not advocating that we perform artificial acts with documents by superimposing structure on them. Instead, he advocates that we capture the inherent structure in all forms of human expression in order to make them easier for people to navigate through, view in different ways, and hyperlink." As you note, there are a lot of similarities in there to Roam Research and those similar systems, especially the cross-linking/blocks type stuff vs just having a single hierarchical document structure. But don't just think of it for your own navigation and recollection, but for others to learn and get up to speed and then offer their perspectives. But I think it's a big disservice to think about it only from a single-person's POV vs a way of capturing knowledge inside a group. Think of "git" for software dev too, which is very along the lines of the logging/journaling of changes discussed in the article, but which differs from some previous iterations by being far more multi-user friendly (remember fighting over SVN locks?).

ultra_nick 4y ago

Our startup's building a browser based AI Tutor that estimates a learner's knowledge level and uses it to explain concepts in context. It's like step-by-step navigation for learning. We're still in the very early stages, so it'd be years before we get to enterprise customers. Our website isn't active yet, but you can follow our LinkedIn if you're interested in updates. We have an MVP release planned for this month. https://www.linkedin.com/company/conceptionary/ https://www.linkedin.com/company/conceptionary/

groby_b 4y ago
You can do better than a linkedin link. This is HN, this is an interesting topic - surely, you have interesting things to say that people here would care about. Recruitment adverts, less so.

8bitsrule 4y ago

Painfully ironic that I should be forced to read about Doug's brilliant from an effing PDF file.

mark_l_watson 4y ago

I permanently saved the PDF. Brilliant stuff. I am probably overly optimistic that some combination of HTML, linked data, GPT-3 like models for effective summarization of very long texts, better user models (their level of expertise, things they have paid attention to in the past, etc.) will get us part way to Engelbart’s vision.

benstopics 4y ago

I think the issue with this is that as individuals we attach our identity to our dialog and put it out into the world to share it, express ourselves, to provide value to the world, and also to collaborate and iterate and learn ourselves… with other individuals. The problem with deconstructing human ideas and information like this is that it ignores temporality. What is true today may not be true tomorrow, what is relevant today may not be relevant tomorrow. For the preservation of scientific or historic information this works great, and even allows copywriting of works. However, the true order of the day is leading and making decisions which ends up taking place in dialogue. Dialogue is the single-threaded brute forcing and tree traversing activity for making decisions. Within a single thread, multiple subthreads do branch off, but then they become decentralized which is not what we want. We want each subthread to be conscious of it’s uniqueness to all other threads globally and for any duplication to be catalogued or ideally merged and then hyperlinked and indexed within the grand scheme. This is difficult when the most granular we can get in terms of categorization is the factual information or simply the topic itself. Places like HN and Reddit do well in allowing you to provide an entire article or website as the context and allow that to be explored, but how would you categorize this information? The real question is how do you make this visible and accessible from a single standpoint: an individual trying to learn something or make a decision. I think search engines like Google have solved this problem well simply by using the heuristic method of merit (backlinks, page rank, keywords) and so to implement this on a miniature scale for an organization would be useful, but the problem is that it would require users within the organization to forgo going to Google and instead use the provided Intranet which initially would be less useful and again may never be as useful as Google. So the solution might be to use Google as an indexing service which currently is not possible. I think at the end of the day we are ignoring is the incentives. If people were incentivized to construct the information in this way, get some sort of reward, then it would happen naturally. But I don’t even think it is possible for a human to do that, say, store all of HN dialogues globally for example in their head and consider THAT before writing anything otherwise hyperlinking to what is already there. The main issue is that knowledge is a graph, not a tree, and dialogue is linear. So in order to consume knowledge through text as quickly and efficiently as possible is more of a graph or traveling salesman problem which is very hard to solve, maybe something that Quantum AI can help with. One thing would be to incentivize hyperlinking. Another might be to store information in images using something like a way more advanced DALL-E that allows the human mind to grasp concepts simply through looking at trippy images or videos. To somehow use spatial and visual understanding to up the number of dimensions of what knowledge we can store and download into our brains.