18 ms·
Show HN: Odin – the integration of LLMs with Obsidian note taking
- AlexIchenskiy 3y agoLately, my colleagues and I have noticed how difficult it is becoming to organize and structure our thoughts, ideas, and notes. As budding engineers and full-fledged graph enthusiasts, we decided to approach this problem in our style - we began to study, research and look for different tools that could help us automate and improve these processes. In the process of brainstorming and research, we have read quite a lot of scientific papers devoted to the concept of knowledge graphs. We found many of these works and other materials with the help of LLM and we came up with the idea of trying to combine these two technologies in one project that was supposed to help us with the most difficult thing - organizing yourself. Of course, besides this, we came up with many more ideas, some of which we had to shelve with great regret, and some we even managed to implement in our other projects. Given that my colleague already had a wealth of experience using Obsidian and I made some contributions to LangChain, a very nice open-source Python framework for LLM communication, we decided to unite forces - I always enjoyed creating rich user interfaces while my colleague, Patrik, dealt with the part of the project that my UI needed to render. This is how our work on ODIN began, but what is Obsidian, how did a god from Norse mythology get in here, and why is TypeScript great? Let's find out. Although I didn’t have such extensive experience working with Obsidian, I fell in love with it at first sight - a pragmatic and user-friendly interface, nice colors, a wide and very open open-source community and other goodies - what else does a good note-taking application need? With these thoughts in mind, I began to study how Obsidian works from the inside, how it can be expanded, and how it can be connected to other technologies - such as LangChain, in our case. I was very pleasantly surprised when I discovered a well-made Developer API that allows you to create your custom community plugins that can easily connect to the application itself. I was even more pleasantly surprised when I discovered that all this is web-based and has support for React, which I worked with, and Svelte for its fans. Although the official documentation is not too full of details and sometimes you can get lost in it, a very convenient UI styling system and strict TypeScript type definitions, coupled with IntelliSense, which I always use, is a game-changer. What's notable is the ease of creating plugins in real time; npm scripts allow you to instantly detect code changes, and the Obsidian Hot Reload plugin speeds up development several times. In addition, Obsidian provides developer tools like popular browsers, simplifying the debugging process by accessing the console and directly inspecting elements on the screen. This flexibility in the Obsidian API allows developers to bring their creativity to life, unlocking the full potential of Obsidian to create unique note-taking experiences and more. Honestly, it feels just like making a web application in a modern and robust environment. I'll miss this when I get back to regular web development. That’s all great, but still, why ODIN? Honestly, it’s quite simple — we liked the project naming style that has existed in Memgraph for a long time — for example, MAGE, also known as Memgraph Advanced Graph Extension, seemed to us a damn original name, sharp as volcanic glass — and so, ODIN was born, from now onwards known as Obsidian Driven Information Network, an Obsidian plugin for knowledge management. This name became the real starting point of our project, representing the goal that we were trying to comprehend and the idea that we were trying to realize. In our R&D process, while working on ODIN, we figured out that data storage is the core of the app's architecture, without which neither the backend nor the frontend would make any sense. Even though we thought a lot about traditional databases such as PostgreSQL, the Obsidian knowledge graph was the main inspiration to try Memgraph, a graph database that excels in modeling complex relationships and connections, making it the ideal choice for creating and navigating intricate information networks. Even I, an ordinary student with a passion for developing user interfaces, was able to play around with both Memgraph and the query language called Cypher. I highly recommend trying the visualization that Memgraph Lab provides, even if you are far from both database graphs and software development in general. By seamless integration with LangChain, we’ve tried to leverage Memgraph's speed and efficiency to transform standard notes into interconnected knowledge graphs empowered by the flexibility of embedding search. This enables you not only to traverse your notes effortlessly but also to extract valuable insights by interacting with LLMs through LangChain implemented in the backend using Python and FastAPI. With Memgraph as our data storage solution, ODIN redefines the way we store and harness knowledge, offering a platform for building an intelligent and dynamic personal knowledge management system just from your notes. Isn’t it wonderful? ODIN introduces a multitude of functionalities to improve your knowledge management experience. At its core, ODIN provides a chat-like prompt bar for answering questions about the data stored in your knowledge graph as if you were just chatting with your knowledge collection. With ODIN's advanced graph visualization capabilities provided by Cytoscape, an amazing library for graph visualization with complete TypeScript compatibility, you gain a whole new view of your Obsidian vault, effortlessly navigating through the complex web of your notes. This extends to individual files view, where you can delve deep into specific topics from your markdown notes. Furthermore, ODIN expands the Obsidian native drop-down menu by adding new advanced functions. Link prediction, as its name suggests, generates connections to other markdown files in your entire vault, enriching your note's content with thematically relevant references. Moreover, you can extract thought-provoking questions from your markdown file, fostering deeper contemplation. Lastly, the node suggestion capability enhances your note-taking by highlighting thematically connected nodes, forging meaningful connections between highlighted text and related knowledge nodes, all while promoting a more comprehensive understanding of your stored information. On the other hand, clicking the node from the file view will highlight related sentences directly in your opened note. These features perfectly represent the goals we were trying to achieve and have proven to be useful in our daily lives, which means they might be of interest to other developers as well as anyone else interested in organizing their knowledge in a better way. The Obsidian community (including you, reading this article!) is the lifeblood of the platform's evolution and continued success. At the forefront of this community-driven ecosystem are the invaluable community plugins, just like ODIN. These user-generated extensions enhance Obsidian's functionality, addressing specific user needs and preferences. Becoming a part of this thriving community is not only about reaping the benefits of these plugins but also contributing to its growth. By actively participating in discussions, providing feedback, or even crafting your own plugins, you can play a pivotal role in shaping the future of Obsidian, enriching the collective knowledge management experience, and ensuring that this vibrant community continues to flourish. The history of the creation of this project was long (and sometimes painful to remember), but if you were to ask me if I would do it again, I would without any hesitation. Creating plugin-type interfaces is a breath of fresh air in frontend development, which still fits perfectly not only into open-source contribution but also into professional experience along with career growth. Although this project could be done better in many ways, it will remain a reliable indicator that modern technologies can and should be used in conjunction with one another. You, too, can become a part of this story — open an issue in our GitHub repository, and we will be happy to make ODIN better together! I hope that I was able to ignite your interest in trying Obsidian or even making a more tangible contribution to this incredible open-source community and thank you for reading this story about my journey.
- throwoutway 3y agoJust want to say this is a great idea and while I cant dive into it now (time) Im excited to dive into this in the future. Ill hve to bookmark this and return to it
- kristofferR 3y agoSummary: https://chat.openai.com/share/0dbfcc68-0ead-4d04-93e0-00a29fb6faf7 https://chat.openai.com/share/0dbfcc68-0ead-4d04-93e0-00a29f...
- asynchronous 3y agoThank you, thought this entire output was generated with ChatGPT
- grozmovoi 3y agolol the irony
- totetsu 3y agoFour layers deep of summation The writer and Patrik created ODIN using Obsidian to improve knowledge management, incorporating advanced features and technologies. They value the Obsidian community and invite GitHub contributions.
- for_i_in_range 3y agoI'm not sure many are going to invest the mental energy to read all of this. A tl;dr summary would be nice.
- bafe 3y agoDid you ask an LLM to write this wall of text?
- PurpleRamen 3y agoIs this a blog-article? Why is there so much meaningless talk about irrelevant details? And why has a project that claims to offer new views, no example of those views? No screenshots, video, whatever to explain what it really sells.
- aphit 3y agoLooks interesting but is in desperate need of an explainer video or some GIFs or something to showcase what it is doing. I'm not going to go through all the trouble of downloading and connecting to my OpenAI api key/Obsidian library without having a better grasp of what exactly it is doing.
- AlexIchenskiy 3y agoThanks for the constructive feedback! I'll expand the readme and add more details
- clscott 3y agoI would very much appreciate the same for RUNE if you feel motivated. I'm looking now to choose the obsidian plugin that helps me take advantage of the vast amount of notes I have created with obsidian over the last few years.
- AlexIchenskiy 3y agoI've added some screenshots to README files of ODIN and RUNE so you can get a feel for what they're like before installing. Hope you like it!
- m3kw9 3y agoInstead of a disclaimer first thing I see and install instructions, they should explain how it works first
- jackthetab 3y agoThey should explain what it _does_. I looked it over and I still have no idea why I should install it other than it uses $CURRENT_FAD.
- m3kw9 3y agoIs very heads down engineer like to not know anything about marketing.
- Brajeshwar 3y agoBest of luck. We need more of such tools in this note-taking and management area. A YC Startup, Khoj[1], recently did a ShowHN[2]. It works and has several options, including an Obsidian Plugin and for Emacs. Unfortunately, it drags my computer, and I can't do anything else much once I run the local server. Perhaps, once I upgrade out of my Intel iMac, I will return to it. (I'm not affiliated to Khoj.) 1. https://khoj.dev https://khoj.dev 2. https://news.ycombinator.com/item?id=36933452 https://news.ycombinator.com/item?id=36933452
- for_i_in_range 3y agoI ran a poll recently in a PKM community of 17,000 members asking who has written a book—or created some other knowledge product like a course—using their tool (Obsidian, Zettelkasten, LogSeq, etc.). I received two responses of people who actually produced something. The vast majority, it seems, are professional note-taking junkies. People in the Obsidian forum spend their lives fooling around with metadata conventions and "tagging" and "wikilinks" instead of producing work that actually matters. Intellectual procrastination at its finest. Now we have this... ODIN—a way to outsource your intellectual note-taking procrastination to AI. Note: The requirement wasn't just a book. It was any "published knowledge product" which I define as book, code, course, essay, etc. (What Naval Ravikant calls Specific Knowledge).
- forward-slashed 3y agoMaybe that's the wrong question. Maybe ask how many publish papers or write serious blog posts?
- smeej 3y agoWhy is writing long-form works the only type of output that matters? I use my PKM to connect the ideas I've read about over time. When I come across a topic again, I can quickly review what I've learned about it in the past, and how it connects, sometimes in obscure ways, to other things that have crossed my radar. This helps me come up with a wider variety of ideas for how to solve problems in front of me, or new connections to draw on or people to contact for feedback. I don't care about any one topic enough to devote the necessary time to writing a book about it, but that doesn't mean the system as a whole doesn't provide me a whole lot of value. Maybe this is just a misunderstanding between the specialist mind and generalist minds. The variety of things I can speak about and connect together intelligently is the value I bring to a company or a conversation, not the extraordinary depth I can go to on any one individual topic. I suspect there are a lot more generalists like me using PKM software like Obsidian than there are specialists.
- j45 3y agoOP is not aware that expert publishing can also be short, concise and illuminating. Using a PKM frees up time for other pursuits.
- ttamslam 3y agoI seem to missing something during install -- Odin is not showing up in the list of Community Plugins. I'm running locally via docker and have tried updating the Obsidian client + toggling restricted mode + restarting. Very excited to try it out if I can get the install to work.
- curo 3y agoOne alternative. If you live in VS Code, consider Foam which gets you Roam/Obsidian like flows alongside an LLM (Github Copilot). I have no affiliation to Foam. It's just awesome.
- digdugdirk 3y agoHave you tried Dendron? I'm at the point where I'm debating if I want to migrate my workflow from vscode over to emacs so I can use org-roam in a combined notes/code/documents workflow, and both dendron and foam popped up in my searching for alternatives. I'm curious if you had any preference or reason why you went with foam.
- bachmeier 3y agoDendron shut down a long time ago: https://github.com/dendronhq/dendron/discussions/3890 https://github.com/dendronhq/dendron/discussions/3890 The repo is up, but the project is dead.
- arde 3y agoI wish somebody takes it up again. I'll keep using it for the foreseeable future because it's great, and haven't found better alternatives.
- lolive 3y ago"It is recommended that you have access to GPT-4 via the OpenAI API." Does this sentence imply that some data of my vault will be sent to OpenAI? [nut sure my company will be fond of that]
- xcdzvyn 3y agoYes. I wish more services like these at least had the option of using a local model..
- freedomben 3y agoAnyone know of something like this for Logseq? Given that it's ultimately just a bunch of markdown lists in flat files, it doesn't even seem like it would need Logseq specific integration anyway. Can one of the llama projects read notes from local files and answer questions about them?
- 3abiton 3y agoThis tool also connects via OpenAI's API. If you are comfortable with that, I'm waiting for local model support.
- timetraveller26 3y ago"Generate questions: Extract thought-provoking questions from your markdown files, encouraging deeper contemplation and critical thinking." Yeah... I don't think we need more self congratulating note taking questions /s
- tinix 3y agoisn't this antithetical to the spirit of obsidian? tossing one's private notes to openai seems odd for such a tool that is marketed this way: > Obsidian is the private and flexible writing app that adapts to the way you think. > Obsidian stores notes on your device, so you can access them quickly, even offline. No one else can read them, not even us.
- spondyl 3y ago> isn't this antithetical to the spirit of obsidian? I think so, yeah. Kepano (CEO of Obsidian, the company) has stated that they're waiting for the technology to progress to where models can be run locally. Anything goes with third party plugins of course EDIT: Jumping to timestamp 50:08 is a good summarisation of their thoughts from the man himself: https://youtube.com/watch?v=TkNTuFF2t-c https://youtube.com/watch?v=TkNTuFF2t-c
- technovangelist 3y agoBut I think we are there for most tasks. I thought a bit about this today and wrote a blog post about how to use a local ai tool, Ollama, to summarize, make links, and ask questions about my notes. It’s not a fully fleshed out plugin but shows the start in about 20 or so lines of typescript. Check out the post here https://ollama.ai/blog/llms-in-obsidian https://ollama.ai/blog/llms-in-obsidian and a companion video here: https://youtu.be/ubXNPHpYREE https://youtu.be/ubXNPHpYREE
- alanbernstein 3y agoDid you miss the "flexible" part of your quote? Obsidian's value proposition is not simply "private". It does a lot more than that. Personally, I'm interested in this sort of functionality because I would like my notes to be densely cross-referenced whenever possible, automatically and intelligently.
- tomComb 3y agoMost obsidian user seem to have tons of community plug-ins installed, so I wouldn’t really call them privacy obsessives. And obsidian has lots of other great features, so I think it is an attractive option even for people who don’t really care about the ‘local’ angle.
- cellularmitosis 3y agoUnfortunate choice of name, there is already a programming language with the same name.
- deleted 3y ago[deleted]