7 ms·
Show HN: Lumos, a local LLM Chrome extension
Lumos is an LLM co-pilot for browsing the web, powered by local LLMs (Ollama).
- Summarize long threads on issue tracking sites, forums, and social media sites
- Summarize news articles
- Ask questions about reviews on business and product pages
- Ask questions about long, technical documentation
- What else?
- dbdoskey 3y agoI have really been looking for something that summarizes all the websites I visit, and then pushes that into a vector database, to allow me to easily search through the contents of my history in an easy and fast way.
- gabev 3y agoShameless plug but this is precisely what Zenfetch does: https://zenfetch.com https://zenfetch.com Happy to elaborate more. We are gearing up for a wider HN launch soon
- orbisvicis 3y agoI too have been looking for a memex such as this, as well as a method for visualizing tabs I wish to save.
- bluerooibos 3y agoAhh, the 2024 equivalent of having thousands of bookmarks, 99% of which will never see the light of day again.
- smusamashah 3y agoI made an extension (its on github) to ask questions on selected text via chatgpt (by simply sending selected text). Can this do that too, as in asking question about the highlight text?
- andrewnguonly 3y agoThe app doesn't have this functionality, but I think it's a great idea. It's a nice (precision) alternative to the current scraping implementation. Tracking issue here: https://github.com/andrewnguonly/Lumos/issues/40 https://github.com/andrewnguonly/Lumos/issues/40
- helloericsf 3y agowhy do you have to do it with a local LLM? Plenty of extensions do the same stuff with more powerful LLM.
- kin 3y agoMost useful would be to browse Discord in the web view and summarize all the noisy chats Would also be cool to have the extension click into every link of search results and spit out an aggregate of the results
- andrewnguonly 3y agoI hadn't considered Discord, but good idea! I personally use the Discord Mac app. I'll look into figuring out how to parse the chats.
- smusamashah 3y agoWould it be a good idea to allow people putting in their own parsers for any app they need using css selectors or a combination with some JS in settings of the extension?
- andrewnguonly 3y agoYup, still working on it! Open PR: https://github.com/andrewnguonly/Lumos/pull/34 https://github.com/andrewnguonly/Lumos/pull/34
- giovannibonetti 3y agoGood idea. I'll try it with Whatsapp through web.whatsapp.com
- andrewnguonly 3y agoSomeone else mentioned Slack as well. I guess web apps for chat products are actually a big use case.
- hiatus 3y agoI have been looking for something like this! I noticed you suggest setting the ollama host as a variable (I personally run ollama on a beefier desktop and query remotely), hopefully that will translate over to the options page as well (I noticed the options page seems to have the ollama host hardcoded, though https://github.com/andrewnguonly/Lumos/pull/34/files#diff-83823e43e1951243df37ebb39915e62af9ed56ce1763f560b492056f9abe8472R19 https://github.com/andrewnguonly/Lumos/pull/34/files#diff-83...)
- andrewnguonly 3y agoYes! I'm planning to migrate the Ollama hostname configuration to the Options page as well.
- regularfry 3y agoUntil then, there's SSH port forwarding.
- nkko 3y agoWould be great to be able to query the LLM about search history, for example I know that I have seen something but can't remember exactly where or what.
- fragmede 3y agohttp://rewind.ai http://rewind.ai is working on something like this as a product.
- andrewnguonly 3y agoGreat idea. I created an issue to keep track of this. https://github.com/andrewnguonly/Lumos/issues/35 https://github.com/andrewnguonly/Lumos/issues/35
- anthuswilliams 3y agoThis is great! I would love to extend it to hide sponsored results, ads, and call-to-action popups.
- tourmalinetaco 3y agoWould it have any practical applications that you can see beyond uBlock Origin, or is it just for the cool factor?
- anthuswilliams 3y agoI imagine you'd use it in combination with uBlock Origin. The latter would prevent ads from being loaded in the first place, but when the site itself is presenting you with sponsored results, email signup overlays, and so on, you'd use a content filter like this to prevent them from rendering. I know you can accomplish a lot of this kind of direct content filtering with uBlock Origin, but as far as I'm aware you have to configure e.g. CSS selectors per site.
- gabev 3y agoThis is sick! We've built a similar product for LLM-powered browsing at https://zenfetch.com https://zenfetch.com Admittedly, we use cloud hosted providers since we found the quality to be much higher, though awesome to see local versions as well.
- andrewnguonly 3y agoVery cool! Have you considered local LLMs for a free/lite tier? I do agree that 3rd party LLM services (e.g. OpenAI) are much more powerful. For basic tasks (e.g. summarization, Q&A), local LLMs seem to perform decently.
- gabev 3y agoWe have considered it, though most of our users use Zenfetch for things like 1. Research across an industry 2. Generating new content 3. Connecting ideas / building networked thought For that reason we were deterred from using local models. Hopefully the Mistral team and similar keep pushing out new developments to get comparable with GPT-4 levels
- alwayslikethis 3y agoIs there a firefox version?
- andrewnguonly 3y agoI didn't build a dedicated Firefox version, but I suspect there's a way to install it without significant code changes. I actually just tried and I ran into an error: `background.service_worker is currently disabled` I'll figure out a workaround and document the steps to install: https://github.com/andrewnguonly/Lumos/issues/36 https://github.com/andrewnguonly/Lumos/issues/36
- regularfry 3y agoCan you give it YouTube video transcriptions?
- andrewnguonly 3y agoI haven't tried it, but in general, it accepts any text. Or are you asking if it can automatically get the transcript from a YouTube video? At the moment, it can't do the latter. I do plan to support multi-modal LLMs (llava via Ollama), but I'm not sure how the mechanics of the extension will work yet. Open issue: https://github.com/andrewnguonly/Lumos/issues/27 https://github.com/andrewnguonly/Lumos/issues/27
- fragmede 3y agohttp://YouTubetranscript.com http://YouTubetranscript.com might be of use here.
- regularfry 3y agoI was thinking specifically about getting it the youtube-provided transcript of a video you're watching, rather than doing the transcription itself. With access to the transcript, I could say to it "find me anything in my browser history related to this subject" (or whatever) and have it understand what I've been watching.
- castles 3y agoI've been working on something in the same vein, though just using manual selections, or selections auto expanding from the visible viewport. I really like the per-site parsers, which make a lot of sense Congrats :)
- andrewnguonly 3y agoThanks! I elaborate a little bit more on my thinking and approach for parsing in this blog post (4 min read): https://medium.com/@andrewnguonly/local-llm-in-the-browser-powered-by-ollama-part-2-6eb10caf39a1 https://medium.com/@andrewnguonly/local-llm-in-the-browser-p...
- castles 3y agoI had similar ideas on the back burner for per site scripts. MV3 sure makes things annoying at times! I want to build something that's kind of a mix between what you're doing here and what I've been experimenting with. Did you experiment with the sidePanel api yet? You can open it, but the <input> will not get focus like with the popups. You'd think at least with the commands keybindings, or reacting to the browser action icon click this should be possible?! I asked Lumos some questions and it's hitting the embeddings endpoint many times, and then when I asked a follow up question, it hit the same endpoints again, each taking a few hundred ms. It's not caching them? TBD? p.s. iirc There's some way to set the key in the manifest such that you'll get a static id, useful for OLLAMA_ORIGINS where you don't need * in the examples. p.p.s starling-lm is really great for a default local model for these types of things edit: https://github.com/andrewnguonly/Lumos/pull/38 https://github.com/andrewnguonly/Lumos/pull/38
- andrewnguonly 3y agoI did not look into the sidePanel API yet. I will look though. I'm still fairly new to Chrome extension development. I just discovered the Options page heh... Regarding the caching -- the entire RAG workflow needs an update. I'll spend some time on it. Thanks for calling out and thanks for the PR!
- gnicholas 3y agoI'm not very technical, but I use ollama on my Mac and know how to side-load browser extensions. Can you ELI5 what I need to do besides downloading and side-loading the extension, and ensuring I have the same model downloaded and running on my machine?
- andrewnguonly 3y agoFor installation, there's nothing else you need to do. It's ready to go. If you want to configure a custom content parser for a specific website (i.e. websites you visit frequently), you'll need to modify `contentConfig.ts` according to the instructions here: https://github.com/andrewnguonly/Lumos?tab=readme-ov-file#custom-content-parser https://github.com/andrewnguonly/Lumos?tab=readme-ov-file#cu... Sorry, I realize these instructions are probably unclear. My approach for parsing content is a bit hacky. I need to create a video tutorial/documentation to show how to inspect the webpage and pick out content to parse. Created this issue to track: https://github.com/andrewnguonly/Lumos/issues/37 https://github.com/andrewnguonly/Lumos/issues/37
- gnicholas 3y agoIt looks like compilation is required, is that right? I don't have NPM installed, and would love to have a zipped version that I can just unzip and sideload. Please let me know if I'm misunderstanding what I'm supposed to do here.
- andrewnguonly 3y agoGot it, you're right. I created a release that includes the packaged code (so you don't need to run `npm`, etc): https://github.com/andrewnguonly/Lumos/releases/tag/1.0.0 https://github.com/andrewnguonly/Lumos/releases/tag/1.0.0 Unzip `lumos_1_0_0.zip` and you can load the unpacked extension.