7 ms·
Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools
Hey everyone,
We have been building SecureAI Tools -- an open-source application layer for ChatGPT and ChatPDF-like AI tools.
It works with locally running LLMs as well as with OpenAI-compatible APIs. For local LLMs, it supports Ollama which supports all the gguf/ggml models.
Currently, it has two features: Chat-with-LLM, and Chat-with-PDFs. It is optimized for self-hosting use cases and comes with basic user management features.
Here are some quick demos:
* Chat with documents using OpenAI's GPT3.5 model: https://www.youtube.com/watch?v=Br2D3G9O47s
* Chat with documents using a locally running Mistral model (M2 MacBook): https://www.youtube.com/watch?v=UvRHL6f_w74
Hope you all like it :)
- deleted 3y ago[deleted]
- cheema33 3y agoI am very new to this field, so forgive my ignorance when I ask super basic questions. 1. Does chat-with-pdfs function work with scanned PDFs? 2. In the video example for chat-with-pdfs you show uploading a document interactively. The part of processing is quite slow. Can the tool be fed these documents offline as well? My use case for each user, there are many, to have their own list of documents that they upload. They come back later, after the LLM has had a chance to process all documents and can ask questions about their own documents only. Is something like this possible?
- d7y 3y agoGreat questions. > 1. Does chat-with-pdfs function work with scanned PDFs? Not yet. We don't do OCR or anything to extract text from images yet. But that would be an awesome feature, so we would love to add it in the future. > 2. In the video example for chat-with-pdfs you show uploading a document interactively. The part of processing is quite slow. Can the tool be fed these documents offline as well? Not as of right now. But we do have plans to make that an offline/background job so that we can feed a larger corpus of documents into it and query against it later.
- smeej 3y agoIf I've already run OCR on my PDFs and that's added now as an invisible layer, would it work then? I've had a workflow digitizing my incoming paper documents, running OCR, and tagging them, all locally, and it would be great to have an easy front-end to talk to them.
- d7y 3y agoI haven't tried this myself, but I think it should work. It would be worth trying at least, so I highly encourage you to play with it, and file issues if you find any issues with it.
- lhuser123 3y agoI haven't found an OCR tool reliable enough when it comes to scanned PDFs containing financial data where accuracy of amounts in the document is very important.
- catlifeonmars 3y agoPeople spend an inordinate amount of time and money solving this problem rather than spending the same amount of money in lobbying and standardization efforts for financial institutions. I’ll throw this out there: when all you know is a hammer, everything looks like a nail.
- rolisz 3y agoHave you tried Azure Document Intelligence? I've had very good results with it.
- vunderba 3y agoGreat job. This is a relatively crowded area, particularly RAG style chat systems. It might be nice for SecureAI to call out what makes their product different from other open source players in the same space, specifically Khoj and Danswer, both of which allow you to chat with your documents, offer network authentication, and allow you to plug in your own LLM. Danswer https://github.com/danswer-ai/danswer https://github.com/danswer-ai/danswer Khoj https://github.com/khoj-ai/khoj https://github.com/khoj-ai/khoj
- SubiculumCode 3y agoDo you happen to know of some that have been integrated into a slack chatbot?
- vunderba 3y agoI've definitely seen a few that can do this, they're mainly positioned as automatic assistance for technical support. Danswer has a slack connector, so it might be what you're looking for.
- deleted 3y ago[deleted]
- d7y 3y agoA great question. We are trying to build a single platform for all the AI tool needs. Chat-with-LLM and chat-with-documents are just a couple of apps or experiences that we have started with, but we have ambitious goals. In future, we would love to provide an SDK that exposes common abstractions and lets everyone build apps/experiences for the long tail of use cases.
- abnry 3y agoIs there a good ML tool for renaming PDFs? There are some tools out there but they assume a journal format.
- Reptur 3y agoI tend to just select all the files and then paste their filenames into a Chat AI and tell it to write mv commands to rename them however I need. May be more complicated if you need it to open the PDF to get content for the rename though.
- d7y 3y ago> renaming PDFs Sorry, I didn't understand. Why do you need ML tool for renaming PDFs? or did you mean rephrasing or rewriting in a different format?
- mattsan 3y agoMaybe they mean walking a file tree and cleaning up file names to match the titles on the covers of them more accurately?
- abnry 3y agoExactly. You download something from arxiv and the filename has no meaningful content in it. Generally speaking, you want the filename to be descriptive in some way, extracting the title of the document is a good start.
- bravura 3y agoYou can just import them all into paperpile, which has good ways of inferring the metadata like title author year etc. and then connect to your google drive. Which will download them with nice filenames.
- jart 3y agoI'm not able to use this locally on Alpine Linux because ollama needs glibc.
- yreg 3y agoI'm not able to use this on TempleOS, please fix.
- testernews 3y agothis doesn’t work on Nix, pls fix
- stuzenz 3y agoI also use NixOS, but if the target was not NixOS, I don't think you should be requesting the set up as needing fixing by the author. It just doesn't sound right - or maybe it is just me. NixOS isn't the defacto standard, and breaks the Linux FHS to achieve all the good stuff it does do. Either try to package it or use a docker image or maybe raise an issue noting the blocker and request it as a feature for some changes to give an easier path for to having it build more easily for NixOS Apart from that, as expected, the docker image that is produced following the instructions is working fine with NixOS as host. All it needed for the build was the openssl packaged on the host.
- hruzgar 3y agoYou guys really didn't get the joke, right? <-)
- testernews 3y agolol I thought we were all joking on this thread???
- jart 3y agoYes but they claim it runs on Linux when it runs on Systemd.
- caseyf7 3y agoHow do you get value from chatting with documents? I can scan and read a pdf faster than I can chat with an AI about it. There must be more to it than I realize.
- netcraft 3y agowhat if you want to extract certain information from 100 pdfs?
- naiv 3y agoAlso if they are in spanish?
- katrinarodri 3y agoyou mean ability to translate PDFs into english?
- capableweb 3y ago> There must be more to it than I realize. PDF material comes with different information density. If you have a lose collection of 100 manuals, and you need to find a snippet of information that could be in 10 different ones, I'm guessing something like this can help you navigate and locate what you need.
- gosub100 3y agoThat would be a great litmus test for these programs. Dump gigabytes of manuals and ask "how many pins does the 74LS04 have?" "What size bolts hold the oil pan on a 73 Porsche?"
- freedomben 3y agoa one-page PDF, sure. But if it's a 500 page pdf of a law and/or regulation, then definitely not.
- bravura 3y ago
- mikeytown2 3y agoWhat LLM is best for giving it your entire code base and database structure and then using it to help with your project?
- oaththrowaway 3y agoMaybe Refact? They have a self hosted version that can index your repos. They have their own LLM or you can use others. Nice tooling for VS Code as well
- hifreq 3y agoI couldn't find this info in the readme... does this tool anonymize ChatGPT requests? What does it mean that it's a private an secure tool in the context of using ChatGPT?
- glerk 3y agoMy understanding is that this doesn't use ChatGPT at all for the "private and secure" case. It is running LLM models locally using ollama and just provides a ChatGPT-like interface.
- willi59549879 3y agoit runs an llm in a docker container. doesn't send any requests to chatgpt
- hifreq 3y agoThe demo video is using ChatGPT.
- d7y 3y agoIt is secure because it allows you to fully customize where to process the data (i.e. LLM inference), where to store it, and data-retention policies, etc. You can choose to use a locally running LLM (like it does in my second video) or use a secure third-party service provider like Azure OpenAI. For example, if you want GDPR compliance, then you can choose Azure OpenAI running in the EU region. For HIPAA compliance, you should choose a service provider that provides the Business Associate Agreement (BAA). You can even run it in air-gapped facilities (like GitLab's offline mode [1]). In all of these cases, you can always run an Ollama-like inference service on your infra and point SecureAI Tools to it) [1]: https://docs.gitlab.com/ee/topics/offline/ https://docs.gitlab.com/ee/topics/offline/
- teruakohatu 3y agoCan a directory of PDFs be queried or does it only support a single document?
- d7y 3y agoHi there! We just added this in the latest release (v0.0.2). You can now create a document collection and upload as many PDFs into it as needed. The documents are processed in the background and once processing finishes, you can create as many chats with it as needed. Quick demo: https://youtu.be/PwvfVx8VCoY https://youtu.be/PwvfVx8VCoY Installation Instructions: https://github.com/SecureAI-Tools/SecureAI-Tools?tab=readme-ov-file#docker-compose-recommended https://github.com/SecureAI-Tools/SecureAI-Tools?tab=readme-... Please try it out, and let me know how it goes. We're always looking to improve the tool so let us know if you have any feedback for us :)
- d7y 3y agoRight now it supports selecting & uploading a _few_ PDFs on chat-creation. Those PDFs get indexed online -- i.e. while the user waits. So it doesn't scale well with the number of PDFs selected in a chat because you'd have to wait that long before the chat responds with your initial question/prompt. We plan to make this indexing process offline, where you can create a document collection based on either a directory upload or an integrated data source like Google Drive, Notion, Confluence, etc. Then the system would start indexing that collection in the background and notify you once indexing is complete. Once a collection is indexed, users can select it when creating a new chat and query against it. Let us know if you have any thoughts on this proposed solution.
- lysecret 3y agoI see many of my friends building some kind of RAG system with chat interface. I have been building some stuff on top of the OpenAi interface (to use their store) but find myself wanting to implement some simple UI elements (like a date selected or a simple dashboard). So I feel like these types of apps have a few re occurring elements: 1. A chat interface „frontend“ (with threads, interfaces to popular APIs or local models) nice Ui ideally extensibility to some custom UI elements authentication etc. 2. API calls. (E.g. like OpenAI actions) Simplest case just reading and writing to a db (simple crud). 3. Local data + RAG. With a custom retrieval/search logic could be embeddings or simpler search methods. Do you know open source software for all three elements? Of course you can piece it together and maybe this is the best approach. But maybe you could build something integrated.
- dluc 3y agoabout #3 I’ll recommend https://github.com/microsoft/kernel-memory https://github.com/microsoft/kernel-memory :)
- d7y 3y agoI love the idea of allowing anyone to build apps or experiences on top of some of these common elements. We have briefly discussed an approach where we make some of these common elements available as abstractions and let people build "apps" on top of it. It would operate kind of similar to how Google's app store does in that the head use cases (email, photos, camera, etc) are first-party apps, but then anyone can build and publish a third-party app using the Android SDK.
- foxhop 3y agohttps://github.com/russellballestrini/flask-socketio-llm-completions#flask-socketio-llm-completions https://github.com/russellballestrini/flask-socketio-llm-com... I'm building a similar app but uses python/socket.io
- lee101k 3y ago[dead]
- leeeeeepw 3y ago[dead]
- sunnysogra 3y agoNice tool. However, these days, people are looking for something that is more feature-rich and can perform various tasks. Building an RAG system with a chat interface is common. There are many ChatPDF alternatives available that work great. Here you can check one of the popular ChatPDf alternative that is capable of doing much more than ChatPDF. https://www.thesamur.ai/chatpdf-alternative https://www.thesamur.ai/chatpdf-alternative keep Building!