9 ms·
PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
- AJRF 3y agoIs there a company that makes a hosted version of something like this? I quite want a little AI that I can feed all my data to to ask questions to.
- luccasiau 3y agohttps://libraria.dev/ https://libraria.dev/ offers this and more as a service. It has added conveniences like integration with your google drive, youtube videos, and such
- egonschiele 3y agoDepending on the size of your data, chiseleditor.com is a free option.
- tikkun 3y agohttps://news.ycombinator.com/item?id=36649777 https://news.ycombinator.com/item?id=36649777
- sdan 3y agoIf you subscribe to ChatGPT plus, you can use ChatWithPDF (https://plugins.sdan.io https://plugins.sdan.io) which has 50k+ daily active users!
- _pdp_ 3y agohttps://chatbotkit.com https://chatbotkit.com
- csjh 3y agoWhy have the OpenAI dependency when there's local embeddings models that would be both faster and more accurate?
- chaxor 3y agoThe most frustrating thing about the many, many clones of this exact type of idea is that pretty much all of them require OpenAI. Stop doing that. You will have way more users if you make OpenAI (or anything that requires cloud) the 'technically possible but pretty difficult art of hoops to make it happen' option, instead of the other way around. The best way to make these apps IMO is to make them work entirely locally, with an easy string that's swappable in a .toml file to any huggingface model. Then if you really want OpenAI crap, you can make it happen with some other docker secret or `pass` chain or something with a key, while changing up the config. The default should be local first, do as much as possible, and then if the user /really/ wants to, make the collated prompt send a very few set of tokens to openAI.
- zikohh 3y agoHave you seen PrivateGPT. It's quite good and free.
- yoyopa 3y agowhat hardware do you need for that?
- drdaeman 3y agoConsumer-grade, AFAIK it's GPT4All with LLaMA.
- EGreg 3y agoLink?
- hoopsman 3y agoPresumably https://github.com/imartinez/privateGPT https://github.com/imartinez/privateGPT
- raffraffraff 3y ago
- hi 3y agoKeep your data private and don't leak it to third parties. Use something like privateGPT (32k stars). Not your keys, not your data. "Interact privately with your documents using the power of GPT, 100% privately, no data leaks"[0] [0] https://github.com/imartinez/privateGPT https://github.com/imartinez/privateGPT
- leach 3y agoHow does this run on an Intel Mac? I have a 6 core i9. Haven't been able to get an M series yet so Im wondering if it would be more worth it to run it in a cloud computing environment with a GPU.
- hi 3y ago>Mac Running Intel When running a Mac with Intel hardware (not M1), you may run into clang: error: the clang compiler does not support '-march=native' during pip install. If so set your archflags during pip install. eg: ARCHFLAGS="-arch x86_64" pip3 install -r requirements.txt https://github.com/imartinez/privateGPT#mac-running-intel https://github.com/imartinez/privateGPT#mac-running-intel
- leach 3y agoI’m curious about the response times though, i imagine they will be quite slow on an intel Mac
- 3rdrodeo 3y ago[flagged]
- EGreg 3y agoOh, not this again. No, it's not a net negative in ALL forms. And if you really were concerned about downsides, AI has a ton more potential downsides than Web3 ever did, including human extinction, as many of its top proponents have come out and publicly said. Nothing even remotely close to that is the case for Web3 at all: https://www.nytimes.com/2023/05/30/technology/ai-threat-warning.html https://www.nytimes.com/2023/05/30/technology/ai-threat-warn... https://www.theguardian.com/technology/2023/may/10/ai-poses-existential-threat-and-risk-to-health-of-millions-experts-warn https://www.theguardian.com/technology/2023/may/10/ai-poses-... https://www.theguardian.com/technology/2023/may/30/risk-of-extinction-by-ai-should-be-global-priority-say-tech-experts https://www.theguardian.com/technology/2023/may/30/risk-of-e... https://www.theverge.com/2023/5/30/23742005/ai-risk-warning-22-word-statement-google-deepmind-openai https://www.theverge.com/2023/5/30/23742005/ai-risk-warning-...
- quickthrower2 3y agoThe author has a demo of this here: https://www.swamisivananda.ai/ https://www.swamisivananda.ai/
- raghavankl 3y agoI appreciate you finding and sharing this demo. I have also written a blog post on the vision of building a personal ChatGpt here https://devden.raghavan.studio/p/chatgpt-using-your-own-data https://devden.raghavan.studio/p/chatgpt-using-your-own-data
- syntaxing 3y agogpt4all has this truly locally. I recommend those with a decent GPU to give it a go.
- fbdab103 3y agoI assume this is the link: https://github.com/nomic-ai/gpt4all https://github.com/nomic-ai/gpt4all ?
- JimmyRuska 3y agoAnyone know how milvus, quickwit, pinecone compares? I've been thinking about seeing if there's consulting opportunities for local businesses for LLMs, finetuning/vector search, chat bots. Also making tools to make it easier to drag and drop files and get personalized inference. Recently I saw this one pop into my linkedin feed, https://gpt-trainer.com/ https://gpt-trainer.com/ . There's been a few others for documents I've found https://www.explainpaper.com/ https://www.explainpaper.com/ https://www.konjer.xyz/ https://www.konjer.xyz/ Nope nope, wouldn't want to compete with that on pricing. Local open source LLMs on a 3090 would also be a cool service, but wouldn't have any scalability. Are there any other finetuning or vector search context startups you've seen?
- tikkun 3y agoPinecone and Milvus would be alternatives for their use of FAISS for the vector store and search component. I think more of the embeddings difference would be noticed by what’s used for creating the embeddings (eg the ones here https://news.ycombinator.com/item?id=36649579 https://news.ycombinator.com/item?id=36649579 instead of the OpenAI embeddings API they used), rather than noticing differences from the embedding store/search alternatives which I can’t think of what the difference would be other than maybe performance at a large scale and cost and personal preference / developer experience. Hadn’t heard of Quickwit but from a quick glance at their site it doesn’t look like a vector store, seems perhaps unrelated. For tools for making custom ChatGPTs see my list: https://llm-utils.org/List+of+tools+for+making+a+%22ChatGPT+for+your+data%22 https://llm-utils.org/List+of+tools+for+making+a+%22ChatGPT+... Fine tuning as a service there’s Lamini AI, aimed at enterprises. Other embeddings startups there’s Weaviate.
- eddieweng 3y agoqdrant is better in my opinion
- sdan 3y agoI am working on a simple vector db just with numpy: https://github.com/sdan/vlite https://github.com/sdan/vlite I think milvus, quickwit, and pinecone are geared more towards enterprise and are hard to use.
- zikohh 3y agoAlso what does this do that llamaindex doesn't?
- einpoklum 3y agoDon't build a personal ChatGPT, and don't let OpenAI, Microsoft and their business partners (and probably the US government) have a bunch of your personal and private information.
- WhackyIdeas 3y agoSo avoid all Microsoft products too?
- deleted 3y ago[deleted]
- cj 3y agoDoes Microsoft have an AI opt out? AWS has an AI opt out at the organizational level that prevents them from using your data to “improve” their other services. (I personally recommend everyone opt out now in AWS if you haven’t already…) https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_ai-opt-out.html https://docs.aws.amazon.com/organizations/latest/userguide/o...
- tedsanders 3y agoBy default, data sent to the OpenAI API is never used for training and is deleted after a maximum of 30 days (mostly). Data usage policies: https://openai.com/policies/api-data-usage-policies https://openai.com/policies/api-data-usage-policies Data usage policies by model: https://platform.openai.com/docs/models/how-we-use-your-data https://platform.openai.com/docs/models/how-we-use-your-data
- two_handfuls 3y agoSo, they don’t promise they won’t look at it - just that they won’t use it for training.
- einpoklum 3y agoYou many want to read about National Security Letters: https://www.eff.org/issues/national-security-letters/faq https://www.eff.org/issues/national-security-letters/faq
- gigel82 3y agoI don't get it, GPT-2 is (one of the few) open models from OpenAI, you can just run it locally, why would you use their API for this? https://github.com/openai/gpt-2 https://github.com/openai/gpt-2
- sumedh 3y agoI am assuming GPT 4 will provide better answers to your queries compared to GPT 2.
- simonw 3y agoIt's not using GPT-2 - the README is incorrect. It's using "from langchain.embeddings import OpenAIEmbeddings" - which is the OpenAI embeddings API, text-embedding-ada-002 The only aspect of GPT-2 this is using is GPT2TokenizerFast.from_pretrained("gpt2") - which it uses as the length function to count tokens for the RecursiveCharacterTextSplitter() langchain utility. Which doesn't really make sense - why use the GPT-2 tokenizer for that? May as well just count characters or even count words based on .split(), it's not particularly important how the counting works here.
- raghavankl 3y agoThe embedding model used is the default OpenAI API embedding, text-embedding-ada-002. GPT2 is only used during the tokenization process to calculate token lengths efficiently. I have updated my readme to reflect this information correctly.
- eminent101 3y agoIs it going to send my personal data to OpenAI? Isn't that a serious problem? Does not sound like a wise thing to do, not at least without redacting all sensitive personal data from the data. Am I missing something?
- baby_souffle 3y agoThis is my question as well. Is there a more nuanced way to tell how personal data is used other than confirming that an OpenAI key is or is not needed?
- tedsanders 3y agoBy default, data sent to the OpenAI API is never used for training and is deleted after a maximum of 30 days (mostly). Data usage policies: https://openai.com/policies/api-data-usage-policies https://openai.com/policies/api-data-usage-policies Data usage policies by model: https://platform.openai.com/docs/models/how-we-use-your-data https://platform.openai.com/docs/models/how-we-use-your-data
- nomilk 3y agoA few weeks ago GitHub made a strong statement about code in repos not being viewed by humans, that was very liberating. If OpenAI could offer similar privacy statements it would immediately be much more useful. E.g. if they simply add a 'private' option, I'd pay double or triple for it. OpenAI's tools are incredibly good and so easy to use, it's just that I simply cannot use them for most the things I want to do with them because of the privacy considerations, and that sucks. I suspect OpenAI value the insights they get from looking at the data more than they do the extra revenue they'd receive if they could ensure privacy.
- cloudking 3y agoAm I the only one who doesn't need to search across my data? What are the use cases here
- BeetleB 3y agoExample use case: We have a group at work that meets and discusses various investment topics. The guy organizing it is fairly well connected and every week he tries to get an external speaker to come and present. Very educational. I have raw notes for each of these presentations. My goal has always been to go through those notes, and properly organize the knowledge in there into a wiki of sorts. It's been 3 years since this all started and I still haven't found the time to do it. If I want to be realistic, I should accept that it'll never happen. How do I go about finding information that I have in those notes? I could use text search but it's too sensitive to my search string - I'll often fail to find what I need. Also, the information may be scattered across several files, and I'd have to open all the hits and scan to find what I need. With technology like this, I can put all my notes into some vector DB, and then use AI to ask in plain English what I need. Locally the system interprets my query and finds the most relevant documents in the DB. It then sends my query and those hits to OpenAI to interpret my question, and find the answer amongst my notes. A while ago I used Langchain to set it up and I got it working as a proof of concept. An Aha moment was when I asked it something and it gave me a response with information that was scattered over two different presentations. My challenge is that there are so many parameters I could play with, and I haven't yet thought of a way/metric to assess the performance of my system (any pointers would be appreciated!) There's nothing personal in these notes, so no privacy concerns. I did want to set a similar thing up with over 20 years of emails, but didn't due to privacy. Also, I use a mail indexer (notmuch) which is fairly good so the need to use AI is not as strong. But for other (non-personal) notes? If I can get this system working fairly well, it'd be a life saver. I've made so many notes on so many topics over the years, and it's worth real money not to have to organize it well. Just let me write my notes, and use an AI to retrieve what I need.
- theonlybutlet 3y agoYou're creating additional hardship for yourself. Why create a pdf only to convert it out of pdf again. Just insert all your notes into the LLM model.
- Imnimo 3y agoThis readme is very confusing. It says we're going to use the GPT-2 tokenizer, and use GPT-2 as an embedding model. But looking at the code, it seems to use the default LangChain OpenAIEmbeddings and OpenAI LLM. Aren't those text-embedding-ada-002 and text-davinci-003, respectively? I don't understand how GPT-2 enters into this at all.
- raghavankl 3y agoThe embedding model used is the default OpenAI API embedding which is text-embedding-ada-002. GPT2 is only used during the tokenization process to efficiently calculate token lengths.
- deleted 3y ago[deleted]
- bella001 3y ago[dead]
- emmender 3y agoPlease provide this reference in your readme / blog as it is the original source for your work... and provides the background for the tradeoff between the 2 approaches: 1) fine-tuning vs 2) Search-ask https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb https://github.com/openai/openai-cookbook/blob/main/examples...
- raghavankl 3y agoI respect OpenAI for creating a comprehensive cookbook, and my tooling uses OpenAI for embeddings and chat completion which I have mentioned in the Readme. However, it was not built using a single reference or code example, and rather it is a combination of ideas from huggingface, openAI and langchain documentation.
- N4au5n 3y agoI’m working for a company that works as a security layer between any sensitive enterprise data and the LLMs. Regardless of the model (HF, ChatGPT, Bard), and regardless of the medium - conversational data, pdf, knowledge bases like Notion etc. It hides the sensitive data, preventing risky use and fact checking at the same time. Happy to make an intro if that’s what you’re looking for! tothepoint.tech