3 ms·

I've written something similar in python — a script that renames documents based on their contents using Ollama. It's very easy to write things like that, I rec

by caseyy 2y ago

I've written something similar in python — a script that renames documents based on their contents using Ollama. It's very easy to write things like that, I recommend anyone interested in local LLMs try a fun project such as that. My first question was why would this be a file system rather than an app or a script, but I see it's actually an Electron app and python scripts, which I think is the right approach. I think that something that would have a UI for automatically tagging, renaming, and moving files on request but not constantly would be very handy. Also, if you could somehow steer files into binning directories ("if X, put it in bin ~/X, if Y, put the file in bin ~/Y", "if it's an invoice or deals with payment, put it in ~/Documents/Finance"), that would be cool. Finally, Windows support would be amazing.

TylerE 2y ago
Well, I see very good reasons for writing it as an actual file system... it allows the rest of the ecosystem to interact with it on an equal footing.
btbuildem 2y ago
> automatically tagging, renaming, and moving files on request but not constantly Key point here -- I might be old, but I like my files to stay where/how I left them.
3abiton 2y ago
I feel naming this is rather personal as we all think differently, and organize files mirroring our way of thinking. I find it odd to have an LLM do that. How often do you use it and how does it fit withing your workflow?
- caseyy 2y ago
  I sometimes scan over 300 documents at a time. They are OCR-ed, but this does not make them easy to search when they are unindexed (on cloud drives, or on disk/tape for archival purposes). So it is important to name them and tag them with relevant keywords. Usually I just want the LLM to extract the document title and unique information (document numbers, companies mentioned on the document, people mentioned in the document, dates, and similar). The entire workflow is this: 1. Prepare a batch of documents with Patch-T codes and load them into an automatic feed scanner. 2. Scan documents with NAPS2 and OCR, producing PDF files as they are scanned. 3. My python script monitors the output directory, picks up batches of files at a time, and processes them sequentially. The first page of each document is loaded as text, the text is fed into llama2 (the last one I used) through Ollama with a few-shot prompt, roughly in-context trained for my sort of documents, it outputs a file name. 4. The file is renamed and moved to a processed directory. 5. Optionally, I will also run a second step to generate tags. I do not organize my every day files this way, it is specifically for digitizing and making redundant archival copies of old documents, which I collect. This is a bespoke pipeline that fits very well for my workflow. The renaming is very accurate after I have perfected the context that I pass to Ollama, and the final files are easily searchable on slow-read media. I have not yet added any kind of computer vision because I scan so few photographs that I can label them myself. I agree with you that naming is personal, but I think that we can train the LLMs to name things the way we expect them to be named. And at some point, the human cost of labelling large amounts of data may make certain workloads simply impossible. Interestingly, my pipeline is very slow. It runs on an old GPU and takes about 30 seconds to produce just the few tokens to name a file, and about the same for the tags. I will probably move it to an even slower pipeline on a Mac mini M1 when I have the time, just to save on electricity. Because ultimately, it finishes in a few hours and it doesn't take up any of my time. What would be a full-time labelling job for a human is now a full-time job for a machine, which makes the archival hobby feasible and cheap.
leobg 2y ago
I’m a lawyer and do the same thing. YYMMDD_ENTITY_TOPIC is a template I use a lot when dealing with lots of documents in large cases. Regex and Keyboard Maestro were my main tools before LLMS became a thing.

by caseyy 2y ago

TylerE 2y ago

Well, I see very good reasons for writing it as an actual file system... it allows the rest of the ecosystem to interact with it on an equal footing.

btbuildem 2y ago

> automatically tagging, renaming, and moving files on request but not constantly Key point here -- I might be old, but I like my files to stay where/how I left them.

3abiton 2y ago

I feel naming this is rather personal as we all think differently, and organize files mirroring our way of thinking. I find it odd to have an LLM do that. How often do you use it and how does it fit withing your workflow?

caseyy 2y ago
I sometimes scan over 300 documents at a time. They are OCR-ed, but this does not make them easy to search when they are unindexed (on cloud drives, or on disk/tape for archival purposes). So it is important to name them and tag them with relevant keywords. Usually I just want the LLM to extract the document title and unique information (document numbers, companies mentioned on the document, people mentioned in the document, dates, and similar). The entire workflow is this: 1. Prepare a batch of documents with Patch-T codes and load them into an automatic feed scanner. 2. Scan documents with NAPS2 and OCR, producing PDF files as they are scanned. 3. My python script monitors the output directory, picks up batches of files at a time, and processes them sequentially. The first page of each document is loaded as text, the text is fed into llama2 (the last one I used) through Ollama with a few-shot prompt, roughly in-context trained for my sort of documents, it outputs a file name. 4. The file is renamed and moved to a processed directory. 5. Optionally, I will also run a second step to generate tags. I do not organize my every day files this way, it is specifically for digitizing and making redundant archival copies of old documents, which I collect. This is a bespoke pipeline that fits very well for my workflow. The renaming is very accurate after I have perfected the context that I pass to Ollama, and the final files are easily searchable on slow-read media. I have not yet added any kind of computer vision because I scan so few photographs that I can label them myself. I agree with you that naming is personal, but I think that we can train the LLMs to name things the way we expect them to be named. And at some point, the human cost of labelling large amounts of data may make certain workloads simply impossible. Interestingly, my pipeline is very slow. It runs on an old GPU and takes about 30 seconds to produce just the few tokens to name a file, and about the same for the tags. I will probably move it to an even slower pipeline on a Mac mini M1 when I have the time, just to save on electricity. Because ultimately, it finishes in a few hours and it doesn't take up any of my time. What would be a full-time labelling job for a human is now a full-time job for a machine, which makes the archival hobby feasible and cheap.

leobg 2y ago

I’m a lawyer and do the same thing. YYMMDD_ENTITY_TOPIC is a template I use a lot when dealing with lots of documents in large cases. Regex and Keyboard Maestro were my main tools before LLMS became a thing.