9 ms·

Paperless-NGX

169 points by PetitPrince 5y ago

robinparriath 5y ago
If I understand this correctly, the original Paperless was archived (Archival notice)[https://github.com/the-paperless-project/paperless/commit/9b0063c9731f7c5f65b1852cb8caff97f5e40ba4 https://github.com/the-paperless-project/paperless/commit/9b...], so Paperless-NG was created. Now that Paperless-NG seems to be going unmaintained (last commit on 15th Sep 2021), Paperless-NGX has been created with a focus on an org, so that the continuity of the project can be maintained with a simple path for the original creators to join back if they want to. I don't think the community could have handled this better!
meibo 5y ago
Excited to see that this is the way the project will be going forward, been using Paperless-NG for quite a while now and it's a pleasure.
Semaphor 5y ago
The /r/selfhosted announcement [0] had some more information about it, including links to issues where the fork-plans were discussed. This is a great reminder for me to migrate ;) [0]: https://old.reddit.com/r/selfhosted/comments/tbcuf0/announcing_first_release_of_paperlessngx_the/ https://old.reddit.com/r/selfhosted/comments/tbcuf0/announci...
Semaphor 5y ago
For those who don't know, paperless is a Document Management System. More information in the docs [0]. It’s the 2nd fork, with -NG being the first one. They each rose up when the former version became unmaintained. [0]: https://paperless-ngx.readthedocs.io/en/latest/ https://paperless-ngx.readthedocs.io/en/latest/
phkx 5y ago
Interesting - will check it out to see whether it is better than my current approach, in which I scan to PDF-A including OCR and let Spotlight do the indexing. Some additions: I found that black-and-white scans at 300 dpi work for almost all documents, resulting in a small file size and decent readability. Occasionally I switch to gray and 200 dpi and rarely to color. After looking up how long the originals of different types of documents need to be kept for legal reasons, I settled simply for the maximum time (6 years in my case) and file documents in a binder, sorted by the year in which I can discard them. Then, at the beginning of each year, I can get rid off one section of the documents which are older than 6 years. There is a second binder for active contracts (insurance etc). As soon as one of them ends, it goes into the first binder. I‘ve started organizing my Downloads folder in a similar way - sorting stuff by when I think I can delete it (either because it‘s not relevant any more or because I simply never touched it), typically a few months in the future. Both systems have helped to keep the clutter low and.
wfriesen 5y ago
I just migrated my instance by repointing the webserver container image at the one in this repo. So far, so good.
throwaway290 5y ago
> I wrote this to make “going paperless” easier. I do not have to worry about finding stuff again. I feed documents right from the post box into the scanner and then shred them. Perhaps you might find it useful too. I know this strongly depends per country, but could anyone else share real-life experiences of going paperless? From my understanding, if there is a sheet of paper that has special meaning, presumably you must be able to produce the original, with wet stamps and signatures, upon request (since any copy can be created from scratch digitally). So I keep loads of papers I am afraid to dispose of.
- phkx 5y ago
  This is my understanding as an individual in Germany who is not self-employed: Original documents help you to make or defend against legal claims, so for the time being, I'd keep them during any limitation period which applies. Afterwards that paper is not really useful any more (but keeping a digital copy doesn't hurt). I suggest to find out, which periods apply in your jurisdiction. In my case, I simply settled for the maximum of all the different periods I encountered and file originals by the year after which I can trash them and use the digitized versions for actually working with them (e.g. my tax declaration is now much faster to do). For Germany, I found that 6 years grace period should be fine. 10 years, if you're self-employed.
- L3viathan 5y ago
  I use paperless-ng to go mostly paperless, but I keep the paper: I bought a simple self-incrementing stamp, which I stamp all incoming documents with. When adding the document to paperless-ng, I set the document's number accordingly, and finally I file the paper away purely sorted by ID (so I have a (physical) folder titled 1-101, one 102-231, etc.). When I need the original to a document, the lookup is very fast, and when I know I won't need the original ever again, I don't stamp it and tag it as "digital only" in paperless-ng.
  - caeruleus 5y ago
    Although I have not used paperless-ng (extensively) so far, I have been using almost the same method for many years and could not be happier. When I receive a document, I do the same every time: 1) Stamp it with the pagination stamp. 2) Scan it. 3) Dump the scan into a single large digital folder. 4) Dump the original into a physical folder (titled as you do). It relieves the mental burden of categorising since you do the same every time – stamp, scan, dump. And repeat. In the very rare case you need to find the original it's also superior, thanks to OCR and the sequential numbering being represented on the scan.
brunoqc 5y ago
I wish there was a standalone version, so my mom could use it. Or maybe a desktop + a mobile version that are synced without a server, so you could still be able to use your phone to scan new documents.
- uniqueuid 5y ago
  There is DevonThink [1]. Has been around for decades, does OCR, syncs to a huge bunch of services including WebDav, has mobile clients. It's a bit of an old paradigm, though - powerful, client-side, offers a lot of freedom and therefore wants you to put in manual work for setting up a system. I love it. [1] https://www.devontechnologies.com/apps/devonthink https://www.devontechnologies.com/apps/devonthink
  - vertis 5y ago
    I've been moving more and more to apps like Devonthink. I use Obsidian and Zotero, which while they different use cases are a similar 'old paradigm'. My thinking recently has been on the storing items in a way that will last. It seems every year another SaaS gets bought and/or shutdown. Text based storage (Obsidian/Markdown) and desktop/foss apps are one way to combat that.
    - solarkraft 5y ago
      If "old paradigm" means self-owned data and no dependence on a subscription that can go away at any time I'm all in on it. (Small) SaaS is not dependable.
  - LeSaucy 5y ago
    DEVONThink is criminally underrated. The mobile apps are fantastic. It's almost a super power being able to pull up any document/receipt I've dropped in my scanner over the last decade on my phone from anywhere.
- anthropodie 5y ago
  Host it on Raspberry PI at home. That's what I did.
moveax 5y ago
There is a linuxserver.io container image now, which should act as drop in replacement for the paperless-ng linuxserver image. Haven't tried it yet, but it looked good so far https://hub.docker.com/r/linuxserver/paperless-ngx https://hub.docker.com/r/linuxserver/paperless-ngx
anaisbetts 5y ago
As a user of Paperless I highly recommend setting it up, it is so incredibly useful as soon as you have to deal with any kind of paperwork-intensive activity (buying a house, getting a mortgage, paying taxes). Take literally everything you get and throw it into Paperless, then when someone asks for all of these documents it will take you minutes instead of hours to track them all down - when you need it, it's magic.
- einpoklum 5y ago
  Can you explain how it is better than carefully scanning and arrangement the scanned files yourself? (Not being facetious, I'm really asking.)
  - vertis 5y ago
    Not the parent, but I suspect it's not better in the outcome, just in the amount of effort you have to go into creating and arranging vs just throwing documents at it and searching later.
  - anthropodie 5y ago
    You can categorise those documents, tag them, OCR them, search them. Access them on phone or any other device with browser.
    - nojito 5y ago
      How is that any different from Onedrive/iCloud/etc?
    - h0l0cube 5y ago
      I'm looking at this ecosystem and wish I'd seen it sooner. I've been using the Google Drive Android app to scan documents. It does auto-cropping and perspective correction, but it doesn't OCR, and I manually rename the files so that I can order them by date and cross reference with my banking transactions. With OCR I might at least be able to extract some useful data and avoid some tedious data entry.
    - 5y ago
einpoklum 5y ago
IMHO, this seems to consist of two very separate pieces of software: Scanning, and complex management of an archive of files and documents. Both are interesting challenges, but focusing on the second one - I don't see why it should be tied-in with scanning. It's actually something that I feel is missing in operating systems, or at least desktop environments: The ability to arrange your files in more than one way, rather than having to force them into a static hierarchy ... while also not having them lost behind some piece of software which prevents direct access.
- mxuribe 5y ago
  > ...The ability to arrange your files in more than one way, rather than having to force them into a static hierarchy ... while also not having them lost behind some piece of software which prevents direct access... I wish all common operating systems supported labels/tagging in a compatible, easy-to-migrate manner...so that a person could migrate from windows to linux or back again, and all their file meta data - like file labels, file tags! - was preserved.
kloudleinc 5y ago
It's a very useful tool. I mainly use it to back up my documents instead of only keeping the paperless ones though.
gsich 5y ago
Automatic deskewing of images?
m3nu 5y ago
This was one of the first apps I added to run on PikaPods. So free to use during beta and I hope to set up a revenue sharing agreement with the project soon: https://www.pikapods.com/apps https://www.pikapods.com/apps
- gurjeet 5y ago
  PikaPods looks like an interesting idea. I invested some time setting up account, and trying to create Baserow pod. I was greeted with "Failure while adding container. Our team has been informed."
  - m3nu 5y ago
    Yeah, still in public beta and multi-container apps like Baserow are pretty new and marked as "experimental". Already looking into the error. Edit: Should work, but takes several minutes before the web UI becomes available. We do provide the logs for such cases. Most other apps are easier to deal with. :-)
    - nigggle 5y ago
      Baserow dev who worked on our new multi-container single image (baserow/baserow on dockerhub) here. I assume you are using the baserow/baserow image for pikapods (but perhaps not?). We have a few environment variables (https://baserow.io/docs/installation%2Fconfiguration https://baserow.io/docs/installation%2Fconfiguration) that can be tweaked for this image to make it start-up faster and with a lower resource footprint: - `SYNC_TEMPLATES_ON_STARTUP=false` to turn off the initial load of example Baserow templates. Our template collection is growing and the loading of this into the database is probably what is causing the several minute startup. This will be optimized in the future. - `BASEROW_RUN_MINIMAL=true`. This will combine two of the backend async queue processes into a single one. This might result into higher priority async tasks getting stuck behind slower tasks (an large import/export might slow down the broadcast of realtime events for example). But this tradeoff is perfectly reasonable in most small self hosted environments. - `BASEROW_AMOUNT_OF_GUNICORN_WORKERS=1` to reduce the number of concurrent backend api processes from the default of 3. Once again this might cause a degradation in performance for higher volumes of traffic, but a tradeoff worth considering if memory etc is a concern. This image is brand new and because of it's multi-process nature is hiding some interesting complexity. We are still working out the kinks and a sensible set of defaults. It's also very easy for us to offer different variants of this image with different defaults set. Any feedback, suggestions or questions are very welcome here or at http://community.baserow.io http://community.baserow.io.
upofadown 5y ago
Just to save others from having to look it up, paperless-xxx uses OCRmyPDF to do the work. So you don't have to host anything if you are willing to touch a file or two...
- subeadia 5y ago
  What do you mean by hosting? Both paperless-NGX and OCRmyPDF can be installed completely locally. Why not just install paperless which takes care of the OCR integration for you?
frameset 5y ago
I use Papermerge, which seems to be a very similar software. Does anyone have experience with both and prefer Paperless?
evolve2k 5y ago
At one point I was using Evernote combined with a Fujitsu Scanner to scan in all my paperwork and Evernote would automatically OCR the scans and make it easy for me to search for this later. Evernote is a commercial offering. Does this provide similar or better yet, better functionality? If I scan my days mail in one go.. say water bills, property expenses etc in one batch.. will this then sort the water bill from the telephone bill in what is saved? Ps. Bills contrived for illustration I receive much more of this online these days but still paperwork abounds.
- tenebrisalietum 5y ago
  Yes. It uses tesseract for OCR and gathers the OCR text and puts it in a database, and you can then search. So if you enter something you know is on the water bill then it'll come up in the search, such as the name of the water company. I've never used Evernote so I'm not sure it's better but I had no problem finding stuff so far. Also you can tag documents and search by tag so you have that too. Paperless kicks ass.
bravura 5y ago
This is cool, but can I pay for a managed service? I clicked through the documentation and I don't understand how easy/hard it is to make cloud backups and do disaster recovery.
nbenitezl 5y ago
This is also an interesting one: https://openpaper.work/en/ https://openpaper.work/en/

169 points by PetitPrince 5y ago

If I understand this correctly, the original Paperless was archived (Archival notice)[https://github.com/the-paperless-project/paperless/commit/9b0063c9731f7c5f65b1852cb8caff97f5e40ba4 https://github.com/the-paperless-project/paperless/commit/9b...], so Paperless-NG was created. Now that Paperless-NG seems to be going unmaintained (last commit on 15th Sep 2021), Paperless-NGX has been created with a focus on an org, so that the continuity of the project can be maintained with a simple path for the original creators to join back if they want to. I don't think the community could have handled this better!

meibo 5y ago

Excited to see that this is the way the project will be going forward, been using Paperless-NG for quite a while now and it's a pleasure.

Semaphor 5y ago

The /r/selfhosted announcement [0] had some more information about it, including links to issues where the fork-plans were discussed. This is a great reminder for me to migrate ;) [0]: https://old.reddit.com/r/selfhosted/comments/tbcuf0/announcing_first_release_of_paperlessngx_the/ https://old.reddit.com/r/selfhosted/comments/tbcuf0/announci...

Semaphor 5y ago

For those who don't know, paperless is a Document Management System. More information in the docs [0]. It’s the 2nd fork, with -NG being the first one. They each rose up when the former version became unmaintained. [0]: https://paperless-ngx.readthedocs.io/en/latest/ https://paperless-ngx.readthedocs.io/en/latest/

phkx 5y ago

Interesting - will check it out to see whether it is better than my current approach, in which I scan to PDF-A including OCR and let Spotlight do the indexing. Some additions: I found that black-and-white scans at 300 dpi work for almost all documents, resulting in a small file size and decent readability. Occasionally I switch to gray and 200 dpi and rarely to color. After looking up how long the originals of different types of documents need to be kept for legal reasons, I settled simply for the maximum time (6 years in my case) and file documents in a binder, sorted by the year in which I can discard them. Then, at the beginning of each year, I can get rid off one section of the documents which are older than 6 years. There is a second binder for active contracts (insurance etc). As soon as one of them ends, it goes into the first binder. I‘ve started organizing my Downloads folder in a similar way - sorting stuff by when I think I can delete it (either because it‘s not relevant any more or because I simply never touched it), typically a few months in the future. Both systems have helped to keep the clutter low and.

wfriesen 5y ago

I just migrated my instance by repointing the webserver container image at the one in this repo. So far, so good.

throwaway290 5y ago

> I wrote this to make “going paperless” easier. I do not have to worry about finding stuff again. I feed documents right from the post box into the scanner and then shred them. Perhaps you might find it useful too. I know this strongly depends per country, but could anyone else share real-life experiences of going paperless? From my understanding, if there is a sheet of paper that has special meaning, presumably you must be able to produce the original, with wet stamps and signatures, upon request (since any copy can be created from scratch digitally). So I keep loads of papers I am afraid to dispose of.

phkx 5y ago
This is my understanding as an individual in Germany who is not self-employed: Original documents help you to make or defend against legal claims, so for the time being, I'd keep them during any limitation period which applies. Afterwards that paper is not really useful any more (but keeping a digital copy doesn't hurt). I suggest to find out, which periods apply in your jurisdiction. In my case, I simply settled for the maximum of all the different periods I encountered and file originals by the year after which I can trash them and use the digitized versions for actually working with them (e.g. my tax declaration is now much faster to do). For Germany, I found that 6 years grace period should be fine. 10 years, if you're self-employed.
L3viathan 5y ago
I use paperless-ng to go mostly paperless, but I keep the paper: I bought a simple self-incrementing stamp, which I stamp all incoming documents with. When adding the document to paperless-ng, I set the document's number accordingly, and finally I file the paper away purely sorted by ID (so I have a (physical) folder titled 1-101, one 102-231, etc.). When I need the original to a document, the lookup is very fast, and when I know I won't need the original ever again, I don't stamp it and tag it as "digital only" in paperless-ng.
- caeruleus 5y ago
  Although I have not used paperless-ng (extensively) so far, I have been using almost the same method for many years and could not be happier. When I receive a document, I do the same every time: 1) Stamp it with the pagination stamp. 2) Scan it. 3) Dump the scan into a single large digital folder. 4) Dump the original into a physical folder (titled as you do). It relieves the mental burden of categorising since you do the same every time – stamp, scan, dump. And repeat. In the very rare case you need to find the original it's also superior, thanks to OCR and the sequential numbering being represented on the scan.

brunoqc 5y ago

I wish there was a standalone version, so my mom could use it. Or maybe a desktop + a mobile version that are synced without a server, so you could still be able to use your phone to scan new documents.

uniqueuid 5y ago
There is DevonThink [1]. Has been around for decades, does OCR, syncs to a huge bunch of services including WebDav, has mobile clients. It's a bit of an old paradigm, though - powerful, client-side, offers a lot of freedom and therefore wants you to put in manual work for setting up a system. I love it. [1] https://www.devontechnologies.com/apps/devonthink https://www.devontechnologies.com/apps/devonthink
- vertis 5y ago
  I've been moving more and more to apps like Devonthink. I use Obsidian and Zotero, which while they different use cases are a similar 'old paradigm'. My thinking recently has been on the storing items in a way that will last. It seems every year another SaaS gets bought and/or shutdown. Text based storage (Obsidian/Markdown) and desktop/foss apps are one way to combat that.
  - solarkraft 5y ago
    If "old paradigm" means self-owned data and no dependence on a subscription that can go away at any time I'm all in on it. (Small) SaaS is not dependable.
- LeSaucy 5y ago
  DEVONThink is criminally underrated. The mobile apps are fantastic. It's almost a super power being able to pull up any document/receipt I've dropped in my scanner over the last decade on my phone from anywhere.
anthropodie 5y ago
Host it on Raspberry PI at home. That's what I did.

moveax 5y ago

There is a linuxserver.io container image now, which should act as drop in replacement for the paperless-ng linuxserver image. Haven't tried it yet, but it looked good so far https://hub.docker.com/r/linuxserver/paperless-ngx https://hub.docker.com/r/linuxserver/paperless-ngx

anaisbetts 5y ago

As a user of Paperless I highly recommend setting it up, it is so incredibly useful as soon as you have to deal with any kind of paperwork-intensive activity (buying a house, getting a mortgage, paying taxes). Take literally everything you get and throw it into Paperless, then when someone asks for all of these documents it will take you minutes instead of hours to track them all down - when you need it, it's magic.

einpoklum 5y ago
Can you explain how it is better than carefully scanning and arrangement the scanned files yourself? (Not being facetious, I'm really asking.)
- vertis 5y ago
  Not the parent, but I suspect it's not better in the outcome, just in the amount of effort you have to go into creating and arranging vs just throwing documents at it and searching later.
- anthropodie 5y ago
  You can categorise those documents, tag them, OCR them, search them. Access them on phone or any other device with browser.
  - nojito 5y ago
    How is that any different from Onedrive/iCloud/etc?
  - h0l0cube 5y ago
    I'm looking at this ecosystem and wish I'd seen it sooner. I've been using the Google Drive Android app to scan documents. It does auto-cropping and perspective correction, but it doesn't OCR, and I manually rename the files so that I can order them by date and cross reference with my banking transactions. With OCR I might at least be able to extract some useful data and avoid some tedious data entry.
  - 5y ago

einpoklum 5y ago

IMHO, this seems to consist of two very separate pieces of software: Scanning, and complex management of an archive of files and documents. Both are interesting challenges, but focusing on the second one - I don't see why it should be tied-in with scanning. It's actually something that I feel is missing in operating systems, or at least desktop environments: The ability to arrange your files in more than one way, rather than having to force them into a static hierarchy ... while also not having them lost behind some piece of software which prevents direct access.

mxuribe 5y ago
> ...The ability to arrange your files in more than one way, rather than having to force them into a static hierarchy ... while also not having them lost behind some piece of software which prevents direct access... I wish all common operating systems supported labels/tagging in a compatible, easy-to-migrate manner...so that a person could migrate from windows to linux or back again, and all their file meta data - like file labels, file tags! - was preserved.

kloudleinc 5y ago

It's a very useful tool. I mainly use it to back up my documents instead of only keeping the paperless ones though.

gsich 5y ago

Automatic deskewing of images?

m3nu 5y ago

This was one of the first apps I added to run on PikaPods. So free to use during beta and I hope to set up a revenue sharing agreement with the project soon: https://www.pikapods.com/apps https://www.pikapods.com/apps

gurjeet 5y ago
PikaPods looks like an interesting idea. I invested some time setting up account, and trying to create Baserow pod. I was greeted with "Failure while adding container. Our team has been informed."
- m3nu 5y ago
  Yeah, still in public beta and multi-container apps like Baserow are pretty new and marked as "experimental". Already looking into the error. Edit: Should work, but takes several minutes before the web UI becomes available. We do provide the logs for such cases. Most other apps are easier to deal with. :-)
  - nigggle 5y ago
    Baserow dev who worked on our new multi-container single image (baserow/baserow on dockerhub) here. I assume you are using the baserow/baserow image for pikapods (but perhaps not?). We have a few environment variables (https://baserow.io/docs/installation%2Fconfiguration https://baserow.io/docs/installation%2Fconfiguration) that can be tweaked for this image to make it start-up faster and with a lower resource footprint: - `SYNC_TEMPLATES_ON_STARTUP=false` to turn off the initial load of example Baserow templates. Our template collection is growing and the loading of this into the database is probably what is causing the several minute startup. This will be optimized in the future. - `BASEROW_RUN_MINIMAL=true`. This will combine two of the backend async queue processes into a single one. This might result into higher priority async tasks getting stuck behind slower tasks (an large import/export might slow down the broadcast of realtime events for example). But this tradeoff is perfectly reasonable in most small self hosted environments. - `BASEROW_AMOUNT_OF_GUNICORN_WORKERS=1` to reduce the number of concurrent backend api processes from the default of 3. Once again this might cause a degradation in performance for higher volumes of traffic, but a tradeoff worth considering if memory etc is a concern. This image is brand new and because of it's multi-process nature is hiding some interesting complexity. We are still working out the kinks and a sensible set of defaults. It's also very easy for us to offer different variants of this image with different defaults set. Any feedback, suggestions or questions are very welcome here or at http://community.baserow.io http://community.baserow.io.

upofadown 5y ago

Just to save others from having to look it up, paperless-xxx uses OCRmyPDF to do the work. So you don't have to host anything if you are willing to touch a file or two...

subeadia 5y ago
What do you mean by hosting? Both paperless-NGX and OCRmyPDF can be installed completely locally. Why not just install paperless which takes care of the OCR integration for you?

frameset 5y ago

I use Papermerge, which seems to be a very similar software. Does anyone have experience with both and prefer Paperless?

evolve2k 5y ago

At one point I was using Evernote combined with a Fujitsu Scanner to scan in all my paperwork and Evernote would automatically OCR the scans and make it easy for me to search for this later. Evernote is a commercial offering. Does this provide similar or better yet, better functionality? If I scan my days mail in one go.. say water bills, property expenses etc in one batch.. will this then sort the water bill from the telephone bill in what is saved? Ps. Bills contrived for illustration I receive much more of this online these days but still paperwork abounds.

tenebrisalietum 5y ago
Yes. It uses tesseract for OCR and gathers the OCR text and puts it in a database, and you can then search. So if you enter something you know is on the water bill then it'll come up in the search, such as the name of the water company. I've never used Evernote so I'm not sure it's better but I had no problem finding stuff so far. Also you can tag documents and search by tag so you have that too. Paperless kicks ass.

bravura 5y ago

This is cool, but can I pay for a managed service? I clicked through the documentation and I don't understand how easy/hard it is to make cloud backups and do disaster recovery.

nbenitezl 5y ago

This is also an interesting one: https://openpaper.work/en/ https://openpaper.work/en/