5 ms·

Backups are one of those things that I don't do well at the moment. Fortunately, Mayan doesn't have any special requirements - basically back up the database a

Backups are one of those things that I don't do well at the moment. Fortunately, Mayan doesn't have any special requirements - basically back up the database and back up its media directory[0]. My ideal solution is to have everything backed up "in the cloud" plus have a backup at home. So far I've got the home backup happening - every machine uses 'attic' to back up onto my NAS. I'm still figuring out what the "in the cloud" side looks like. One thing you can do is keep copies of important documents elsewhere - like Dropbox, a USB drive in a safe deposit box, etc. It all depends on what technologies you trust and how much you trust them. For OCR, I can't say I've had much trouble with it. Mayan breaks the PDFs into pages, then feeds each page to tesseract. All of that happens in a worker process and I haven't found that it has a significant impact on the use of Mayan. The OCR section for the document will be incomplete for a while though. [0] http://mayan.readthedocs.io/en/v2.1.4/topics/backups.html http://mayan.readthedocs.io/en/v2.1.4/topics/backups.html

coaxial 9y ago
I don't trust dropbox, at all. I'd like to store encrypted backups online for the Mayan db and every document. For the OCR, my issue is that if I import several documents Mayan will feed them all to tesseract in parallel. Then I have two available cores, but 40 tesseract workers and the system comes down because it runs out of RAM and CPU. How did you set up your OCR?
- Mister_Snuggles 9y ago
  I just used the instructions here[0], with some minor changes (pathnames, db server, etc). I've never seen more than two tesseract processes running on my two-core VM, no matter how many documents I give it. Maybe something changed at some point to limit the active workers to the number of cores. [0] http://mayan.readthedocs.io/en/stable/topics/deploying.html http://mayan.readthedocs.io/en/stable/topics/deploying.html
  - siloraptor 9y ago
    Mayan EDMS issue to add rate limit support to the OCR backend. https://gitlab.com/mayan-edms/mayan-edms/issues/287 https://gitlab.com/mayan-edms/mayan-edms/issues/287
    - Mister_Snuggles 9y ago
      I wonder if there was a change in celery that limits the number of active workers to the number of cores. This is the behavior I see on my instance.
      - coaxial 9y ago
        That could explain it, it would be a nice addition

coaxial 9y ago

I don't trust dropbox, at all. I'd like to store encrypted backups online for the Mayan db and every document. For the OCR, my issue is that if I import several documents Mayan will feed them all to tesseract in parallel. Then I have two available cores, but 40 tesseract workers and the system comes down because it runs out of RAM and CPU. How did you set up your OCR?

Mister_Snuggles 9y ago
I just used the instructions here[0], with some minor changes (pathnames, db server, etc). I've never seen more than two tesseract processes running on my two-core VM, no matter how many documents I give it. Maybe something changed at some point to limit the active workers to the number of cores. [0] http://mayan.readthedocs.io/en/stable/topics/deploying.html http://mayan.readthedocs.io/en/stable/topics/deploying.html
- siloraptor 9y ago
  Mayan EDMS issue to add rate limit support to the OCR backend. https://gitlab.com/mayan-edms/mayan-edms/issues/287 https://gitlab.com/mayan-edms/mayan-edms/issues/287
  - Mister_Snuggles 9y ago
    I wonder if there was a change in celery that limits the number of active workers to the number of cores. This is the behavior I see on my instance.
    - coaxial 9y ago
      That could explain it, it would be a nice addition