3 ms·
Thank you for your advice, that's handy to know. What's the database format it uses, sqlite3? (I tried looking on the repo but couldn't any obvious reference)
by laumars 10y ago
Thank you for your advice, that's handy to know.
What's the database format it uses, sqlite3? (I tried looking on the repo but couldn't any obvious reference)
- joshstrange 10y agoI believe it is all flat-file. When I say "Database" I am just referencing what they call it: -d DB_DIR, --db-dir DB_DIR But from what I see the structure looks like this: db/ YYYY-MM/ 1234554543262346.eml.gz - I assume the meat and potatoes of the email along with attachments, not sure 1234554543262346.meta - JSON file with msg_id, thread_id, flags, labels, subject, etc
- leeoniya 10y agoif you want to try something that downloads Gmail via imap and indexes it into an sqlite3 db (with FTS5 fulltext of from/to/cc/bcc/subject/body fields) and extracts attachments to filesystem, take a look at a recent project of mine: https://github.com/good-3g-apps/email/tree/master/imap-importer https://github.com/good-3g-apps/email/tree/master/imap-impor... It also saves the raw .eml files to disk. No support for labels (yet), but it does properly link up threads in the db using `References` from the parsed headers (setting both MPTT and adjacency-list fields) It's WIP, so contributions welcome :)
- leeoniya 10y agoFWIW regarding speed, i was able to download, index & extract my entire INBOX + Sent Items (14k emails, 3.5GB total) in < 10min on a fast connection. the limiting factor by far was connection/imap speed.
- leeoniya 10y agomoved to https://github.com/Good-3G/email https://github.com/Good-3G/email