4 ms·
Mine all feel like "we do these not because they're easy, but because we thought they would be easy". Turning a thousand-page book - PAIP - into a stack of Mar
by pronoiac 2y ago
Mine all feel like "we do these not because they're easy, but because we thought they would be easy".
Turning a thousand-page book - PAIP - into a stack of Markdown files in a git repo, readable online. The print book received more editing and revisions than the ebook. I converted an ebook's ... odd formatting ... into Markdown, remade diagrams, generated new ePub and pdf files, and had the spine cut off a print copy to make a fresh scan. Working on that scan, I made Scantailor, an X program, easier to access from a Mac, via Docker. I tried different OCR engines, and pored over the diffs, incorporating dozens (hundreds?) of improvements. I got to find so many differences between Markdown engines. I have ideas on how to make Pandoc links between chapters. There's still a lot to do!
My current WIP: Lars Wirzenius posted about file systems with a billion (empty) files. I started exploring because I was curious, if I was remembering correctly, how well a mostly empty image file would recompress - like, drive_image.gz.gz. Lars offered a Rust program; I was curious about how other methods compared. Like, how about nested shell loops, tar, and touch? And, hey, how well can we archive and compress them? I've gotten to see some issues, bottlenecks, and outright failure modes with SMR hard drives, Samba re: sparse files, and parallel gzip compression. I've accumulated some shell script boilerplate to make it easier to go back and verify my processes, and harder to accidentally wipe out past work if I rerun it.