13 ms·

Portable EPUBs

659 points by sohkamyung 3y ago

wolverine876 3y ago
It's a very well thought through article by the developer of Nota, trying to bring EPUB format up to parity with PDF. It's a serious start and they've already written a viewer. In fact, the article itself is displayed in a browser-based wasm port of the viewer (and looks good!). One issue is how precisely EPUB, which is really XHTML, can reproduce layout. What are the possibilities here? The OP's standard is that the document will look "reasonable". The imply that HTML would need new layout capabilities to match PDF, at least for line breaking: There's two ways to make progress here. One is for browsers to provide more typography tools. Allegedly, text-wrap: pretty is supposed to help, but in my brief testing it doesn't seem to improve line-break quality. The other way is to pre-calculate line breaks, which would only work for fixed-layout renditions. Also, though the author mentions annotations, I don't see how they intend to implement them.
- joshjob42 3y ago
  The author discusses fixed layout epubs. Effectively, the epub can give a default pagination, line-breaks, font, font size, page size, and positioning for images etc., making it render identically on everything (one might optionally omit pagination if opening in a browser but keep everything else). This can be done already in epub3. But that's not ideal, because then it doesn't look good anymore on a phone, etc. Depending on the reader though, you could override the default, but then you have to hope that your reader does a good job of making a nice document. An alternative is for the epub to specify multiple renderings, for various common screen types. I don't think this is unreasonable as a solution. By all means let's try to get a smart reader, but letting people create defaults for their documents that can be overridden if desired by the user is a good middle ground.
  - idoubtit 3y ago
    Indeed, EPUB3 provides all the features that the author wishes. His "portable EPUB" format is just a loosely specified subset. It's unclear if some extra features are included in the format as they are in his "Bene" tool, like the rendering of references (i.e. links with a data-target attribute). The EPUB3 standard is much more complex than EPUB2 (media overlays, mixing fixed layout with reflowed, MathML…). In my experience the implementations are much more varying, and most of them aren't complete. So a "Portable EPUB" may not render as expected because the reader tool lacks some specific feature. The author also requires full JS support, which I supose does not help with portability.
- zozbot234 3y ago
  Doesn't CSS support layout capabilities for paged media out of the box? An EPUB reader just has to implement good old-fashioned "Print Preview" display mode, and you're set.
- zdunn 3y ago
  > Also, though the author mentions annotations, I don't see how they intend to implement them. It's discussed at the very end of section 8 and all of section 9 that interactive functionality would use web components.
- nine_k 3y ago
  PDF does not have any capabilities of line breaking. It is a picture format, similar to SVG, only more rigid. That's why it can't have text reflow, etc. What an ebook format needs is a semantic form of markup, which adapts to devices it is rendered on. HTML + CSS were invented for this goal. With that, book layout authors should consciously relinquish some control on how the book looks, and hand it to the reader. Slight visual imperfections are a small price to pay for this. Who needs visual perfection should go for a PDF. This, of course, becomes hard if any interactive stuff is involved. I would suggest that larger interactive elements should open in a dedicated view when needed, and tiny interactive elements should embrace reflow.
  - kps 3y ago
    > HTML + CSS were invented for this goal. HTML (with SVG and MathML) is probably fine for most books, but CSS has spent 30 years resolutely resisting basic typography, i.e. default text baseline alignment.
  - wolverine876 3y ago
    You will enjoy the article, which goes into these issues in some detail.
- BlueTemplar 3y ago
  Ironically, the very example the author uses for annotations doesn't work properly for me : on touchscreen Android Firefox I get a link instead of a popup when press-holding. And aren't annotations (and references) already part of the EPUB specification, and probably even the HTML specification ?!? Finally, I disagree with the press-and-hold for popup being better than the usual practice of hyperlink anchors, IMHO their jumping around is much less disruptive. (As long as the reader's "return" function is working properly, and/or - for the bijective ones - they provide a "back" hyperlink.)
  - wolverine876 3y ago
    > And aren't annotations (and references) already part of the EPUB specification I'm pretty sure they are not, based on looking carefully a year or two ago, on recent discussions here on HN, and on the OP's belief that they need to invent it.
- int_19h 3y ago
  Back in the day, there was something called XSL Formatting Objects (XSL-FO). Meant to be something like XML-based PDF, but somewhat higher level - e.g. it did text reflow. The idea was that you'd distribute an XML document combined with an XSL stylesheet that would transform it to FO for rendering; and the same document could have multiple stylesheets for different target devices, page sizes etc.
  - wolverine876 3y ago
    I suppose EPUB could have multiple style sheets and provide defaults for various uses. Professional publishers could do something fancy and beautiful, companies could have their own defaults, individuals could use the EPUB defaults, and there could be FOSS stylesheets that people download and use as desired.
xnx 3y ago
8 days ago, 134 points: "Portable Web Documents – An Alternative to PDF Based on HTML5 (2019)" https://news.ycombinator.com/item?id=39036774 https://news.ycombinator.com/item?id=39036774
- crabmusket 3y ago
  And this current post is exactly what I was wishing for in my reply[1]. Really glad this was posted! [1] https://news.ycombinator.com/item?id=39037135 https://news.ycombinator.com/item?id=39037135
mr_mitm 3y ago
I'm fully on board with the author's "I want to replace PDF" sentiment. It's true that running code in the document has some downsides, but the vast majority of people does it all the time in their browsers. And it comes with tremendous upsides. Just imagine large amount of data presented in interactive tables which can sort, filter and export or interactive graphs inside the document. We already use HTML+JS so much, why should we stop at documents? Yes, they can't be printed, but in my observation less and less people even own a printer these days, and I see no reason why this trend should not continue. I bet the future will be mostly living, interactive documents. It's funny that I just mentioned this in the other thread [1], but I also felt that there is a need for a format that is self-contained and widely supported by standard software (by which I mean browsers). A well-specified open format would be great, but until then I tackled the self-containedness problem with JS and wrote a Python script that zips and bundles all assets and embeds them as a SPA into one HTML file [2]. The focus is on Sphinx docs but it should work in general with all distributed HTML docs. [1] https://news.ycombinator.com/item?id=39138444 https://news.ycombinator.com/item?id=39138444 [2] https://github.com/AdrianVollmer/Zundler https://github.com/AdrianVollmer/Zundler
- fodkodrasz 3y ago
  > It's true that running code in the document has some downsides, but the vast majority of people does it all the time in their browsers. This probably has to do something with them having nothing to do, as the big companies managed to convince the frontend dev community that the single best thing to generate layout is on the client machine on the fly. Of course they did it so the users will have a hard time selectively blocking the layout scripts from the ad/spyware most contemporary (web)software development is about. This led us to the point where saving (or God forbid printing!) an article needs a lot of effort in many cases. My observation is: when I need to go to work on the field, I need printed documents. Printed documents don't need firmware updates, their batteries don't run out, and no, I don't need interactivity in documents. Self contained HTML is a good - and necessary - step, but interactivity and executable code is not something we usually need in documents, I only saw somewhat legit need for it on corporate abomination of documents (and some teaching materials possibly).
simonw 3y ago
Here's some really insightful feedback on this idea from Baldur Bjarnason, who has spent significant time working with various W3C groups relevant to EPUB: https://toot.cafe/@baldur/111819472053623911 https://toot.cafe/@baldur/111819472053623911 Example note: "EPUB originally didn't support remote resources and people put a lot of work into changing. Loading stuff over the network is HTML's killer feature. Blocking network assets is a setback for format adoption, not progress."
- zaphirplane 3y ago
  Allow loading of network resources is not good for security. Surprised this isn’t a worry, tbh didn’t read baldur’s writing
  - criddell 3y ago
    But it’s good for tracking what books people are reading. If the history of the internet shows anything, it’s that if surveillance is possible, it will eventually happen. Lots of governments around the world would love to know what their citizens are reading. Few would be bold enough to go after this directly, but if some company operating in their country has the data then there’s a path for that government to get the data.
- starkparker 3y ago
  Oh for Christ's sake, someone pry Baldur off his cross again. > Blocking network assets is a setback for format adoption People are standing here telling him that _allowing_ network assets _is a setback for format adoption_ and he's just going to keep pounding this stupid, obnoxious drum of his until he runs everyone off. > almost all of the problems described would be solved by getting OS vendors (Google, MS, Apple) to invest more money in EPUB That's back-asswards. Google, MS, and Apple don't give a shit about EPUB, they never will, and it's arguable that we're better served with them not buying a seat at that table considering how poorly their "help" has helped web standards, as much of the rest of his dismissive thread helpfully notes. If he wants money for EPUB standards he should shake the cup at IDPF members who rely on it, and particularly Amazon, to whom he quite vocally abdicated the publishing space 12 years ago.[1] Barking at operating system companies is nonsense at best and how we wind up with another, even more avoidable situation where the space is held hostage by them at worst. At least Amazon can chuck some goodwill money at EPUB development while continuing to kick its ass up and down the market with MOBI. (Aside from all this, his dismissals of the "clunky" reading system complaint, citing how EPUB has "too many divergences", only further proves to me how tunneled the vision is of the people involved. To hell with forking or improving EPUB, then, because it can't be improved if that's the attitude of the people most involved with or influential within it. What bloody point is there in the customizability of a format that _nobody_ can effectively build tools for or consume?) 1: https://www.baldurbjarnason.com/notes/amazon-wins/ https://www.baldurbjarnason.com/notes/amazon-wins/, in which he also admits that he has no idea how to work with IDPF, which is a really great sign of how long things have been going this badly in this space
reacharavindh 3y ago
The thing I wish the most with epub or technically the epub readers is the ability to scribble and hand write notes in them using a stylus and for them to keep them while reading again. I do that with PDFs on my iPad, but have a lot of tech books for which I took manual notes nowhere to be found again - even if I did, they are not inline with what I was reading and thinking.
- eviks 3y ago
  In general, the modern docs should be easily editable, not just allow annotations, since it's easy to preserve the original content/layout
- beckerdo 3y ago
  I agree, I would like an ePub to have a robust note taking and exporting ability. For instance, if I highlight in Chapter 8 "In 539" [next paragraph] "Belisarius" [next paragraph] "marched on Ravenna" [10 paragraphs later] "In 540 Belisarius entered Ravenna". I would like to export this with the Chapter header and detailed highlight locations OR just as one sentence with subtle links to the locations.
- AlanYx 3y ago
  I'm on the same page. I convert all my ePubs to PDF because I want to keep my handwritten annotations in-place alongside the text I'm annotating, including things like circled words. Recent Kobos (Elipsa and Sage) take a decent stab at solving this problem while retaining the ePub format and reformattability, but it breaks too easily.
- sriacha 3y ago
  Same, its a big wish of mine, just to have simple portable highlights in epub. There's some interesting news in this space: next version of Zotero will have epub annotation (WADM) support: https://forums.zotero.org/discussion/110487/embedding-annotations-in-epub-file-itself https://forums.zotero.org/discussion/110487/embedding-annota....
eviks 3y ago
Commendable effort of trying to get rid of the ancient paper-based legacy in the digital world that is PDF Though I'm curious whether the clunky old-but-still-living HTML (especially in its ugly XML variety) + CSS are the right foundations for the portable format of the future? Since the author has also developed the whole new document language would be nice to read a more in-depth overview on that subject. Or why limit to the ugly duckling of JS in the future when WASM exists? > content by pages. Instead, we should mandate a consistent numbering scheme for block elements within a document, and have people cite using that scheme. that's indeed the proper and more precise approach, though we could still have those "fixed layout epub" pages as a backup coordinate system
thayne 3y ago
> A PDF is a single file that contains all the images, fonts, and other data needed to render it. A PDF can include the fonts. But it often doesn't, and relies on system fonts. One reason for that is because including fonts in the PDF can dramatically increase the size of the file. In some cases a single font could be larger than the entire rest of the file. I've also worked on implementing embedding fonts in some software that generated PDFs. It was surprisingly difficult to figure out how to get it to work reliably. > PDFs are rendered consistently. Not as much as you would think. There are several cases where the same PDF will render differently depending on which PDF viewer you use. Usually the differences are pretty subtle, but occasionally there are edge cases that result in pretty significant differences. I've even run into a case where the same version of Acrobat reader will render a PDF differently depending on what OS you are using.
- EE84M3i 3y ago
  Is there software that minimizes the fonts by removing code points that aren't used in the document?
  - adrian_b 3y ago
    This is a standard feature of the PDF format. Normally all PDF documents include only the glyphs corresponding to the code points actually used in the text rendered with that font. That is why you can go for instance to any site of a vendor of fonts and you can download freely a PDF sample text of an expensive font. You can easily extract the font from the sample PDF, but it will be useless, as it will contain only the few letters that had been used in the sample text.
- geraldhh 3y ago
  > A PDF can include the fonts. But it often doesn't, and relies on system fonts. found this out, after 20-something years of consistent pdf renderings, in a job interview because my docs allegedly looked odd :/ the daily wtf ...
  - jxdxbx 3y ago
teekert 3y ago
As someone who occasionally tries to read scientific literature on their e-reader, which is nice, I can just mail it to my PocketBook account and it shows up, I have a deep hate for PDF. Please let this be a popular thing.
mrich 3y ago
Ironically this did not render in Firefox on Android (just the spinner kept spinning) Worked in Chrome. That said, epubs are great for reading books on mobile. The advantage for pdfs is that they contain highlights/notes, so you can directly import them into Zotero and all your annotations are there. For epub, you have to hope there is a way to export the annotations that are stored by the reader app, and then you have to process them further. Readera is a great reader for mobile that makes this possible. I'm currently working on a script that will convert an epub to pdf, extract the annotations from Readera, and mark them in the pdf. Then I can import the pdf into Zotero, while still retaining the great reading experience of epubs.
- Symbiote 3y ago
  Works fine in Firefox for Android 122.0 for me.
  - mrich 3y ago
    Also loads instantly for me now, didn't make any changes.
- zozbot234 3y ago
  There is a Web Annotation standard that could be used to export the notes to.
- staz 3y ago
  It is working for me on my Firefox on Android. One of the nice benefits I can already experience in his document it the working TOC sidebar which allow navigation in the document. (Compared to classical HTML not PDF)
- mwilliamson 3y ago
  I had a similar problem loading the page on Firefox for desktop with private browsing. It turns out service workers don't work in private browsing, which it seems Bene (the software rendering the page) requires. Switching to a normal Firefox window solved the problem.
DeathArrow 3y ago
I didn't know that EPUB is based on HTML. I always had the impression that it has its own binary format. Using HTML as a base has a lot of sense.
- simongray 3y ago
  W3C standards basically always build on top of other existing W3C standards.
- anthk 3y ago
  It's just a zip file. Under Linux/Mac/BSD you can trivially write a script which unzip's and outputs the ebook's HTML files into a large text stream and that output can be used as the input of a text mode web browser, allowing you to read ebooks everywhere with just two lines of code.
- arp242 3y ago
  It's just a zip file with HTML documents and some (ePub-specific) XML files to define metadata, chapters, and a few things like that. I use this "epub-edit" script to edit them: #!/bin/zsh # # Extract epub file to a temp directory, launch shell to edit it, and re-zip # it. Nothing about this is really epub-specific as such. echo " $@" | grep -q -- ' -h' && { sed '1,2d; /^[^#]/q; s/^# \?//;' "$0" | sed '$d'; exit 0; } # Show docs [ "${ZSH_VERSION:-}" = "" ] && echo >&2 "Only works with zsh" && exit 1 setopt err_exit no_unset no_clobber pipefail full=$1:a tmp=$(mktemp -d) bsdtar xf $1 -C $tmp cd $tmp print "Editing $1; press ^D to exit" zsh ||: mv -f $full $full.orig zip -f $full * cd - rm -r $tmp And then I use vim to edit the HTML files and such.
emayljames 3y ago
The download of the page epub dispays out of the viewport on google books app. Bene seems to be in alpha stage.
diebeforei485 3y ago
I personally think PDF's are a terrible legacy format with unnecessary complexities[1] and most uses of PDF's do not involve printing so the typesetting arguments don't make sense to me. For the vast majority of use cases it's far more important to be readable on phone, tablet, and computer. I was surprised when the author mentioned iBooks doesn't support scrolling view, so I tried it myself. Turns out iBooks on macOS does not support scrolling for ePub files, but it does on iOS and iPadOS. Very strange decision by Apple. 1. https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-into-nso-zero-click.html https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-i...
- baq 3y ago
  but but but... if I really need to print something, a PDF is the most reliable times portable route. I guess a multipage svg would work, too, maybe, if exported to a pdf to properly print multiple pages first. (Looking at you, inkscape...)
- adrian_b 3y ago
  PDF is an annoying specification, but there exists absolutely no replacement for it. I have never seen any kind of technical documentation published in any other format than PDF that is comfortable for reading and searching, even when that is done on a mobile phone. I do not want a document that changes appearance depending on the device used for reading or depending on its temporary state, like window size. I want a document whose layout has been well conceived by its author and which is fixed, regardless of what I happen to use for reading it. When I happen to read it on a smaller screen or window, except for trivial text-only documents, I do not want changes in layout, but I only want a smart reader, with comfortable means for fast zoom and pan, and which does not have stupid behaviors (like some Android readers), for instance where scrolling vertically (including Page Up/Page Down) also moves the document horizontally (preventing the easy reading of a column of text). The traditional recommendations for the maximum width of a text column are good enough, if observed, to ensure comfortable reading even on a mobile phone. Only when the author breaks the traditional typographic rules by making extra-wide columns, the reading on a mobile phone becomes inconvenient.
  - 3y ago
czierleyn 3y ago
When the IDPF merged with the W3C a couple of years back they tried to develop a new standard called PWP, Portable Web Publications, which was supposed to be a new 4.0 version of EPUB, as far as i know. But there was much resistance from the publishing community and the project was shelved a couple of years ago. See: https://w3c.github.io/dpub-pwp/publishing-snapshots/FPWD/Overview.html https://w3c.github.io/dpub-pwp/publishing-snapshots/FPWD/Ove...
watwut 3y ago
> PDFs are rendered consistently. A PDF specifies precisely how it should be rendered, so a PDF author can be confident that a reader will see the same document under any conditions. And that is why PDF sux for reading on the phone. And why epub is massively better if you want to read articles and books.
geokon 3y ago
It feels like a bit of doomed a project simply bc browsers don't open EPubs. You can link a PDF and while it's a bit of a context switch, the browser will open and display it Since as described EPubs are basically HTML its kinda dumb browsers don't open them - but good luck convincing the Chrome/Mozilla bureaucrats I think another discouraging aspect is HTML CSS are so huge and bloated at this point that few people can implement a "reader" for EPUB/HTML. It's basically "go implement a new browser". It makes one think a easy-to-parse markdown (like Djot) with some extra rendering bells and whistles would be a more likely long term solution My personal interim compromise solution is embedded everything (CSS, svgs, scripts and base64 images) into an HTML file. It's similar to an EPUB. It's a bit bloated and ugly but with a bit of care it works and naturally browsers (and by extension basically every user) can open it Unfortunately a user has no way to really know "oh I can download and store this web page offline". It'd be nice to have some thing like a .htmls extension that indicates it's an HTML but it doesn't have any external resources.
- zerof1l 3y ago
  > It feels like a bit of doomed a project simply bc browsers don't open EPubs. Not that long time ago, browsers could not open PDFs as well. Now all browsers come with PDF reader written in ASM/JS. I see nothing that prevents browsers doing the same for EPUBs. There exist browser extensions that do exactly this already. Its a matter of EPUB format gaining popularity.
  - geokon 3y ago
    I think Google is actively against offline data. It's not aligned with their business interests My mental analogy is, you can also have offline apps on Android. You can specify this in app manifest. But internet access isn't exposed to the user as a permissions. Like the author says, Google already injects online fonts into the EPubs they generate. Meanwhile PDF is a battle they've already lost
    - BlueTemplar 3y ago
      Indeed, and I would add that there's no reason for browser to be able to open PDFs : this sounds as yet another attempt for Google to wrestle with Microsoft over the control of the OS (by having everything happen in the browser instead). And also probably why we still have to rely on 3rd party hacky browser extensions to be able to save web pages as a single file.
zvmaz 3y ago
The author is a post-doc advised by Shriram Krishnamurthi [1], the author of Programming Languages: Application and Interpretation (PLAI), and one of the authors of Data-Centric Introduction to Computing (DCIC). I am currently reading both PLAI and DCIC and I am truly delighted by the minute care the authors have put into making the books pedagogical works of art. That's true love! [1] https://willcrichton.net/ https://willcrichton.net/
- verisimi 3y ago
  > the minute care the authors have put into making the books pedagogical works of art. That's true love! Works of art. True love! That's very high praise.
  - zvmaz 3y ago
    I mean it. Both books are free, and PLAI has an interactive tutorial on the language used in the book called SMoL [1] done by one of Shriram's students. The tutorial is not a passive one by any means; it forces you to think and highlights pitfalls students often fall into when reading the material. This whole ethos on readers learning is in stark contrast to books that feel like the authors show off how smart and clever and profound they are instead of caring about their readers comprehension. I include very highly praised books even on HN. [1] https://www.plai.org/#direct-links-to-the-tutor https://www.plai.org/#direct-links-to-the-tutor N.B. The author of this post, portable EPUBs, also works on language learning. A whole ethos... https://arxiv.org/abs/2401.01257 https://arxiv.org/abs/2401.01257
- sotix 3y ago
  And the author has worked on making Rust easier to learn[0]! [0] https://rust-book.cs.brown.edu/ https://rust-book.cs.brown.edu/
morelisp 3y ago
The bar for epubs is so fucking low I have a hard time believing this matters at all. Just last week I bought a book set in the late Middle Ages which managed to transcribe all “þ” as “p”. Until publishers care about that stuff, none of these high-falutin technical discussions change anything.
- offices 3y ago
  I don't see how this relates to the link.
  - morelisp 3y ago
    The file format doesn’t matter one bit when the reading and authoring tools are shit and the editors can’t/don’t fix anything. And papers will generally have a lot fewer resources to deal with this than major book publishers, who have been epub-focused for over a decade now and actually make money from it.
    - harshreality 3y ago
      Any decent publishing or html editing tools fully support utf-8 by now. It's not the tools. Publisher and editor laziness may be a reason to be cautious about epubs currently for niche or esoteric works, but that's not the same thing. > I bought a book set in the late Middle Ages which managed to transcribe all “þ” as “p”. Until publishers care... The book market these days makes it challenging to do high-quality editing up front for republishing niche books in a new format. Publishers try to cut corners, outsourcing epub conversions to people who don't care and don't know what they're doing, or they OCR it, have an in-house editor (who also doesn't have a personal affinity to the subject) give it a once-over (maybe), and release it.
      - BlueTemplar 3y ago
        As an aside : Unicode support was still an issue in TeX last I checked, because most of the LaTeX tools don't support it (well, having been made before it was expected). Now, there are some attempts to fix this situation by Xe(La)TeX and Lua(La)TeX, but since TeX seems to be so much tied to PDF these days, it should probably just be abandoned by most scientific publishing in favor of the likes of GNU TeXmacs (note : it's NOT TeX in GNU Emacs) and HTML with MathML.
sirsinsalot 3y ago
I agree with most of the content of this post. One of the key things for me would be a requirement that figures and diagrams which can be expressed as SVG should be. Images, limited to things that need grid-of-pixels representation like photographs, should be limited to that.
livrem 3y ago
I use the SinglePage add-on for Firefox (think it is available for Chrome as well?) to save the current page DOM to HTML as a self-contained file (inlined CSS and data:-URLs for all images) with no dependencies and all the scripts removed etc. It is not perfect, and I do not trust browsers to always remain backwards compatible, but I prefer it to save pages as PDF or as multiple files. Interestingly one of the few pages I ever saw it fail on was this article on portable EPUBs. Guess it has too much magic going on to make the formatting work. The saved page is perfectly readable, but the style is nothing at all like what the original page was for some reason. I like how fbreader on Android just displays all books exactly the same, and as configured in the app rather than using any of the styling from the EPUB file. I never noticed that it tried to apply CSS or run scripts included in files and I hope it never tries to do either of those things. Loading external dependencies sounds like an even worse idea and I did not think that was even allowed.
- actionfromafar 3y ago
  SingleFile, right? Edit: On that note, what's up with the Firefox Add-Ons? Currently, they are all setup so that to do something interesting, they need all the permissions. Which leads to a natural market being created, of bad actors go shopping for Add-Ons they can take over. Can something be done about this? For instance for this "SingleFile" addon. It needs to access the rendered document in the DOM to be able to introspect and save it all to a file. But why does it need access to everything? Can't it have just permissions: - "snapshot DOM once" - "write to a single file"
  - pbronez 3y ago
    Agree. There are many extensions that I want to use on arbitrary websites, but rarely. This could be handled by the browser locking the extension out from everything, until the moment I manually invoke it, at which point it’s allowed access to the page I’m currently looking at. Now, maybe that’s how it already works, but I have no confidence in it.
  - gildas 3y ago
    Author here, I wished SingleFile would use less permissions, but it's unforntunately not possible from a technical point of view. Anyway, if you run some code I've written but are suspicious, you have to trust me or review the code which is open-source.
znpy 3y ago
ePub aren't that great either. PDF might not be perfect, but it's likely the best we have and the best we'll have in a long time. ePub renders wildly different on my eReader (kobo), my linux laptop (various apps), my iPad and my iphone. And i'm not talking about screen sizes, i'm talking about various elements being rendered largely incorrectly (and there's a matrix of incorrectness across implementations. PDF documents on the other hands... they render just right, everywhere. I have to zoom and scroll, but I will never have to ask myself "will I be able to actually read this document?" when dealing with PDF. Oh and by the way... I still print stuff from time to time. Yeah way less than it was needed in the past, but it's still a necessity. Can you even print stuff (sections? pages? selections?) from an ePub? > Bene is designed to make opening and reading an EPUB feel fast and non-committal. The app is much quicker to open on my Macbook (<1sec) than other desktop apps. This is elitist at best. Claiming something "is fast" on top-class hardware is misleading at best (if not dishonest). Try and running that on low class hardware (stuff like chromebooks but also laptops from at least 7-8 years ago) and let's see if it's still "fast". I'm not convinced.
- broscillator 3y ago
  > PDF documents on the other hands... they render just right, everywhere. I have to zoom and scroll, To me it feels the other way around, if I have to zoom and scroll, they render just almost right. And that almost is extremely important for actual reading. If you're quickly skimming a PDF, sure. But to sit down and read for 30 minutes? One hour? Fuck zooming and scrolling. My kindle just displays a page, and I tap it and it goes to the next page. Can't really get a much better reading experience than that.
  - znpy 3y ago
    If I’m reading that long, I don’t need zooming and scrolling on my ipad. Epub on ereaders work well but only if you’re reading fiction. Most images and almost all tables and charts have been messed up by epub rendering anyway. And ereaders are black and white, so you’re losing information anyways.
    - 3y ago
crabmusket 3y ago
> There's two ways to make progress [on document aesthetics] here. One is for browsers to provide more typography tools. ... The other way is to pre-calculate line breaks, which would only work for fixed-layout renditions. The third way is to develop non-browser clients, like the author's own Bene. While it currently uses Tauri and therefore the system webview, there's no reason it should always do that, or that another client couldn't be developed with a focus on typography. Want your documents to look super nice with all the kerning and line breaking your heart desires? Get a proper reader app. Just want the content? Sure, open it in a browser or a basic reader.
BonoboIO 3y ago
Wow, I m blown away how fast this site is. It loads instant on my iPhone. Perfect text sizes … really nice.
- crabmusket 3y ago
  You didn't get a huge multi-second loading spinner?
  - BonoboIO 3y ago
    Nope. Normally i'm used to surfing the web even with adblockers to get slow loading times from bloated websites with megabytes of useless libs. This was refreshingly quick.
upofadown 3y ago
>You might dislike the idea that document authors can run arbitrary Javascript on your personal computer. How I feel about this depends a lot on how much I trust the people who created the document and/or the person who sent it to me. I would trust a website I specifically selected with a defined TLS web of trust more than I would trust a random spam email. When we think about the risks associated with the complex and inherently insecure format known as HTML we tend to assume the level of trust available on the web. If we package up a bunch of HTML in a standalone document then we lose that assumption.
BlueTemplar 3y ago
See also : "The decades long quagmire of encapsulated HTML" (2022) : https://www.russellbeattie.com/notes/posts/the-decades-long-html-bundle-quagmire.html https://www.russellbeattie.com/notes/posts/the-decades-long-... (Which still hasn't been posted as a "news" it seems... should I just submit it myself ??)
harshreality 3y ago
I think precisely dictating layout is the wrong objective, although some areas (legal documents, academic papers) are still obsessed with that. Arxiv recently started offering a subset of papers in html, which is a short step from epub. If quick high-resolution referencing (page x, yth paragraph, zth line) is necessary, I think the way to handle it is to reference a phrase on that line ("the cat ran"...), which the reader can search for. If the search interface is lacking, that's an epub reader failure, not a format failure. Or, if the search option is considered insufficient (it does require typing with a [virtual] keyboard), paragraphs can be numbered—as many ancient works or works in translation already are, because such works have many editions and can't be layout-exact copies of each other. If paragraph referencing is necessary, visibly styling the paragraphs with numbers helps dramatically. There's no reason it has to be exclusive to high-profile ancient or classical works, like Plato [1]. Classic poetry and plays, where referencing needs to be most exact and fast, already tend to give up any hope of everyone using the same edition, and simply avoid flowed text and then number the lines. [1] https://en.wikipedia.org/wiki/Stephanus_pagination https://en.wikipedia.org/wiki/Stephanus_pagination
- foofie 3y ago
  > Classic poetry and plays, where referencing needs to be most exact and fast (...) I believe that paragraph referencing is, by far and without any contest, used primarily by any text subject to reviews and revisions. This means technical reports and academic papers. All academic papers I subjected to review were forced to use templates that enforced paragraph numbering. Even though each version of those documents were only read by a dozen readers or so, all papers submitted to those journals had to use the template. This means hundreds of documents (see half a dozen revisions per paper submitted per each edition) had that hard requirement, and this took place for each edition of a single journal.
xoac 3y ago
I love this unreservedly. It's almost an embarrassment that something like this does not exist already. Big thank you to Will Crichton for putting all of this together and actually giving this idea a chance to take hold.
- foofie 3y ago
  > It's almost an embarrassment that something like this does not exist already. To be fair, Epub has been largely ignored and neglected by everyone in the world. Virtually no reader supports rendering math notation, and virtually no significant publisher on earth, including the likes of Arxiv, offers Epub downloads. Commercial publishers force DRM onto every format, which excludes Epub, and non-commercial either stick with PDF or don't care.
  - julielit 3y ago
    Sorry, but this is not true. Every significant publisher produces EPUB these days. Most reading apps support EPUB (including apps from Apple and Google), Readium and EDRLab offer open-source SDKs that ease the development of mobile, desktop and Web reading software with strong EPUB 3 support, including MathML. Readium LCP is a DRM for EPUB, especially for public libraries that need an e-lending end date. More, EPUB is much more accessible than PDF for blind people and other people with disabilities. PDF has no interest for ebooks (but for short documents, yes).
gmuslera 3y ago
Another standard? (https://xkcd.com/927/ https://xkcd.com/927/). It is not like the publishing world will switch to it overnight, a lot is tied to the devices they sell, so it might be not enough motivation. It is not simpler to convert epubs into epubs with all remote content embedded and relinked? Another single file book format that used to be popular many years ago was chm, that had its own security problems. Maybe adding the possibility of executing (js) code is not the best for something that should be mostly static, and css be used to enable some level of safe interactivity.
asimpletune 3y ago
This is something I deeply care about, as I'm also very interested in the intersection of ebooks, security, and a LowJS web. 1. absolutely we should have a single-file, portable ebook format, and since PDF doesn't reflow text then it's not that. 2. HTML + CSS in 2024 is capable of reproducing virtually any kind of printed medium, but it can also reflow text. 3. I don't personally think JS should be a requirement, but a lot of conversations break down at this point so let do my best to explain and please understand I'm not simply being religious about this. In my view, an ebook is a book that can change its size in a way that makes sense. Bearing this in mind, I believe that if a reader has JS turned off, this book should work as intended. In other words JS shouldn't be required for it to perform its basic function. However, if there is some necessity to add interactivity or to augment the book, then yeah, why not, use JS. That's what it's there for. However, from the perspective of being a book, if it doesn't work as a book without JS then it's a bug in my view. (for standard ereader capabilities, I think the browser should offer that, but the book itself shouldn't be shipped with an ereader) 4. I think it's a mistake to embed all the styling as that may violate some people's CSP's. It's safer to specify separate styling as resources relative to the HTML, and ebooks should forbid loading resources from a separate domain. In this way, ebooks would always work offline, but it has the added benefit of working online too and would automatically adhere to the strictest possible CSP. This achieves the same goal as offline, but in a safer and more universally compliant way. 5. Finally, just distribute it in a zip file. That's how ebooks already work right?
- gildas 3y ago
  > 5. Finally, just distribute it in a zip file. That's how ebooks already work right? What about self-extracting ZIP files like this page? https://gildas-lormeau.github.io/ https://gildas-lormeau.github.io/ (note that it includes the CSP to make it safe)
  - asimpletune 3y ago
    I don't think there's any CSP? https://observatory.mozilla.org/analyze/gildas-lormeau.github.io https://observatory.mozilla.org/analyze/gildas-lormeau.githu... In any case, ideally, any ebook solution should have all resources loaded as files relative to the current document, and nothing inline. Like this, the book would be compatible both offline and to be hosted online, without requiring any changes to the book. The one file thing is cool, but again it requires JS to show anything, so that's not really inline with what I was talking about. In my view, an HTML document should work like a book, and any JS is purely to augment and extend that book if necessary. Those situations are rare though, and most JS is just to give a reader like experience. I think if someone is going to go through the trouble to host a book, they probably don't mind unziping a file before putting it on their server. The one file thing I said earlier was more about sharing the book, similar to an app package. but once it's on someone's servers presumably it's ready to be read. Basically .webarchive
Finnucane 3y ago
I see a couple of roadblocks for this. One is that is suggestion of a restrictive subset of HTML for coding seems like a potential accessibility problem, which is to say, you'd have to make your documents less semantically rich. For instance, he seems to be suggesting using. It's already hard enough to get epubs to work right when reading systems lag behind what browsers can support, saying 'let's have less' is not going to make things easier if you have complex content. The problem is not that there is too much html or css, the problem is that reading systems don't support them properly. Also, most dedicated reading systems (Kindle, Kobo, etc) don't allow javascript, which means your components will not work. That might of course change, but I wouldn't hold my breath for it.
- velcrovan 3y ago
  > One is that is suggestion of a restrictive subset of HTML for coding seems like a potential accessibility problem, which is to say, you'd have to make your documents less semantically rich. "less semantically rich" than what? Web pages? Or less rich than PDFs, which is what he's actually proposing to replace?
  - Finnucane 3y ago
    Than the epub standard as it exists. I guess I don't understand what the advantage here is, really. If you want to make an epub file that works universally, you can do that, within the existing standard. If makers of reading systems and software would actually support the full standard, which they currrently don't. If you don't want to use pdfs, don't use pdfs. pdfs get used for a lot of things they aren't actually very good for.
    - velcrovan 3y ago
      > If you want to make an epub file that works universally, you can do that, within the existing standard. Yes, that’s what the author is proposing we do.
      - Finnucane 3y ago
ryukafalz 3y ago
I did a double-take when I reached this part: > Therefore I decided to build a lighter EPUB reading system, Bene. You're using it right now. This document is an EPUB — you can download it by clicking the button in the top-right corner. Because, reading this on a desktop browser, I didn't even notice until it was pointed out. It's more obvious on mobile because the header takes up more of the viewport, but it otherwise behaves pretty much like a normal web page. This is probably a good thing. For what it's worth, I didn't see (or at least didn't notice) a spinner when loading the doc for the first time like some other people in the comments reported. I did notice it on my phone, but it went by pretty quickly. I'm not sure if that's the WASM program loading and if it only happens the first time you load the page.
- edflsafoiewq 3y ago
  The main thing the spinner waits on is the .epub file to be downloaded. That file is 4.77 MB, which is appreciable for anyone without a fast connection. Most of that weight is images (99% after decompression). Unlike a normal webpage (or a PDF), rendering doesn't appear to start until all the assets, the whole ePub, has been downloaded. This segues into a point of difference I thought the article would mention, but didn't: performance. A PDF can be optimized so the pages are substantially independent of each other, which makes rendering pages progressive, random-access, and highly parallelizable.
  - AA-BA-94-2A-56 3y ago
    This makes me wonder if EPUB pagination is possible? I know nothing about it, but if the format is standard enough, it might be possible to hack together an endpoint that returns the requested page(s).
- sinuhe69 3y ago
  On my phone, I didn’t notice it at all until I read it. I just thought it is a normal webpage describing the project.
aabdulllah 3y ago
Hi, I was wondering if someone could explain what software solution I need to create and implement in my calculator for it to have a fast response time when calculating. How does this work, in other words what enables it to work so fast?
aabdulllah 3y ago
p.s. sorry I am new to engineering.
SamBam 3y ago
One thing I'm very interested in, as a grad student who has to consume a huge number of PDFs, is whether there are good tools for converting existing PDFs to portable EPUBs or HTML documents. If I use, for instance, CloudConvert [1], I generally get a document that gets flowing text roughly right, but still interrupts the text with page numbers and book titles (that were originally at the top of each page) and includes additional bizarre line breaks, etc. Every so often I wonder if this is an LLM problem ("please reformat the following text to...") but I think that one shouldn't reach for an LLM for these kinds of things. 1. https://cloudconvert.com/pdf-to-epub https://cloudconvert.com/pdf-to-epub
nmz 3y ago
> * PDFs cannot easily express interaction. PDFs were primarily designed as static documents that cannot react to user input beyond filling in forms. I am glad about this, I do not want to download a document and it require any input. a document should be a document, nothing more. If I'm getting a book to read (pdf), I expect a book, not a webapp.
sotix 3y ago
I really like this proposal. Just last night, I was reading an EPUB of The Hobbit and clicked on a footnote, which instructed me to refer to page 24. It turned out that it meant page 24 of the printed edition of the book, which was in the first chapter of the book. Page 24 in the EPUB was still part of the prologue. So I had no idea which page it was referencing. As it stands, my Kobo has an increased font size, so I notice that I can flip the page a few times and still be on page 4 before it finally turns to page 5, which I assume is referencing the pages of the written text. This is a nice compromise, but doesn't solve the issue with the hard coded footnote being misleading. I wonder if we could instead look at religious texts such as the Bible (e.g. John 3:16) and code editors (e.g. Ln 4, Col 12) for referencing locations in reflowable text. The same way you can jump to a footnote in a document should allow you to have an actionable reference to a specific location anywhere in the text. But I don't think the text should be stylized like how the Bible has the numbers (e.g. 16) scattered within the text itself. Those should probably be hidden within the text and leave the reading software to display the first line number of the page down at the bottom instead of the page number. That might look like "4" the same as it currently does, but this 4 references a section of the text rather than a page number. Perhaps it could be togglable for greater detail and display word 23 of section 4 as "4:23". Or maybe it could consider the chapter too. For example, chapter 2 section 4, word 23 would look like "2, 4:23". This might get funky in a Terry Pratchett novel, but it would hopefully allow for easier discussion of exact parts in a document and significantly easier linking. I love the interactive code example for marking up The Rust Programming Language. That gets at what I was saying above although is more targeted at a document than referencing parts of a novel. Kudos to author for creating Bene[0] as part of this proposal. That was cool to discover I was using their tool to read the proposal itself! [0] https://github.com/nota-lang/bene/ https://github.com/nota-lang/bene/
Shorel 3y ago
On the one hand, these are just EPUBs, nothing in the article makes the generated EPUBs different or incompatible from the ones I usually read. On the other hand: this looks awesome for the Web. Mix it with a blog platform like Medium, and it will improve my Web browsing experience tenfold.
1vuio0pswjnm7 3y ago
It appears this site (Github pages) is trying to require Javascript in order to read it. At the same time the top comment is suggesting EPUB should not require Javascript. I never use the HTML markup, CSS, etc. when I read an "ebook". I dump it to text and read it with less(1). I do not need to see images. I just want to search and read text. I use stupid simple shell scripts; something like the following. #!/bin/sh printf "(\n" for x in ${1-*.epub};do echo printf \'\\n\\n,,,,,,,,,,,,,,,,,,,,, "$x" ,,,,,,,,,,,,,,,,,,,,,,,\\n\\n\' 7z l $x|sed -n "s/.* /7z x -so $x /;/nav.xhtml/d;/[tx]ml$/p"|sort -n done echo ")|tr -cd '[\\\12\40-\176]'" Example usage: epub.sh 1.epub |sh > 1.htm links -dump 1.htm > 1.txt less 1.txt If I need to to adjust the order of the sections, I do something like epub.sh 1.epub > 1.sh vi 1.sh 1.sh > 1.htm links -dump 1.htm > 1.txt If I want to save the "ebook", I can compress the .txt smaller than any of the "ebook" formats. zstd -19 1.txt zstdless 1.txt.zst Plain text is portable for me; works on both BSD and Linux.
akoboldfrying 3y ago
What does an EPUB reader provide that a browser doesn't? Because right now it seems to me that the answer is just one, trivial thing: It can treat a .epub file (that is, a zip file) like a directory. If that really is the only thing that differentiates an EPUB reader from (a subset of) a browser, it seems that the simplest and least-effort thing to do would be: 1. Enable existing browsers to treat a zip file like a directory, under certain circumstances. 2. Describe a machine-checkable restriction of HTML that guarantees fully local assets as described in TFA (this might be as straightforward as requiring all URLs (except for links) to be relative). Declare any zip file that satisfies this restriction to be "a portable EPUB". 3. Forget about special-purpose EPUB readers altogether, both existing and new.
- LorenPechtel 3y ago
  Agreed, this sounds like the real solution. As for fully local assets--what I would like to see is a requirement that all objects have locally available fallback options. You can't get that font, use this standard one. Might not look perfect but it should work reasonably well if you limit yourself to things in the fallback. There is still one failure mode he didn't fully address--images that don't work right. Fundamentally, this is because images can't reflow. To a large degree I think this could be addressed with using SVG, but that's not going to handle things which are truly raster rather than vector. Unfortunately, I think the only way to address that is provide a few resolutions. (And I have faced a related problem that I haven't seen a good answer for but I haven't looked hard: SVG that is aware of it's resolution and can bring in more detailed images if it has the resolution to do so. Try to draw sub-pixel lines and you're going to get a blur, but blowing up the same image and those lines become a good thing. Windows acknowledges this with icons, you can include a variety of resolutions and it will work with whatever is most appropriate for the context.)
  - akoboldfrying 3y ago
    >a requirement that all objects have locally available fallback options If we instead mandate that every font used is either packaged within the zip, or one of a small, fixed, well known list of "standard" fonts that every system can be required to support, what would be lost? (IOW, what do we gain by allowing the concept of "fallbacks"? Because what we'd lose would be consistency between internetful and internetless renderings, and self-containedness.) >images that don't work right What's an example where storing a single high-resolution image, which can be dynamically scaled down if needed, doesn't work well?
watersb 3y ago
I care very much about ebooks and readability. One stunning failure of current ePub technology is the lack of publication-quality justification and microtypography. Currently, PDF authors are overloading "Tagged PDF" annotations, which were initially designed for accessibility features such as high contrast, large text, or screen reader (text-to-speech) document flow. It's not a general solution to reflowable PDF output, but it's better than nothing, perhaps. Adobe realizes that such additional markup does nothing for the vast universe of existing PDFs. They have trained an AI model to infer document structure and create the tags on the fly; the marketing term for this is called "Liquid Mode" in recent versions of Adobe Acrobat Reader. I'd like to see a reflow engine in TeX, and there are some good implementations https://hint.userweb.mwn.de/hint/format.html https://hint.userweb.mwn.de/hint/format.html The CSS Text Level 4 has a `text-align: auto` attribute. They point out that text layout is complicated, various human languages and scripts have different approaches to adjusting text width, some of which can be generalized. https://www.w3.org/International/articles/typography/justification https://www.w3.org/International/articles/typography/justifi...
Holmze 3y ago
I currently read through both PDF and EPUB formats, and I don't think PDF is that bad. I think the advantage of PDF is that you can take notes like a paper book, and you can write directly on the document through software such as Goodnote on the iPad. However, PDF is indeed inconvenient to read on small-screen devices, and high-quality PDF files are too large. The PDF of a book may be as high as several hundred MB. The advantage of the EPUB format is that it can adapt to more devices and the files are smaller. It is more troublesome to mark EPUB. Maybe new devices like Kindle Scribe can do it, but as far as my experience of reading EPUB on Kindle paperwhite and iPhone, I can only make some simple marks or record some text by typing. I think PDF is closer to the experience of a paper book, but only if you use a screen no less than the size of a book. The EPUB format is more suitable for mobile electronic reading.
SergeAx 3y ago
I don't like it. EPUB should be as simple as possible, so we may have simple programs to parse/render it on universal devices (ex. PCs/tablets) and cheap (including DYI) specialized book readers. It is quite complex already, and making it into portable document format will just make an order of magnitude more complex.
ijhuygft776 3y ago
[flagged]
- zwayhowder 3y ago
  I had the same thought, but wanted to know why the Author thought they were not. For example, a major issue for self-containment is that EPUB content can embed external assets. A content document can legally include an image or font file whose src is a URL to a hosted server. This is not hypothetical, either; as of the time of writing, Google Doc's EPUB exporter will emit CSS that will @include external Google Fonts files. The problem is that such an EPUB will not render correctly without an internet connection, nor will it render correctly if Google changes the URLs of its font files. The article raises some interesting ideas. Much like PDF and PDF/A, I would say an EPUB/A standard would be potentially useful.
  - adamzochowski 3y ago
    But same font problem exists with PDFs. If font is not embedded into PDF, or rendered into a vector shape that embedded, then PDF will display garbage.
    - BHSPitMonkey 3y ago
      Isn't that solved in PDF/A, which the GP was implying could also be done for EPUB?
      - jasomill 3y ago
        Yes: among other things, PDF/A requires all fonts to be embedded.
  - BarbaryCoast 3y ago
    Thanks for that. I can't read the article, probably because I block WASM (and Javascript) for security. None of my ebook readers have Internet access (for security and for privacy), so none of those internet-only epub files would work for me. This might be "legal", since XHTML was intended for the web, but I assume Google's using it to collect more user interaction data that they can sell to data brokers. FWIW, PDF is simply Postscript that's been compressed. As far as I can tell, almost all documents these days are created with Microsoft Word, TeX, or Postscript. I'm lumping things like PageMaker and LaTeX in with the base they were derived from.

659 points by sohkamyung 3y ago

It's a very well thought through article by the developer of Nota, trying to bring EPUB format up to parity with PDF. It's a serious start and they've already written a viewer. In fact, the article itself is displayed in a browser-based wasm port of the viewer (and looks good!). One issue is how precisely EPUB, which is really XHTML, can reproduce layout. What are the possibilities here? The OP's standard is that the document will look "reasonable". The imply that HTML would need new layout capabilities to match PDF, at least for line breaking: There's two ways to make progress here. One is for browsers to provide more typography tools. Allegedly, text-wrap: pretty is supposed to help, but in my brief testing it doesn't seem to improve line-break quality. The other way is to pre-calculate line breaks, which would only work for fixed-layout renditions. Also, though the author mentions annotations, I don't see how they intend to implement them.

joshjob42 3y ago
The author discusses fixed layout epubs. Effectively, the epub can give a default pagination, line-breaks, font, font size, page size, and positioning for images etc., making it render identically on everything (one might optionally omit pagination if opening in a browser but keep everything else). This can be done already in epub3. But that's not ideal, because then it doesn't look good anymore on a phone, etc. Depending on the reader though, you could override the default, but then you have to hope that your reader does a good job of making a nice document. An alternative is for the epub to specify multiple renderings, for various common screen types. I don't think this is unreasonable as a solution. By all means let's try to get a smart reader, but letting people create defaults for their documents that can be overridden if desired by the user is a good middle ground.
- idoubtit 3y ago
  Indeed, EPUB3 provides all the features that the author wishes. His "portable EPUB" format is just a loosely specified subset. It's unclear if some extra features are included in the format as they are in his "Bene" tool, like the rendering of references (i.e. links with a data-target attribute). The EPUB3 standard is much more complex than EPUB2 (media overlays, mixing fixed layout with reflowed, MathML…). In my experience the implementations are much more varying, and most of them aren't complete. So a "Portable EPUB" may not render as expected because the reader tool lacks some specific feature. The author also requires full JS support, which I supose does not help with portability.
zozbot234 3y ago
Doesn't CSS support layout capabilities for paged media out of the box? An EPUB reader just has to implement good old-fashioned "Print Preview" display mode, and you're set.
zdunn 3y ago
> Also, though the author mentions annotations, I don't see how they intend to implement them. It's discussed at the very end of section 8 and all of section 9 that interactive functionality would use web components.
nine_k 3y ago
PDF does not have any capabilities of line breaking. It is a picture format, similar to SVG, only more rigid. That's why it can't have text reflow, etc. What an ebook format needs is a semantic form of markup, which adapts to devices it is rendered on. HTML + CSS were invented for this goal. With that, book layout authors should consciously relinquish some control on how the book looks, and hand it to the reader. Slight visual imperfections are a small price to pay for this. Who needs visual perfection should go for a PDF. This, of course, becomes hard if any interactive stuff is involved. I would suggest that larger interactive elements should open in a dedicated view when needed, and tiny interactive elements should embrace reflow.
- kps 3y ago
  > HTML + CSS were invented for this goal. HTML (with SVG and MathML) is probably fine for most books, but CSS has spent 30 years resolutely resisting basic typography, i.e. default text baseline alignment.
- wolverine876 3y ago
  You will enjoy the article, which goes into these issues in some detail.
BlueTemplar 3y ago
Ironically, the very example the author uses for annotations doesn't work properly for me : on touchscreen Android Firefox I get a link instead of a popup when press-holding. And aren't annotations (and references) already part of the EPUB specification, and probably even the HTML specification ?!? Finally, I disagree with the press-and-hold for popup being better than the usual practice of hyperlink anchors, IMHO their jumping around is much less disruptive. (As long as the reader's "return" function is working properly, and/or - for the bijective ones - they provide a "back" hyperlink.)
- wolverine876 3y ago
  > And aren't annotations (and references) already part of the EPUB specification I'm pretty sure they are not, based on looking carefully a year or two ago, on recent discussions here on HN, and on the OP's belief that they need to invent it.
int_19h 3y ago
Back in the day, there was something called XSL Formatting Objects (XSL-FO). Meant to be something like XML-based PDF, but somewhat higher level - e.g. it did text reflow. The idea was that you'd distribute an XML document combined with an XSL stylesheet that would transform it to FO for rendering; and the same document could have multiple stylesheets for different target devices, page sizes etc.
- wolverine876 3y ago
  I suppose EPUB could have multiple style sheets and provide defaults for various uses. Professional publishers could do something fancy and beautiful, companies could have their own defaults, individuals could use the EPUB defaults, and there could be FOSS stylesheets that people download and use as desired.

xnx 3y ago

8 days ago, 134 points: "Portable Web Documents – An Alternative to PDF Based on HTML5 (2019)" https://news.ycombinator.com/item?id=39036774 https://news.ycombinator.com/item?id=39036774

crabmusket 3y ago
And this current post is exactly what I was wishing for in my reply[1]. Really glad this was posted! [1] https://news.ycombinator.com/item?id=39037135 https://news.ycombinator.com/item?id=39037135

mr_mitm 3y ago

I'm fully on board with the author's "I want to replace PDF" sentiment. It's true that running code in the document has some downsides, but the vast majority of people does it all the time in their browsers. And it comes with tremendous upsides. Just imagine large amount of data presented in interactive tables which can sort, filter and export or interactive graphs inside the document. We already use HTML+JS so much, why should we stop at documents? Yes, they can't be printed, but in my observation less and less people even own a printer these days, and I see no reason why this trend should not continue. I bet the future will be mostly living, interactive documents. It's funny that I just mentioned this in the other thread [1], but I also felt that there is a need for a format that is self-contained and widely supported by standard software (by which I mean browsers). A well-specified open format would be great, but until then I tackled the self-containedness problem with JS and wrote a Python script that zips and bundles all assets and embeds them as a SPA into one HTML file [2]. The focus is on Sphinx docs but it should work in general with all distributed HTML docs. [1] https://news.ycombinator.com/item?id=39138444 https://news.ycombinator.com/item?id=39138444 [2] https://github.com/AdrianVollmer/Zundler https://github.com/AdrianVollmer/Zundler

fodkodrasz 3y ago
> It's true that running code in the document has some downsides, but the vast majority of people does it all the time in their browsers. This probably has to do something with them having nothing to do, as the big companies managed to convince the frontend dev community that the single best thing to generate layout is on the client machine on the fly. Of course they did it so the users will have a hard time selectively blocking the layout scripts from the ad/spyware most contemporary (web)software development is about. This led us to the point where saving (or God forbid printing!) an article needs a lot of effort in many cases. My observation is: when I need to go to work on the field, I need printed documents. Printed documents don't need firmware updates, their batteries don't run out, and no, I don't need interactivity in documents. Self contained HTML is a good - and necessary - step, but interactivity and executable code is not something we usually need in documents, I only saw somewhat legit need for it on corporate abomination of documents (and some teaching materials possibly).

simonw 3y ago

Here's some really insightful feedback on this idea from Baldur Bjarnason, who has spent significant time working with various W3C groups relevant to EPUB: https://toot.cafe/@baldur/111819472053623911 https://toot.cafe/@baldur/111819472053623911 Example note: "EPUB originally didn't support remote resources and people put a lot of work into changing. Loading stuff over the network is HTML's killer feature. Blocking network assets is a setback for format adoption, not progress."

zaphirplane 3y ago
Allow loading of network resources is not good for security. Surprised this isn’t a worry, tbh didn’t read baldur’s writing
- criddell 3y ago
  But it’s good for tracking what books people are reading. If the history of the internet shows anything, it’s that if surveillance is possible, it will eventually happen. Lots of governments around the world would love to know what their citizens are reading. Few would be bold enough to go after this directly, but if some company operating in their country has the data then there’s a path for that government to get the data.
starkparker 3y ago
Oh for Christ's sake, someone pry Baldur off his cross again. > Blocking network assets is a setback for format adoption People are standing here telling him that _allowing_ network assets _is a setback for format adoption_ and he's just going to keep pounding this stupid, obnoxious drum of his until he runs everyone off. > almost all of the problems described would be solved by getting OS vendors (Google, MS, Apple) to invest more money in EPUB That's back-asswards. Google, MS, and Apple don't give a shit about EPUB, they never will, and it's arguable that we're better served with them not buying a seat at that table considering how poorly their "help" has helped web standards, as much of the rest of his dismissive thread helpfully notes. If he wants money for EPUB standards he should shake the cup at IDPF members who rely on it, and particularly Amazon, to whom he quite vocally abdicated the publishing space 12 years ago.[1] Barking at operating system companies is nonsense at best and how we wind up with another, even more avoidable situation where the space is held hostage by them at worst. At least Amazon can chuck some goodwill money at EPUB development while continuing to kick its ass up and down the market with MOBI. (Aside from all this, his dismissals of the "clunky" reading system complaint, citing how EPUB has "too many divergences", only further proves to me how tunneled the vision is of the people involved. To hell with forking or improving EPUB, then, because it can't be improved if that's the attitude of the people most involved with or influential within it. What bloody point is there in the customizability of a format that _nobody_ can effectively build tools for or consume?) 1: https://www.baldurbjarnason.com/notes/amazon-wins/ https://www.baldurbjarnason.com/notes/amazon-wins/, in which he also admits that he has no idea how to work with IDPF, which is a really great sign of how long things have been going this badly in this space

reacharavindh 3y ago

The thing I wish the most with epub or technically the epub readers is the ability to scribble and hand write notes in them using a stylus and for them to keep them while reading again. I do that with PDFs on my iPad, but have a lot of tech books for which I took manual notes nowhere to be found again - even if I did, they are not inline with what I was reading and thinking.

eviks 3y ago
In general, the modern docs should be easily editable, not just allow annotations, since it's easy to preserve the original content/layout
beckerdo 3y ago
I agree, I would like an ePub to have a robust note taking and exporting ability. For instance, if I highlight in Chapter 8 "In 539" [next paragraph] "Belisarius" [next paragraph] "marched on Ravenna" [10 paragraphs later] "In 540 Belisarius entered Ravenna". I would like to export this with the Chapter header and detailed highlight locations OR just as one sentence with subtle links to the locations.
AlanYx 3y ago
I'm on the same page. I convert all my ePubs to PDF because I want to keep my handwritten annotations in-place alongside the text I'm annotating, including things like circled words. Recent Kobos (Elipsa and Sage) take a decent stab at solving this problem while retaining the ePub format and reformattability, but it breaks too easily.
sriacha 3y ago
Same, its a big wish of mine, just to have simple portable highlights in epub. There's some interesting news in this space: next version of Zotero will have epub annotation (WADM) support: https://forums.zotero.org/discussion/110487/embedding-annotations-in-epub-file-itself https://forums.zotero.org/discussion/110487/embedding-annota....

eviks 3y ago

Commendable effort of trying to get rid of the ancient paper-based legacy in the digital world that is PDF Though I'm curious whether the clunky old-but-still-living HTML (especially in its ugly XML variety) + CSS are the right foundations for the portable format of the future? Since the author has also developed the whole new document language would be nice to read a more in-depth overview on that subject. Or why limit to the ugly duckling of JS in the future when WASM exists? > content by pages. Instead, we should mandate a consistent numbering scheme for block elements within a document, and have people cite using that scheme. that's indeed the proper and more precise approach, though we could still have those "fixed layout epub" pages as a backup coordinate system

thayne 3y ago

> A PDF is a single file that contains all the images, fonts, and other data needed to render it. A PDF can include the fonts. But it often doesn't, and relies on system fonts. One reason for that is because including fonts in the PDF can dramatically increase the size of the file. In some cases a single font could be larger than the entire rest of the file. I've also worked on implementing embedding fonts in some software that generated PDFs. It was surprisingly difficult to figure out how to get it to work reliably. > PDFs are rendered consistently. Not as much as you would think. There are several cases where the same PDF will render differently depending on which PDF viewer you use. Usually the differences are pretty subtle, but occasionally there are edge cases that result in pretty significant differences. I've even run into a case where the same version of Acrobat reader will render a PDF differently depending on what OS you are using.

EE84M3i 3y ago
Is there software that minimizes the fonts by removing code points that aren't used in the document?
- adrian_b 3y ago
  This is a standard feature of the PDF format. Normally all PDF documents include only the glyphs corresponding to the code points actually used in the text rendered with that font. That is why you can go for instance to any site of a vendor of fonts and you can download freely a PDF sample text of an expensive font. You can easily extract the font from the sample PDF, but it will be useless, as it will contain only the few letters that had been used in the sample text.
geraldhh 3y ago
> A PDF can include the fonts. But it often doesn't, and relies on system fonts. found this out, after 20-something years of consistent pdf renderings, in a job interview because my docs allegedly looked odd :/ the daily wtf ...
- jxdxbx 3y ago

teekert 3y ago

As someone who occasionally tries to read scientific literature on their e-reader, which is nice, I can just mail it to my PocketBook account and it shows up, I have a deep hate for PDF. Please let this be a popular thing.

mrich 3y ago

Ironically this did not render in Firefox on Android (just the spinner kept spinning) Worked in Chrome. That said, epubs are great for reading books on mobile. The advantage for pdfs is that they contain highlights/notes, so you can directly import them into Zotero and all your annotations are there. For epub, you have to hope there is a way to export the annotations that are stored by the reader app, and then you have to process them further. Readera is a great reader for mobile that makes this possible. I'm currently working on a script that will convert an epub to pdf, extract the annotations from Readera, and mark them in the pdf. Then I can import the pdf into Zotero, while still retaining the great reading experience of epubs.

Symbiote 3y ago
Works fine in Firefox for Android 122.0 for me.
- mrich 3y ago
  Also loads instantly for me now, didn't make any changes.
zozbot234 3y ago
There is a Web Annotation standard that could be used to export the notes to.
staz 3y ago
It is working for me on my Firefox on Android. One of the nice benefits I can already experience in his document it the working TOC sidebar which allow navigation in the document. (Compared to classical HTML not PDF)
mwilliamson 3y ago
I had a similar problem loading the page on Firefox for desktop with private browsing. It turns out service workers don't work in private browsing, which it seems Bene (the software rendering the page) requires. Switching to a normal Firefox window solved the problem.

DeathArrow 3y ago

I didn't know that EPUB is based on HTML. I always had the impression that it has its own binary format. Using HTML as a base has a lot of sense.

simongray 3y ago
W3C standards basically always build on top of other existing W3C standards.
anthk 3y ago
It's just a zip file. Under Linux/Mac/BSD you can trivially write a script which unzip's and outputs the ebook's HTML files into a large text stream and that output can be used as the input of a text mode web browser, allowing you to read ebooks everywhere with just two lines of code.
arp242 3y ago
It's just a zip file with HTML documents and some (ePub-specific) XML files to define metadata, chapters, and a few things like that. I use this "epub-edit" script to edit them: #!/bin/zsh # # Extract epub file to a temp directory, launch shell to edit it, and re-zip # it. Nothing about this is really epub-specific as such. echo " $@" | grep -q -- ' -h' && { sed '1,2d; /^[^#]/q; s/^# \?//;' "$0" | sed '$d'; exit 0; } # Show docs [ "${ZSH_VERSION:-}" = "" ] && echo >&2 "Only works with zsh" && exit 1 setopt err_exit no_unset no_clobber pipefail full=$1:a tmp=$(mktemp -d) bsdtar xf $1 -C $tmp cd $tmp print "Editing $1; press ^D to exit" zsh ||: mv -f $full $full.orig zip -f $full * cd - rm -r $tmp And then I use vim to edit the HTML files and such.

emayljames 3y ago

The download of the page epub dispays out of the viewport on google books app. Bene seems to be in alpha stage.

diebeforei485 3y ago

I personally think PDF's are a terrible legacy format with unnecessary complexities[1] and most uses of PDF's do not involve printing so the typesetting arguments don't make sense to me. For the vast majority of use cases it's far more important to be readable on phone, tablet, and computer. I was surprised when the author mentioned iBooks doesn't support scrolling view, so I tried it myself. Turns out iBooks on macOS does not support scrolling for ePub files, but it does on iOS and iPadOS. Very strange decision by Apple. 1. https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-into-nso-zero-click.html https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-i...

baq 3y ago
but but but... if I really need to print something, a PDF is the most reliable times portable route. I guess a multipage svg would work, too, maybe, if exported to a pdf to properly print multiple pages first. (Looking at you, inkscape...)
adrian_b 3y ago
PDF is an annoying specification, but there exists absolutely no replacement for it. I have never seen any kind of technical documentation published in any other format than PDF that is comfortable for reading and searching, even when that is done on a mobile phone. I do not want a document that changes appearance depending on the device used for reading or depending on its temporary state, like window size. I want a document whose layout has been well conceived by its author and which is fixed, regardless of what I happen to use for reading it. When I happen to read it on a smaller screen or window, except for trivial text-only documents, I do not want changes in layout, but I only want a smart reader, with comfortable means for fast zoom and pan, and which does not have stupid behaviors (like some Android readers), for instance where scrolling vertically (including Page Up/Page Down) also moves the document horizontally (preventing the easy reading of a column of text). The traditional recommendations for the maximum width of a text column are good enough, if observed, to ensure comfortable reading even on a mobile phone. Only when the author breaks the traditional typographic rules by making extra-wide columns, the reading on a mobile phone becomes inconvenient.
- 3y ago

czierleyn 3y ago

When the IDPF merged with the W3C a couple of years back they tried to develop a new standard called PWP, Portable Web Publications, which was supposed to be a new 4.0 version of EPUB, as far as i know. But there was much resistance from the publishing community and the project was shelved a couple of years ago. See: https://w3c.github.io/dpub-pwp/publishing-snapshots/FPWD/Overview.html https://w3c.github.io/dpub-pwp/publishing-snapshots/FPWD/Ove...

watwut 3y ago

> PDFs are rendered consistently. A PDF specifies precisely how it should be rendered, so a PDF author can be confident that a reader will see the same document under any conditions. And that is why PDF sux for reading on the phone. And why epub is massively better if you want to read articles and books.

geokon 3y ago

It feels like a bit of doomed a project simply bc browsers don't open EPubs. You can link a PDF and while it's a bit of a context switch, the browser will open and display it Since as described EPubs are basically HTML its kinda dumb browsers don't open them - but good luck convincing the Chrome/Mozilla bureaucrats I think another discouraging aspect is HTML CSS are so huge and bloated at this point that few people can implement a "reader" for EPUB/HTML. It's basically "go implement a new browser". It makes one think a easy-to-parse markdown (like Djot) with some extra rendering bells and whistles would be a more likely long term solution My personal interim compromise solution is embedded everything (CSS, svgs, scripts and base64 images) into an HTML file. It's similar to an EPUB. It's a bit bloated and ugly but with a bit of care it works and naturally browsers (and by extension basically every user) can open it Unfortunately a user has no way to really know "oh I can download and store this web page offline". It'd be nice to have some thing like a .htmls extension that indicates it's an HTML but it doesn't have any external resources.

zerof1l 3y ago
> It feels like a bit of doomed a project simply bc browsers don't open EPubs. Not that long time ago, browsers could not open PDFs as well. Now all browsers come with PDF reader written in ASM/JS. I see nothing that prevents browsers doing the same for EPUBs. There exist browser extensions that do exactly this already. Its a matter of EPUB format gaining popularity.
- geokon 3y ago
  I think Google is actively against offline data. It's not aligned with their business interests My mental analogy is, you can also have offline apps on Android. You can specify this in app manifest. But internet access isn't exposed to the user as a permissions. Like the author says, Google already injects online fonts into the EPubs they generate. Meanwhile PDF is a battle they've already lost
  - BlueTemplar 3y ago
    Indeed, and I would add that there's no reason for browser to be able to open PDFs : this sounds as yet another attempt for Google to wrestle with Microsoft over the control of the OS (by having everything happen in the browser instead). And also probably why we still have to rely on 3rd party hacky browser extensions to be able to save web pages as a single file.

zvmaz 3y ago

The author is a post-doc advised by Shriram Krishnamurthi [1], the author of Programming Languages: Application and Interpretation (PLAI), and one of the authors of Data-Centric Introduction to Computing (DCIC). I am currently reading both PLAI and DCIC and I am truly delighted by the minute care the authors have put into making the books pedagogical works of art. That's true love! [1] https://willcrichton.net/ https://willcrichton.net/

verisimi 3y ago
> the minute care the authors have put into making the books pedagogical works of art. That's true love! Works of art. True love! That's very high praise.
- zvmaz 3y ago
  I mean it. Both books are free, and PLAI has an interactive tutorial on the language used in the book called SMoL [1] done by one of Shriram's students. The tutorial is not a passive one by any means; it forces you to think and highlights pitfalls students often fall into when reading the material. This whole ethos on readers learning is in stark contrast to books that feel like the authors show off how smart and clever and profound they are instead of caring about their readers comprehension. I include very highly praised books even on HN. [1] https://www.plai.org/#direct-links-to-the-tutor https://www.plai.org/#direct-links-to-the-tutor N.B. The author of this post, portable EPUBs, also works on language learning. A whole ethos... https://arxiv.org/abs/2401.01257 https://arxiv.org/abs/2401.01257
sotix 3y ago
And the author has worked on making Rust easier to learn[0]! [0] https://rust-book.cs.brown.edu/ https://rust-book.cs.brown.edu/

morelisp 3y ago

The bar for epubs is so fucking low I have a hard time believing this matters at all. Just last week I bought a book set in the late Middle Ages which managed to transcribe all “þ” as “p”. Until publishers care about that stuff, none of these high-falutin technical discussions change anything.

offices 3y ago
I don't see how this relates to the link.
- morelisp 3y ago
  The file format doesn’t matter one bit when the reading and authoring tools are shit and the editors can’t/don’t fix anything. And papers will generally have a lot fewer resources to deal with this than major book publishers, who have been epub-focused for over a decade now and actually make money from it.
  - harshreality 3y ago
    Any decent publishing or html editing tools fully support utf-8 by now. It's not the tools. Publisher and editor laziness may be a reason to be cautious about epubs currently for niche or esoteric works, but that's not the same thing. > I bought a book set in the late Middle Ages which managed to transcribe all “þ” as “p”. Until publishers care... The book market these days makes it challenging to do high-quality editing up front for republishing niche books in a new format. Publishers try to cut corners, outsourcing epub conversions to people who don't care and don't know what they're doing, or they OCR it, have an in-house editor (who also doesn't have a personal affinity to the subject) give it a once-over (maybe), and release it.
    - BlueTemplar 3y ago
      As an aside : Unicode support was still an issue in TeX last I checked, because most of the LaTeX tools don't support it (well, having been made before it was expected). Now, there are some attempts to fix this situation by Xe(La)TeX and Lua(La)TeX, but since TeX seems to be so much tied to PDF these days, it should probably just be abandoned by most scientific publishing in favor of the likes of GNU TeXmacs (note : it's NOT TeX in GNU Emacs) and HTML with MathML.

sirsinsalot 3y ago

I agree with most of the content of this post. One of the key things for me would be a requirement that figures and diagrams which can be expressed as SVG should be. Images, limited to things that need grid-of-pixels representation like photographs, should be limited to that.

livrem 3y ago

I use the SinglePage add-on for Firefox (think it is available for Chrome as well?) to save the current page DOM to HTML as a self-contained file (inlined CSS and data:-URLs for all images) with no dependencies and all the scripts removed etc. It is not perfect, and I do not trust browsers to always remain backwards compatible, but I prefer it to save pages as PDF or as multiple files. Interestingly one of the few pages I ever saw it fail on was this article on portable EPUBs. Guess it has too much magic going on to make the formatting work. The saved page is perfectly readable, but the style is nothing at all like what the original page was for some reason. I like how fbreader on Android just displays all books exactly the same, and as configured in the app rather than using any of the styling from the EPUB file. I never noticed that it tried to apply CSS or run scripts included in files and I hope it never tries to do either of those things. Loading external dependencies sounds like an even worse idea and I did not think that was even allowed.

actionfromafar 3y ago
SingleFile, right? Edit: On that note, what's up with the Firefox Add-Ons? Currently, they are all setup so that to do something interesting, they need all the permissions. Which leads to a natural market being created, of bad actors go shopping for Add-Ons they can take over. Can something be done about this? For instance for this "SingleFile" addon. It needs to access the rendered document in the DOM to be able to introspect and save it all to a file. But why does it need access to everything? Can't it have just permissions: - "snapshot DOM once" - "write to a single file"
- pbronez 3y ago
  Agree. There are many extensions that I want to use on arbitrary websites, but rarely. This could be handled by the browser locking the extension out from everything, until the moment I manually invoke it, at which point it’s allowed access to the page I’m currently looking at. Now, maybe that’s how it already works, but I have no confidence in it.
- gildas 3y ago
  Author here, I wished SingleFile would use less permissions, but it's unforntunately not possible from a technical point of view. Anyway, if you run some code I've written but are suspicious, you have to trust me or review the code which is open-source.

znpy 3y ago

ePub aren't that great either. PDF might not be perfect, but it's likely the best we have and the best we'll have in a long time. ePub renders wildly different on my eReader (kobo), my linux laptop (various apps), my iPad and my iphone. And i'm not talking about screen sizes, i'm talking about various elements being rendered largely incorrectly (and there's a matrix of incorrectness across implementations. PDF documents on the other hands... they render just right, everywhere. I have to zoom and scroll, but I will never have to ask myself "will I be able to actually read this document?" when dealing with PDF. Oh and by the way... I still print stuff from time to time. Yeah way less than it was needed in the past, but it's still a necessity. Can you even print stuff (sections? pages? selections?) from an ePub? > Bene is designed to make opening and reading an EPUB feel fast and non-committal. The app is much quicker to open on my Macbook (<1sec) than other desktop apps. This is elitist at best. Claiming something "is fast" on top-class hardware is misleading at best (if not dishonest). Try and running that on low class hardware (stuff like chromebooks but also laptops from at least 7-8 years ago) and let's see if it's still "fast". I'm not convinced.

broscillator 3y ago
> PDF documents on the other hands... they render just right, everywhere. I have to zoom and scroll, To me it feels the other way around, if I have to zoom and scroll, they render just almost right. And that almost is extremely important for actual reading. If you're quickly skimming a PDF, sure. But to sit down and read for 30 minutes? One hour? Fuck zooming and scrolling. My kindle just displays a page, and I tap it and it goes to the next page. Can't really get a much better reading experience than that.
- znpy 3y ago
  If I’m reading that long, I don’t need zooming and scrolling on my ipad. Epub on ereaders work well but only if you’re reading fiction. Most images and almost all tables and charts have been messed up by epub rendering anyway. And ereaders are black and white, so you’re losing information anyways.
  - 3y ago

crabmusket 3y ago

> There's two ways to make progress [on document aesthetics] here. One is for browsers to provide more typography tools. ... The other way is to pre-calculate line breaks, which would only work for fixed-layout renditions. The third way is to develop non-browser clients, like the author's own Bene. While it currently uses Tauri and therefore the system webview, there's no reason it should always do that, or that another client couldn't be developed with a focus on typography. Want your documents to look super nice with all the kerning and line breaking your heart desires? Get a proper reader app. Just want the content? Sure, open it in a browser or a basic reader.

BonoboIO 3y ago

Wow, I m blown away how fast this site is. It loads instant on my iPhone. Perfect text sizes … really nice.

crabmusket 3y ago
You didn't get a huge multi-second loading spinner?
- BonoboIO 3y ago
  Nope. Normally i'm used to surfing the web even with adblockers to get slow loading times from bloated websites with megabytes of useless libs. This was refreshingly quick.

upofadown 3y ago

>You might dislike the idea that document authors can run arbitrary Javascript on your personal computer. How I feel about this depends a lot on how much I trust the people who created the document and/or the person who sent it to me. I would trust a website I specifically selected with a defined TLS web of trust more than I would trust a random spam email. When we think about the risks associated with the complex and inherently insecure format known as HTML we tend to assume the level of trust available on the web. If we package up a bunch of HTML in a standalone document then we lose that assumption.

BlueTemplar 3y ago

See also : "The decades long quagmire of encapsulated HTML" (2022) : https://www.russellbeattie.com/notes/posts/the-decades-long-html-bundle-quagmire.html https://www.russellbeattie.com/notes/posts/the-decades-long-... (Which still hasn't been posted as a "news" it seems... should I just submit it myself ??)

harshreality 3y ago

I think precisely dictating layout is the wrong objective, although some areas (legal documents, academic papers) are still obsessed with that. Arxiv recently started offering a subset of papers in html, which is a short step from epub. If quick high-resolution referencing (page x, yth paragraph, zth line) is necessary, I think the way to handle it is to reference a phrase on that line ("the cat ran"...), which the reader can search for. If the search interface is lacking, that's an epub reader failure, not a format failure. Or, if the search option is considered insufficient (it does require typing with a [virtual] keyboard), paragraphs can be numbered—as many ancient works or works in translation already are, because such works have many editions and can't be layout-exact copies of each other. If paragraph referencing is necessary, visibly styling the paragraphs with numbers helps dramatically. There's no reason it has to be exclusive to high-profile ancient or classical works, like Plato [1]. Classic poetry and plays, where referencing needs to be most exact and fast, already tend to give up any hope of everyone using the same edition, and simply avoid flowed text and then number the lines. [1] https://en.wikipedia.org/wiki/Stephanus_pagination https://en.wikipedia.org/wiki/Stephanus_pagination

foofie 3y ago
> Classic poetry and plays, where referencing needs to be most exact and fast (...) I believe that paragraph referencing is, by far and without any contest, used primarily by any text subject to reviews and revisions. This means technical reports and academic papers. All academic papers I subjected to review were forced to use templates that enforced paragraph numbering. Even though each version of those documents were only read by a dozen readers or so, all papers submitted to those journals had to use the template. This means hundreds of documents (see half a dozen revisions per paper submitted per each edition) had that hard requirement, and this took place for each edition of a single journal.

xoac 3y ago

I love this unreservedly. It's almost an embarrassment that something like this does not exist already. Big thank you to Will Crichton for putting all of this together and actually giving this idea a chance to take hold.

foofie 3y ago
> It's almost an embarrassment that something like this does not exist already. To be fair, Epub has been largely ignored and neglected by everyone in the world. Virtually no reader supports rendering math notation, and virtually no significant publisher on earth, including the likes of Arxiv, offers Epub downloads. Commercial publishers force DRM onto every format, which excludes Epub, and non-commercial either stick with PDF or don't care.
- julielit 3y ago
  Sorry, but this is not true. Every significant publisher produces EPUB these days. Most reading apps support EPUB (including apps from Apple and Google), Readium and EDRLab offer open-source SDKs that ease the development of mobile, desktop and Web reading software with strong EPUB 3 support, including MathML. Readium LCP is a DRM for EPUB, especially for public libraries that need an e-lending end date. More, EPUB is much more accessible than PDF for blind people and other people with disabilities. PDF has no interest for ebooks (but for short documents, yes).

gmuslera 3y ago

Another standard? (https://xkcd.com/927/ https://xkcd.com/927/). It is not like the publishing world will switch to it overnight, a lot is tied to the devices they sell, so it might be not enough motivation. It is not simpler to convert epubs into epubs with all remote content embedded and relinked? Another single file book format that used to be popular many years ago was chm, that had its own security problems. Maybe adding the possibility of executing (js) code is not the best for something that should be mostly static, and css be used to enable some level of safe interactivity.

asimpletune 3y ago

This is something I deeply care about, as I'm also very interested in the intersection of ebooks, security, and a LowJS web. 1. absolutely we should have a single-file, portable ebook format, and since PDF doesn't reflow text then it's not that. 2. HTML + CSS in 2024 is capable of reproducing virtually any kind of printed medium, but it can also reflow text. 3. I don't personally think JS should be a requirement, but a lot of conversations break down at this point so let do my best to explain and please understand I'm not simply being religious about this. In my view, an ebook is a book that can change its size in a way that makes sense. Bearing this in mind, I believe that if a reader has JS turned off, this book should work as intended. In other words JS shouldn't be required for it to perform its basic function. However, if there is some necessity to add interactivity or to augment the book, then yeah, why not, use JS. That's what it's there for. However, from the perspective of being a book, if it doesn't work as a book without JS then it's a bug in my view. (for standard ereader capabilities, I think the browser should offer that, but the book itself shouldn't be shipped with an ereader) 4. I think it's a mistake to embed all the styling as that may violate some people's CSP's. It's safer to specify separate styling as resources relative to the HTML, and ebooks should forbid loading resources from a separate domain. In this way, ebooks would always work offline, but it has the added benefit of working online too and would automatically adhere to the strictest possible CSP. This achieves the same goal as offline, but in a safer and more universally compliant way. 5. Finally, just distribute it in a zip file. That's how ebooks already work right?

gildas 3y ago
> 5. Finally, just distribute it in a zip file. That's how ebooks already work right? What about self-extracting ZIP files like this page? https://gildas-lormeau.github.io/ https://gildas-lormeau.github.io/ (note that it includes the CSP to make it safe)
- asimpletune 3y ago
  I don't think there's any CSP? https://observatory.mozilla.org/analyze/gildas-lormeau.github.io https://observatory.mozilla.org/analyze/gildas-lormeau.githu... In any case, ideally, any ebook solution should have all resources loaded as files relative to the current document, and nothing inline. Like this, the book would be compatible both offline and to be hosted online, without requiring any changes to the book. The one file thing is cool, but again it requires JS to show anything, so that's not really inline with what I was talking about. In my view, an HTML document should work like a book, and any JS is purely to augment and extend that book if necessary. Those situations are rare though, and most JS is just to give a reader like experience. I think if someone is going to go through the trouble to host a book, they probably don't mind unziping a file before putting it on their server. The one file thing I said earlier was more about sharing the book, similar to an app package. but once it's on someone's servers presumably it's ready to be read. Basically .webarchive

Finnucane 3y ago

I see a couple of roadblocks for this. One is that is suggestion of a restrictive subset of HTML for coding seems like a potential accessibility problem, which is to say, you'd have to make your documents less semantically rich. For instance, he seems to be suggesting using. It's already hard enough to get epubs to work right when reading systems lag behind what browsers can support, saying 'let's have less' is not going to make things easier if you have complex content. The problem is not that there is too much html or css, the problem is that reading systems don't support them properly. Also, most dedicated reading systems (Kindle, Kobo, etc) don't allow javascript, which means your components will not work. That might of course change, but I wouldn't hold my breath for it.

velcrovan 3y ago
> One is that is suggestion of a restrictive subset of HTML for coding seems like a potential accessibility problem, which is to say, you'd have to make your documents less semantically rich. "less semantically rich" than what? Web pages? Or less rich than PDFs, which is what he's actually proposing to replace?
- Finnucane 3y ago
  Than the epub standard as it exists. I guess I don't understand what the advantage here is, really. If you want to make an epub file that works universally, you can do that, within the existing standard. If makers of reading systems and software would actually support the full standard, which they currrently don't. If you don't want to use pdfs, don't use pdfs. pdfs get used for a lot of things they aren't actually very good for.
  - velcrovan 3y ago
    > If you want to make an epub file that works universally, you can do that, within the existing standard. Yes, that’s what the author is proposing we do.
    - Finnucane 3y ago

ryukafalz 3y ago

I did a double-take when I reached this part: > Therefore I decided to build a lighter EPUB reading system, Bene. You're using it right now. This document is an EPUB — you can download it by clicking the button in the top-right corner. Because, reading this on a desktop browser, I didn't even notice until it was pointed out. It's more obvious on mobile because the header takes up more of the viewport, but it otherwise behaves pretty much like a normal web page. This is probably a good thing. For what it's worth, I didn't see (or at least didn't notice) a spinner when loading the doc for the first time like some other people in the comments reported. I did notice it on my phone, but it went by pretty quickly. I'm not sure if that's the WASM program loading and if it only happens the first time you load the page.

edflsafoiewq 3y ago
The main thing the spinner waits on is the .epub file to be downloaded. That file is 4.77 MB, which is appreciable for anyone without a fast connection. Most of that weight is images (99% after decompression). Unlike a normal webpage (or a PDF), rendering doesn't appear to start until all the assets, the whole ePub, has been downloaded. This segues into a point of difference I thought the article would mention, but didn't: performance. A PDF can be optimized so the pages are substantially independent of each other, which makes rendering pages progressive, random-access, and highly parallelizable.
- AA-BA-94-2A-56 3y ago
  This makes me wonder if EPUB pagination is possible? I know nothing about it, but if the format is standard enough, it might be possible to hack together an endpoint that returns the requested page(s).
sinuhe69 3y ago
On my phone, I didn’t notice it at all until I read it. I just thought it is a normal webpage describing the project.

aabdulllah 3y ago

Hi, I was wondering if someone could explain what software solution I need to create and implement in my calculator for it to have a fast response time when calculating. How does this work, in other words what enables it to work so fast?

aabdulllah 3y ago

p.s. sorry I am new to engineering.

SamBam 3y ago

One thing I'm very interested in, as a grad student who has to consume a huge number of PDFs, is whether there are good tools for converting existing PDFs to portable EPUBs or HTML documents. If I use, for instance, CloudConvert [1], I generally get a document that gets flowing text roughly right, but still interrupts the text with page numbers and book titles (that were originally at the top of each page) and includes additional bizarre line breaks, etc. Every so often I wonder if this is an LLM problem ("please reformat the following text to...") but I think that one shouldn't reach for an LLM for these kinds of things. 1. https://cloudconvert.com/pdf-to-epub https://cloudconvert.com/pdf-to-epub

nmz 3y ago

> * PDFs cannot easily express interaction. PDFs were primarily designed as static documents that cannot react to user input beyond filling in forms. I am glad about this, I do not want to download a document and it require any input. a document should be a document, nothing more. If I'm getting a book to read (pdf), I expect a book, not a webapp.

sotix 3y ago

I really like this proposal. Just last night, I was reading an EPUB of The Hobbit and clicked on a footnote, which instructed me to refer to page 24. It turned out that it meant page 24 of the printed edition of the book, which was in the first chapter of the book. Page 24 in the EPUB was still part of the prologue. So I had no idea which page it was referencing. As it stands, my Kobo has an increased font size, so I notice that I can flip the page a few times and still be on page 4 before it finally turns to page 5, which I assume is referencing the pages of the written text. This is a nice compromise, but doesn't solve the issue with the hard coded footnote being misleading. I wonder if we could instead look at religious texts such as the Bible (e.g. John 3:16) and code editors (e.g. Ln 4, Col 12) for referencing locations in reflowable text. The same way you can jump to a footnote in a document should allow you to have an actionable reference to a specific location anywhere in the text. But I don't think the text should be stylized like how the Bible has the numbers (e.g. 16) scattered within the text itself. Those should probably be hidden within the text and leave the reading software to display the first line number of the page down at the bottom instead of the page number. That might look like "4" the same as it currently does, but this 4 references a section of the text rather than a page number. Perhaps it could be togglable for greater detail and display word 23 of section 4 as "4:23". Or maybe it could consider the chapter too. For example, chapter 2 section 4, word 23 would look like "2, 4:23". This might get funky in a Terry Pratchett novel, but it would hopefully allow for easier discussion of exact parts in a document and significantly easier linking. I love the interactive code example for marking up The Rust Programming Language. That gets at what I was saying above although is more targeted at a document than referencing parts of a novel. Kudos to author for creating Bene[0] as part of this proposal. That was cool to discover I was using their tool to read the proposal itself! [0] https://github.com/nota-lang/bene/ https://github.com/nota-lang/bene/

Shorel 3y ago

On the one hand, these are just EPUBs, nothing in the article makes the generated EPUBs different or incompatible from the ones I usually read. On the other hand: this looks awesome for the Web. Mix it with a blog platform like Medium, and it will improve my Web browsing experience tenfold.

1vuio0pswjnm7 3y ago

It appears this site (Github pages) is trying to require Javascript in order to read it. At the same time the top comment is suggesting EPUB should not require Javascript. I never use the HTML markup, CSS, etc. when I read an "ebook". I dump it to text and read it with less(1). I do not need to see images. I just want to search and read text. I use stupid simple shell scripts; something like the following. #!/bin/sh printf "(\n" for x in ${1-*.epub};do echo printf \'\\n\\n,,,,,,,,,,,,,,,,,,,,, "$x" ,,,,,,,,,,,,,,,,,,,,,,,\\n\\n\' 7z l $x|sed -n "s/.* /7z x -so $x /;/nav.xhtml/d;/[tx]ml$/p"|sort -n done echo ")|tr -cd '[\\\12\40-\176]'" Example usage: epub.sh 1.epub |sh > 1.htm links -dump 1.htm > 1.txt less 1.txt If I need to to adjust the order of the sections, I do something like epub.sh 1.epub > 1.sh vi 1.sh 1.sh > 1.htm links -dump 1.htm > 1.txt If I want to save the "ebook", I can compress the .txt smaller than any of the "ebook" formats. zstd -19 1.txt zstdless 1.txt.zst Plain text is portable for me; works on both BSD and Linux.

akoboldfrying 3y ago

What does an EPUB reader provide that a browser doesn't? Because right now it seems to me that the answer is just one, trivial thing: It can treat a .epub file (that is, a zip file) like a directory. If that really is the only thing that differentiates an EPUB reader from (a subset of) a browser, it seems that the simplest and least-effort thing to do would be: 1. Enable existing browsers to treat a zip file like a directory, under certain circumstances. 2. Describe a machine-checkable restriction of HTML that guarantees fully local assets as described in TFA (this might be as straightforward as requiring all URLs (except for links) to be relative). Declare any zip file that satisfies this restriction to be "a portable EPUB". 3. Forget about special-purpose EPUB readers altogether, both existing and new.

LorenPechtel 3y ago
Agreed, this sounds like the real solution. As for fully local assets--what I would like to see is a requirement that all objects have locally available fallback options. You can't get that font, use this standard one. Might not look perfect but it should work reasonably well if you limit yourself to things in the fallback. There is still one failure mode he didn't fully address--images that don't work right. Fundamentally, this is because images can't reflow. To a large degree I think this could be addressed with using SVG, but that's not going to handle things which are truly raster rather than vector. Unfortunately, I think the only way to address that is provide a few resolutions. (And I have faced a related problem that I haven't seen a good answer for but I haven't looked hard: SVG that is aware of it's resolution and can bring in more detailed images if it has the resolution to do so. Try to draw sub-pixel lines and you're going to get a blur, but blowing up the same image and those lines become a good thing. Windows acknowledges this with icons, you can include a variety of resolutions and it will work with whatever is most appropriate for the context.)
- akoboldfrying 3y ago
  >a requirement that all objects have locally available fallback options If we instead mandate that every font used is either packaged within the zip, or one of a small, fixed, well known list of "standard" fonts that every system can be required to support, what would be lost? (IOW, what do we gain by allowing the concept of "fallbacks"? Because what we'd lose would be consistency between internetful and internetless renderings, and self-containedness.) >images that don't work right What's an example where storing a single high-resolution image, which can be dynamically scaled down if needed, doesn't work well?

watersb 3y ago

I care very much about ebooks and readability. One stunning failure of current ePub technology is the lack of publication-quality justification and microtypography. Currently, PDF authors are overloading "Tagged PDF" annotations, which were initially designed for accessibility features such as high contrast, large text, or screen reader (text-to-speech) document flow. It's not a general solution to reflowable PDF output, but it's better than nothing, perhaps. Adobe realizes that such additional markup does nothing for the vast universe of existing PDFs. They have trained an AI model to infer document structure and create the tags on the fly; the marketing term for this is called "Liquid Mode" in recent versions of Adobe Acrobat Reader. I'd like to see a reflow engine in TeX, and there are some good implementations https://hint.userweb.mwn.de/hint/format.html https://hint.userweb.mwn.de/hint/format.html The CSS Text Level 4 has a `text-align: auto` attribute. They point out that text layout is complicated, various human languages and scripts have different approaches to adjusting text width, some of which can be generalized. https://www.w3.org/International/articles/typography/justification https://www.w3.org/International/articles/typography/justifi...

Holmze 3y ago

I currently read through both PDF and EPUB formats, and I don't think PDF is that bad. I think the advantage of PDF is that you can take notes like a paper book, and you can write directly on the document through software such as Goodnote on the iPad. However, PDF is indeed inconvenient to read on small-screen devices, and high-quality PDF files are too large. The PDF of a book may be as high as several hundred MB. The advantage of the EPUB format is that it can adapt to more devices and the files are smaller. It is more troublesome to mark EPUB. Maybe new devices like Kindle Scribe can do it, but as far as my experience of reading EPUB on Kindle paperwhite and iPhone, I can only make some simple marks or record some text by typing. I think PDF is closer to the experience of a paper book, but only if you use a screen no less than the size of a book. The EPUB format is more suitable for mobile electronic reading.

SergeAx 3y ago

I don't like it. EPUB should be as simple as possible, so we may have simple programs to parse/render it on universal devices (ex. PCs/tablets) and cheap (including DYI) specialized book readers. It is quite complex already, and making it into portable document format will just make an order of magnitude more complex.

ijhuygft776 3y ago

[flagged]

zwayhowder 3y ago
I had the same thought, but wanted to know why the Author thought they were not. For example, a major issue for self-containment is that EPUB content can embed external assets. A content document can legally include an image or font file whose src is a URL to a hosted server. This is not hypothetical, either; as of the time of writing, Google Doc's EPUB exporter will emit CSS that will @include external Google Fonts files. The problem is that such an EPUB will not render correctly without an internet connection, nor will it render correctly if Google changes the URLs of its font files. The article raises some interesting ideas. Much like PDF and PDF/A, I would say an EPUB/A standard would be potentially useful.
- adamzochowski 3y ago
  But same font problem exists with PDFs. If font is not embedded into PDF, or rendered into a vector shape that embedded, then PDF will display garbage.
  - BHSPitMonkey 3y ago
    Isn't that solved in PDF/A, which the GP was implying could also be done for EPUB?
    - jasomill 3y ago
      Yes: among other things, PDF/A requires all fonts to be embedded.
- BarbaryCoast 3y ago
  Thanks for that. I can't read the article, probably because I block WASM (and Javascript) for security. None of my ebook readers have Internet access (for security and for privacy), so none of those internet-only epub files would work for me. This might be "legal", since XHTML was intended for the web, but I assume Google's using it to collect more user interaction data that they can sell to data brokers. FWIW, PDF is simply Postscript that's been compressed. As far as I can tell, almost all documents these days are created with Microsoft Word, TeX, or Postscript. I'm lumping things like PageMaker and LaTeX in with the base they were derived from.