26 ms·
Standard Ebooks
- zinsn1 3y agoTo have access to what I read and also remember it, I created recently a web app called bookeeper that exports your Kindle highlights to notion, generating a personalized summary of it with AI as well. Try it here if you are interested: https://bookeeper.io/?utm_source=hacker_news&utm_medium=book https://bookeeper.io/?utm_source=hacker_news&utm_medium=book
- knbknb 3y agoWho decides or sets the difficulty level of "reading ease" (which is a sortable metadata attribute on the search page) ? Some classiications seem a bit ...nonintuitive. For example, the Autobiography of John Stuart Mill is classified as "very diffcult" whereas "The Tempest" by Shakespeare is classified as "fairly easy". I would classify it the other way around, but what do I know, I'm a nonnative speaker anyway.
- robin_reala 3y agoWe use the Flesch Reading Ease algorithm. You can find the implementation at https://github.com/standardebooks/tools/blob/master/se/formatting.py#L173 https://github.com/standardebooks/tools/blob/master/se/forma... https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests#Flesch_reading_ease https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readabi...
- kjartanThor 3y agoI have a question regarding books in other languages than English. Is there a technical reason for only allowing English or is it the legal aspect (knowing what is in public domain in countries other than the US) that hinders this? For example I live in Iceland there is a number of texts that are in the public domain, for example the Edda and Icelandic sagas among others. But since we are very few (approximately 380K people) there is no comparable entity and most likely never will be, so the best and probably only way to get something similar would be to be a part of a larger organization.
- robin_reala 3y agoIt’s neither: it’s that our tooling and manual of style[1] is only developed for our specific conventions for English text. Other people have attempted to start alternative projects but getting that set of standards together takes time and effort, and like you say 380k people isn’t that many. If you do decide to start something then SE’s tooling supports a --white-label flag when creating a skeleton, which would at least get the first few productions off the ground. [1] https://standardebooks.org/manual https://standardebooks.org/manual
- kjartanThor 3y agoThank you for this, are your tools licensed in such a way that it would be ok to extend and adopt them so that they can be used for other languages?
- robin_reala 3y agoThe code is GPL-3 and the templates are CC0: https://github.com/standardebooks/tools/blob/master/LICENSE.md https://github.com/standardebooks/tools/blob/master/LICENSE.... Feel free to ask on the mailing list if you have any questions, more likely to be picked up there than in a random HN thread :)
- Handprint4469 3y agoI would love if they offered a download option for a file you could just upload to Lulu (or similar service) to have it printed and mailed to you. Every time I buy one of these public domain books from Amazon, they are invariably shitty, low-quality "printed by Amazon" versions. I miss the time where you could get a high-quality hardcover, but more and more those seem reserved only for the current week's NYT best-seller books.
- moonchild 3y agoLulu does not make 'high-quality hardcovers'.
- dimmke 3y agoThis could be a cool monetization strategy. I don't really read physical books, but the "classics" on Amazon are often complete ripoffs. Here's Crime and Punishment for $10 just to get the Kindle version: https://www.amazon.com/Punishment-Penguin-Classics-Fyodor-Dostoyevsky/dp/0140449132/ https://www.amazon.com/Punishment-Penguin-Classics-Fyodor-Do... I feel like these open domain novels published by big publishing houses have the veneer of legitimacy, but projects like the one this thread is about I think could accomplish much more. Especially for authors where the work is translated into English. Plus the cover designs are much cooler. I will say, the search on their website is kind of slow and could use some work.
- zuminator 3y agoIn fairness though, if you sort by price, you can always find classics on Amazon for dirt cheap. E.g. https://www.amazon.com/Greatest-Works-Dostoyevsky-Punishment-Underground-ebook/dp/B00J0647OA https://www.amazon.com/Greatest-Works-Dostoyevsky-Punishment... Although there's no saying as to whether or not they will have proper spellcheck, TOC, if they are legitimately in the public domain, or even if it's the right book with all the pages. That's where a service like Standard Ebooks is superior to the potluck you get from Amazon.
- smogcutter 3y agoNot the greatest example, as the translation is not public domain.
- dflock 3y agoA well run open source ebook project, producing the highest quality ebooks. Always looking for volunteers as well as donations.
- dalanmiller 3y agoWhat’s the best way to volunteer?
- acabal 3y agohttps://standardebooks.org/contribute https://standardebooks.org/contribute
- dimmke 3y agoThis is really cool. I'm going to donate.
- mmastrac 3y agoI published a couple of books for the project during a sabbatical in 2021 (The Devil's Dictionary [0] and a cheesy, small H. Beam Piper book named Four-Day Planet). The process and tools are quite nice and it's very rewarding to see your work in ebook form. It takes a _long_ time to proof and re-read a book, but it's surprising how many times you can do this and how differently you need to read to catch errors versus just enjoying the damn book. The fascinating part of the project is a _strong_ editorial opinion, which IMO makes the project successful. There is a core group of people that upholds the standards for the project, and the resulting consistency of quality of output derives from that. The team clearly cares about the quality, and has demonstrably maintained that over the huge number of releases. I even went to the archives of the "San Francisco Newletter and California Advertiser" to collect some of Bierce's original work, making it the most complete, and most corrected open-source version of the book. [1] The one previously hosted by Project Gutenburg was quite old and, frankly, quite riddled with transcription errors. I haven't tried reading the Devil's Dictionary back-to-back since I published it, but I might one day. There's a lot of detail in this work that I never saw until I had it under a microscope. [0] https://standardebooks.org/ebooks/ambrose-bierce/the-devils-dictionary https://standardebooks.org/ebooks/ambrose-bierce/the-devils-... [1] https://archive.org/details/san-francisco-newletter-dec-11-1875 https://archive.org/details/san-francisco-newletter-dec-11-1...
- NKosmatos 3y agoFor other curious HNers, what differentiates [0] them from Project Gutenberg [1] is the improved typography/styling and the full usage of modern reader techniques. Think of it like, etext != ebook. [0] https://standardebooks.org/about/what-makes-standard-ebooks-different https://standardebooks.org/about/what-makes-standard-ebooks-... [1] https://www.gutenberg.org https://www.gutenberg.org
- chrismorgan 3y agoSo why don’t they contribute these things back to Project Gutenberg? Particularly the typography ones like curly quotes and proper dashes, as those are almost always corrections where the overly-ASCII Gutenberg source doesn’t match the original.
- mouse_ 3y agoAlso wondering this.
- acabal 3y agoLike PG, our editions are blends of other editions, along with our own updates. Often our edition winds up looking nothing like the PG edition, for example when we combine volumes, extract footnotes into endnotes, remove pagination, and so on. So submitting back to PG would be more like replacing a PG edition, instead of updating it; and I doubt the original PG submitter would like it if their hard work was simply replaced by someone else who thought their version was an improvement. Our volunteers do sometimes submit typos they find back to PG. We don't require that, so some producers do, and others don't.
- chrismorgan 3y agoYeah, I was just looking through A Christmas Carol and observed a handful of editorial changes in the commits <https://github.com/standardebooks/charles-dickens_a-christmas-carol/commits/master https://github.com/standardebooks/charles-dickens_a-christma...> (bran-new → brand-new, frouzy → frowzy, and “Lowercase some gratuitously uppercased words”). Frouzy → frowzy I’m mildly in favour of. Ditching bran-new definitely loses character (he omitted the d on purpose!). One or two of the lowercased words were mildly strange capitalised (e.g. Idol was inconsistent with the previous paragraph); but the lowercasing of many introduces broad stylistic inconsistency, and direct local inconsistency sometimes; and most of the capitalisations were not gratuitous. In fact, more than a few were clearly to be pronounced, as a form of emphasis (e.g. Poor, One, Us); and some were distinctly proper nouns in the context, the removal of which increases the parse difficulty (e.g. One¹, /(Cold )?(Roast|Boiled)/); and some reflect customs still common or even preferred in their domains (e.g. Act, Angelic, Apostles, Star). I just reckon that commit should be reverted, because from my perspective it’s mostly actively bad, and the rest subjective. I’m curious what your reaction is to my opinion here. But yes, I see that you’re practising some editorial oversight and not aiming to faithfully represent the original in all regards, which I gather is more generally Project Gutenberg’s goal; and this would obviously contraindicate upstreaming. On the other hand, when it comes to more stylistic matters, I tend to wish Project Gutenberg had more consistency. There’s too much gratuitous variation in presentation and ridiculous 256-colour backgrounds. It’s often too obvious much of it is the work of a group of individuals rather than a coherent effort. I’m curious about the footnote-to-endnote thing, because I’m not sure how the various formats in question handle them all, but in print endnotes are almost always just awful. If anything, I’d be expecting to replace endnotes with footnotes. (Me, I’m partial to sidenotes.) —⁂— ¹ Hickory dickory dock, three mice ran up the clock; the clock struck one, and has been charged with assault and battery.
- acabal 3y agoEditor-in-Chief here, happy to answer any questions! Of interest might be my blog post on how SE runs on a small VPS using classic web tech: https://alexcabal.com/posts/standard-ebooks-and-classic-web-tech https://alexcabal.com/posts/standard-ebooks-and-classic-web-... (This post is slightly out of date as there is a database now; but it's used for managing Patrons - and soon a cover art listing and approval system - not for serving the actual ebooks, which are still served as described in the post.) Our volunteers have spent the last few months preparing a few notable books published in 1928 to be released today, Public Domain Day. Those are the top 5 books in the ebook list, starting with The Mystery of the Blue Train. Check them out! We welcome new contributors if you'd like to work on producing a new ebook. In the next week we'll also have a brand new cover art database launched, so if you'd rather help by cataloguing new cover art for future ebooks, get in touch at our mailing list!
- collinmanderson 3y agoAmazing that you don't use front-end javascript at all. Is there anything you wish HTML+CSS could do better? Re XHTML - It looks like the website is being served with a text/html content type. Did you give up on your XHTML experiment? How did it go? Maybe it would help if browsers reported back errors to the server, like how "Content Security Policy" reporting works.
- robin_reala 3y agoWe never served as application/xhtml+xml. That has some nasty side effects in browsers, like no incremental rendering. But it’s always been legit to serve XHTML as text/html even if you lose the break-on-error functionality.
- Lukas_S 3y agoThis is such a cool project. Every time it hits the front page I browse the selections like I’m at a book store. Have you considered making books sortable by popularity? It might be more approachable for new users if they see books they recognize at the top.
- harwoodjp 3y agoI wonder if a scan -> OCR -> LLM proofreading pipeline is possible?
- eigenvalue 3y agoI made a tool like that, and I bet with a more powerful LLM like GPT4, and perhaps a better baseline OCR tool (like GPT4 vision), it could work really well for this sort of thing: https://github.com/Dicklesworthstone/llama2_aided_tesseract https://github.com/Dicklesworthstone/llama2_aided_tesseract
- Forge36 3y agoAs a first pass I'm sure it'll save some effort (i.e. l -> 1 in some fonts). I can't imagine it fully replacing and editing/proofreading passes.
- weijiacheng 3y agoI am one of the SE editors/regular contributors and I did play around with this a bit for a poetry collection: https://groups.google.com/g/standardebooks/c/IUvGLmvZrmM/m/sFHo1iO3BgAJ https://groups.google.com/g/standardebooks/c/IUvGLmvZrmM/m/s... I'm sure someone sufficiently determined and good at prompt engineering, and integrating LLMs into a larger toolset, could come up with something even better. I'm personally very skeptical of LLMs as a technology, but even I have to admit that this was a pretty ideal and unobjectionable use of LLMs. That being said, though it was a fun experiment, I later found that it was easier (and less wasteful of natural resources) to just do the same thing with a bit of custom markup and a search and replace script.
- duskwuff 3y agoI don't think that's quite what the parent had in mind. The most natural application of a language model in proofreading is to compute perplexity across the text; if all goes well, errors should be detectable as points of unusually high perplexity. (In principle, this should even be able to spot otherwise undetectable errors like missing words.)
- flxfxp 3y agoVery cool project. Does anyone know of something similar for audiobooks?
- Uvix 3y agoAny audio recording will have its own copyright separate from the base text, so it'll be a while before any quality audiobooks enter the public domain. For now, your best approach would be to take high-quality ebooks like what Standard Ebooks offers, and use text-to-speech software.
- rsanek 3y agohttps://librivox.org/ https://librivox.org/
- jodrellblank 3y agoThere's a compelling listen: https://librivox.org/the-first-fifty-digits-of-pi/ https://librivox.org/the-first-fifty-digits-of-pi/
- acabal 3y agoLibrivox creates audiobooks of PD texts. I've heard good things about their work but I personally don't listen to any audiobooks in general.
- wazdra 3y agoThis is very nice ! I’d love to see this for French literature too
- andrewedstrom 3y agoStandard Ebooks is fantastic! In fact, I love what they're doing so much that I actually built a little SaaS product on top of their ebook collection. The site is called Modern Serial, and it lets you read books from Standard Ebooks in 10 minutes a day as Substack-style email newsletters. https://modernserial.com/ https://modernserial.com/
- 100k 3y agoI’m happy to see Standard Ebooks here! I’ve read their editions of Nostromo by Joseph Conrad and Vanity Fair by William Thackeray and the quality great. I recommend it if you’re interested in classic literature.
- pseingatl 3y agoWhat are the standard dimensions produced by se build-images?
- dang 3y agoRelated: Standard Ebooks - https://news.ycombinator.com/item?id=32215324 https://news.ycombinator.com/item?id=32215324 - July 2022 (256 comments) Free and liberated e-books, carefully produced for the true book lover - https://news.ycombinator.com/item?id=25138534 https://news.ycombinator.com/item?id=25138534 - Nov 2020 (106 comments) Standard Ebooks: Free public-domain ebooks, carefully produced - https://news.ycombinator.com/item?id=20594802 https://news.ycombinator.com/item?id=20594802 - Aug 2019 (129 comments) Standard Ebooks: Free and liberated ebooks, carefully produced - https://news.ycombinator.com/item?id=14570035 https://news.ycombinator.com/item?id=14570035 - June 2017 (96 comments)
- contrarian1234 3y agoThere are HTML versions but they don't seem to reflow example: https://standardebooks.org/ebooks/rudolph-erich-raspe/the-surprising-adventures-of-baron-munchausen/text/single-page https://standardebooks.org/ebooks/rudolph-erich-raspe/the-su... A bit offtopic, but I never understood why .epub is a thing. For instance the linked HTML/XHTML version seems to work just fine (except for the reflow thing.. but I assume that a CSS issue) .epub seems to be mostly HTML with a few pieces missing. I guess I don't understand why we needed a new format? and not just use a strict HTML subset? I'd love some strict HTML subset that indicated the file can be used offline. I personally try to make all my webpages so that they can be saved to disk and opened from a single file (though if you embed images/videos this becomes problematic). But I don't have a way to indicate to a reader "Hey you can Ctrl+S this webpage". I'd publish .epub, but the browser won't open them
- boznz 3y agoepub is just a collection of zipped HTML, CSS FONTS and images and as bog standard as you can get. You can open it with a Zip extractor and see.
- contrarian1234 3y agoHuh.. yeah.. so then why doesn't Firefox/Chrome open it?
- robin_reala 3y agoEdge used to support ePubs directly, but they removed that functionality for some reason. There is a little more to it than just rendering. E.g. ideally you’d want some popup table of contents support.
- boznz 3y agoChrome opens the html pages fine once the epub is unzipped. Some epubs may however have DRM I dont have any of those but they probably wont work.