5 ms·

ISBNdb dump – how many books are preserved forever?

174 points by pilimi_anna 4y ago

billblack 4y ago
As someone who would like to publish, my main concern with ISBN's is the cost, because publishers are required to assign an ISBN to every item in their catalog. Section 6.1 of the ISBN International User Manual "A separate ISBN shall be assigned to each separate monographic publication or separate edition or format of a monographic publication issued by a publisher." This would not be a problem if the numbers were more affordable.
- pilimi_anna 4y ago
  This is interesting to learn about. How expensive is it, and do you know if it differs around the world (since there are lots of national ISBN agencies)?
  - jkingsman 4y ago
    It's $125 list price for a single ISBN, but there are bulk discounts buying direct and purchasing through a large scale supplier can make them as cheap as $10 each. There are deals to be had; Amazon, for example, may give you a free ISBN for your ebook as long as you publish it using KDP, their walled-garden publishing system, but the gotcha is the ISBN is not portable/you're not permitted to use it for other editions outside of the Amazon system. The other downside to these free (just about always) and discounted (sometimes) ISBNs is that they link the publisher as the service you got the ISBN through, rather than yourself, even if you're doing what would classically be considered a self-publishing job. How big of an issue is that? IANAExpert, but it seems like there are some nooks and crannies of IP law that can be swayed by owning the imprint, but little practical concern for the average person putting an ebook on Amazon e.g. Perhaps someone with more in depth publishing knowledge can color the risks better than I.
    - pilimi_anna 4y ago
      $125 is steep, but $10 sounds very doable (for the US). Good point on the "publisher" linking caveat though. I don't know much that matters in this day and age? Would be useful to learn from some published authors.
    - toomuchtodo 4y ago
      Can I just start a non profit “publisher”, buy a block of ISBNs, and hand them out at cost? Costco for ISBNs sort of thing, just enough margin to pay for a few hours a year of my time and a little app to drive the process. Edit: HN throttling, can’t reply. What if the Internet Archive gets a block and hands them out via Open Library? They seem positioned to argue they’re a bonafide publisher.
      - Finnucane 4y ago
        "There are unauthorized re-sellers of ISBNs and this activity is a violation of the ISBN standard and of industry practice. A publisher with one of these re-assigned ISBNs will not be correctly identified as the publisher of record in Books In Print or any of the industry databases such as Barnes and Noble or Amazon or those of wholesalers such as Ingram."
    - Finnucane 4y ago
      >Amazon, for example, may give you a free ISBN for your ebook as long as you publish it using KDP, That is indeed a bit of a ruse, since an ISBN is supposed to identify an edition or format, but not the sales channel. We give our epub files an ISBN, and all the vendors that sell that file (including Amazon) use the same number. But when you publish with KDP, you are not the publisher. Amazon is, so you have less say in the matter.
  - wrs 4y ago
    For the US, between $125 each (for 1) and $1.50 each (for 1000) from the official source, a company called Bowker. The structure is described in Bowker’s FAQ [0]. [0] http://isbn.org/faqs_general_questions#isbn_faq6 http://isbn.org/faqs_general_questions#isbn_faq6
    - fragmede 4y ago
      In the middle is 10 for $295, which is $30 per. That's a small enough number that you can split the cost with a few friends. But you can also get one from Australia, $88 for 10 or $44 for one.
  - jwilk 4y ago
    In Poland, you get ISBNs for free.
- gigel82 4y ago
  That's surprising; I wanted to make a picture book (with a bit of text the kids wrote) to send to grandparents and stumbled upon BookWright; seemed an affordable choice but was very surprised they actually included an ISBN with the little one-off kids picture book. Maybe they're just sitting on a big block of numbers and just giving them away...
mechanical_bear 4y ago
Forever? 0.
bloak 4y ago
Google has claimed that about 130 million books have been published (that factoid is all over the web). The number of 10-digit ISBNs is 1000 million (there's a check digit) and people have only just started using 13-digit ISBNs that start with 979 instead of 978; but of course there must be lots of wasted ISBNs, for example when a publisher optimistically buys a big block and then goes bankrupt. Both those numbers suggest that the "ISBNdb" with less than 31 million ISBNs is far from complete. The frequency of each top-level prefix (which tells you the geographical or language region) would be interesting. That would the first thing I'd calculate if I had the data on my disc.
- eCa 4y ago
  Yeah, lots and lots of unused ISBNs. As an example, O’reilly has the 978-0-596 series. That’s a hundred thousand editions.
- adolph 4y ago
  An ISBN is assigned to each separate edition and variation (except reprintings) of a publication. For example, an e-book, a paperback and a hardcover edition of the same book will each have a different ISBN. Additionally, there is address fragmentation; ISMB has blocks: ISBN issuance is country-specific, in that ISBNs are issued by the ISBN registration agency that is responsible for that country or territory regardless of the publication language. The ranges of ISBNs assigned to any particular country are based on the publishing profile of the country concerned, and so the ranges will vary depending on the number of books and the number, type, and size of publishers that are active. Some ISBN registration agencies are based in national libraries or within ministries of culture and thus may receive direct funding from the government to support their services. In other cases, the ISBN registration service is provided by organisations such as bibliographic data providers that are not government funded. https://en.wikipedia.org/wiki/ISBN https://en.wikipedia.org/wiki/ISBN
- ComputerGuru 4y ago
  I always loved how despite the massive domain differences, the ISBN situation is extremely similar to the IPv4/IPv6 situation (except more aggressively rent-seeking), with prefixes leased out to the old dogs, concerns about eventual address/isbn exhaustion, a scheme for mapping old ISB10 to new ISBN13 codes, etc etc.
pugworthy 4y ago
Define "forever" in this context? 10 years? 100? 1000? It's a legit question to answer.
- kleer001 4y ago
  A real forever? Past the life of Sol and Earth. Unlikely. A more conservative forever... at the end of the human species? Maybe.
omoikane 4y ago
That statement of "before the demise of Google Books" seems unnecessary. The next quoted bit of "at least until Sunday" might have been an attempt to complete the joke, but should be interpreted as the number of books changing rapidly according to the (12 year old) linked article. http://booksearch.blogspot.com/2010/08/books-of-world-stand-up-and-be-counted.html http://booksearch.blogspot.com/2010/08/books-of-world-stand-...
tedivm 4y ago
The timing on this for me is really interesting, as last week I got an ISBN issues for a book I'm working on (9781633438002 if anyone is curious!). This will be the first book I'm the author of, but the second book I've worked on (the first I was the technical editor for). Neither of these books are out yet (I start writing tomorrow) but they both have ISBNs issued. Even if I never publish the book that ISBN is locked in. I imagine there's a lot of books that started out but never got finished. That said it looks like ISBNdb doesn't grab directly from the source, but instead crawls the internet looking for ISBN data to put into its database. I'll be interested to see at which stage my ISBN shows up in the database.
- delecti 4y ago
  What's the rationale behind reserving an ISBN before even beginning the writing process?
  - johannes1234321 4y ago
    Before writing might be a bit early, but before finishing and producing is useful as it can be listed early in catalogs for preorder. Getting the ISBN earlier probably is cheap and allows the publisher to use the ISBN as identifier for the whole project.
    - tedivm 4y ago
      I have no idea why they assigned it this early, but that seems as likely a reason I can think of. It was only assigned the day we finalized the contract, and there was a lot of work before that working the proposal through the system and getting reviews from the target audience and people familiar with the topic. It's only now that I'm expecting to hand content over on a schedule that they assigned the number.
  - lmm 4y ago
    It's a good unique key to use for tracking the book, even internally. You might change the title of the book at a late stage. You could use your own ID scheme, but what if your publisher merges with another while the book production is in process?
ZeroGravitas 4y ago
> extracting ISBNs from the actual book scans themselves (in the case of Z-Library/Libgen). OpenLibrary also uses book scans in Archive.org to extract ISBNs (and a few other bits of metadata, like urls in the text): https://blog.openlibrary.org/2021/08/23/gsoc-2021-making-books-lendable/ https://blog.openlibrary.org/2021/08/23/gsoc-2021-making-boo... And have a software pipeline for that kind of thing available.
- pilimi_anna 4y ago
  There's probably a lot of things that Open Library does that we can try to apply to shadow libraries!
Archelaos 4y ago
The problem with counting "books" is that the term is used in so many different ways, that one might end up with estimates that differ by several magnitudes depending how narrow or wide a definition or charactierization one adopts. How many books is a bible? One or around 80.[1] When there is a new minor edition, do we count no, one or 80 new books? Some of this 80 "books" are only letters and less than a page or only a few pages long. Shall we count them all as "books"? If we do so, should we than count each letter of a modern published correspondence as a single "book"? Poems were often published as very small booklets, but for prominent writers you may be able to purchase their "complete works" in a single more or less thick volumn, or the very same text in one thick volumn or a few more handy volumns. How should we count this? > Physical copies. Obviously this is not very helpful, since they’re just duplicates of the same material. Alas, this is quite often not the case, in particular for older books for various reasons, for example copies were bound from sheets of different print runs that used freshly assembled typesettings containing accidential or deliberate variations, sometimes sheets were missing or the order of pages is not correct, etc., etc.[2] For important "books" we should therefore digitize every available copy. As great it would be to have 129,864,880 "books" scanned, this would be just an initial phase. We would need a quality control: Is the resolution of the scans really always sufficient? Are the colours correctly represented (includes every scan a standard colour chart for comparison)? What about watermarks (they are extremly important for dating old books)? ... ... Besides, I personally prefer to speak of "making books digitally available" rather than of "preserving" them, because many features of a physical copy are impossible to preserve digitally: chemical coposition, (bio-)chemical traces, the DNA of parchment or animal bindings, their texture, how it feels to handle them, their visual appearance under different illuminations ... ... [1] The number varies from denomination to denomination. [2] And even renowned contemporary publishers sometimes silently correct errors without changing the numbering of the edition.
photochemsyn 4y ago
There are some interesting technologies in the pipeline for truly long-term data storage. Synthetic diamond is one option (light-sensitive, so perhaps susceptible to cosmic-ray degradation over time): https://theconversation.com/turning-diamonds-defects-into-long-term-3-d-data-storage-67685 https://theconversation.com/turning-diamonds-defects-into-lo... Another is microetching, i.e. ion-beam insertion of foreign atoms into crystalline materials, such as diamond or nickel, although the data density is lower than the above approach, it seems a lot less sensitive (i.e. light should have less effect): https://en.wikipedia.org/wiki/HD-Rosetta https://en.wikipedia.org/wiki/HD-Rosetta
Tomte 4y ago
ISBNs are supposed to be unique, but they aren't. Publishers reuse ISBNs, by mistake if they are following the rules, or sometimes intentionally. It's not super common, but common enough that I ran across that problem when scanning in my bookshelf years ago.

174 points by pilimi_anna 4y ago

As someone who would like to publish, my main concern with ISBN's is the cost, because publishers are required to assign an ISBN to every item in their catalog. Section 6.1 of the ISBN International User Manual "A separate ISBN shall be assigned to each separate monographic publication or separate edition or format of a monographic publication issued by a publisher." This would not be a problem if the numbers were more affordable.

pilimi_anna 4y ago
This is interesting to learn about. How expensive is it, and do you know if it differs around the world (since there are lots of national ISBN agencies)?
- jkingsman 4y ago
  It's $125 list price for a single ISBN, but there are bulk discounts buying direct and purchasing through a large scale supplier can make them as cheap as $10 each. There are deals to be had; Amazon, for example, may give you a free ISBN for your ebook as long as you publish it using KDP, their walled-garden publishing system, but the gotcha is the ISBN is not portable/you're not permitted to use it for other editions outside of the Amazon system. The other downside to these free (just about always) and discounted (sometimes) ISBNs is that they link the publisher as the service you got the ISBN through, rather than yourself, even if you're doing what would classically be considered a self-publishing job. How big of an issue is that? IANAExpert, but it seems like there are some nooks and crannies of IP law that can be swayed by owning the imprint, but little practical concern for the average person putting an ebook on Amazon e.g. Perhaps someone with more in depth publishing knowledge can color the risks better than I.
  - pilimi_anna 4y ago
    $125 is steep, but $10 sounds very doable (for the US). Good point on the "publisher" linking caveat though. I don't know much that matters in this day and age? Would be useful to learn from some published authors.
  - toomuchtodo 4y ago
    Can I just start a non profit “publisher”, buy a block of ISBNs, and hand them out at cost? Costco for ISBNs sort of thing, just enough margin to pay for a few hours a year of my time and a little app to drive the process. Edit: HN throttling, can’t reply. What if the Internet Archive gets a block and hands them out via Open Library? They seem positioned to argue they’re a bonafide publisher.
    - Finnucane 4y ago
      "There are unauthorized re-sellers of ISBNs and this activity is a violation of the ISBN standard and of industry practice. A publisher with one of these re-assigned ISBNs will not be correctly identified as the publisher of record in Books In Print or any of the industry databases such as Barnes and Noble or Amazon or those of wholesalers such as Ingram."
  - Finnucane 4y ago
    >Amazon, for example, may give you a free ISBN for your ebook as long as you publish it using KDP, That is indeed a bit of a ruse, since an ISBN is supposed to identify an edition or format, but not the sales channel. We give our epub files an ISBN, and all the vendors that sell that file (including Amazon) use the same number. But when you publish with KDP, you are not the publisher. Amazon is, so you have less say in the matter.
- wrs 4y ago
  For the US, between $125 each (for 1) and $1.50 each (for 1000) from the official source, a company called Bowker. The structure is described in Bowker’s FAQ [0]. [0] http://isbn.org/faqs_general_questions#isbn_faq6 http://isbn.org/faqs_general_questions#isbn_faq6
  - fragmede 4y ago
    In the middle is 10 for $295, which is $30 per. That's a small enough number that you can split the cost with a few friends. But you can also get one from Australia, $88 for 10 or $44 for one.
- jwilk 4y ago
  In Poland, you get ISBNs for free.
gigel82 4y ago
That's surprising; I wanted to make a picture book (with a bit of text the kids wrote) to send to grandparents and stumbled upon BookWright; seemed an affordable choice but was very surprised they actually included an ISBN with the little one-off kids picture book. Maybe they're just sitting on a big block of numbers and just giving them away...

mechanical_bear 4y ago

Forever? 0.

bloak 4y ago

Google has claimed that about 130 million books have been published (that factoid is all over the web). The number of 10-digit ISBNs is 1000 million (there's a check digit) and people have only just started using 13-digit ISBNs that start with 979 instead of 978; but of course there must be lots of wasted ISBNs, for example when a publisher optimistically buys a big block and then goes bankrupt. Both those numbers suggest that the "ISBNdb" with less than 31 million ISBNs is far from complete. The frequency of each top-level prefix (which tells you the geographical or language region) would be interesting. That would the first thing I'd calculate if I had the data on my disc.

eCa 4y ago
Yeah, lots and lots of unused ISBNs. As an example, O’reilly has the 978-0-596 series. That’s a hundred thousand editions.
adolph 4y ago
An ISBN is assigned to each separate edition and variation (except reprintings) of a publication. For example, an e-book, a paperback and a hardcover edition of the same book will each have a different ISBN. Additionally, there is address fragmentation; ISMB has blocks: ISBN issuance is country-specific, in that ISBNs are issued by the ISBN registration agency that is responsible for that country or territory regardless of the publication language. The ranges of ISBNs assigned to any particular country are based on the publishing profile of the country concerned, and so the ranges will vary depending on the number of books and the number, type, and size of publishers that are active. Some ISBN registration agencies are based in national libraries or within ministries of culture and thus may receive direct funding from the government to support their services. In other cases, the ISBN registration service is provided by organisations such as bibliographic data providers that are not government funded. https://en.wikipedia.org/wiki/ISBN https://en.wikipedia.org/wiki/ISBN
ComputerGuru 4y ago
I always loved how despite the massive domain differences, the ISBN situation is extremely similar to the IPv4/IPv6 situation (except more aggressively rent-seeking), with prefixes leased out to the old dogs, concerns about eventual address/isbn exhaustion, a scheme for mapping old ISB10 to new ISBN13 codes, etc etc.

pugworthy 4y ago

Define "forever" in this context? 10 years? 100? 1000? It's a legit question to answer.

kleer001 4y ago
A real forever? Past the life of Sol and Earth. Unlikely. A more conservative forever... at the end of the human species? Maybe.

omoikane 4y ago

That statement of "before the demise of Google Books" seems unnecessary. The next quoted bit of "at least until Sunday" might have been an attempt to complete the joke, but should be interpreted as the number of books changing rapidly according to the (12 year old) linked article. http://booksearch.blogspot.com/2010/08/books-of-world-stand-up-and-be-counted.html http://booksearch.blogspot.com/2010/08/books-of-world-stand-...

tedivm 4y ago

The timing on this for me is really interesting, as last week I got an ISBN issues for a book I'm working on (9781633438002 if anyone is curious!). This will be the first book I'm the author of, but the second book I've worked on (the first I was the technical editor for). Neither of these books are out yet (I start writing tomorrow) but they both have ISBNs issued. Even if I never publish the book that ISBN is locked in. I imagine there's a lot of books that started out but never got finished. That said it looks like ISBNdb doesn't grab directly from the source, but instead crawls the internet looking for ISBN data to put into its database. I'll be interested to see at which stage my ISBN shows up in the database.

delecti 4y ago
What's the rationale behind reserving an ISBN before even beginning the writing process?
- johannes1234321 4y ago
  Before writing might be a bit early, but before finishing and producing is useful as it can be listed early in catalogs for preorder. Getting the ISBN earlier probably is cheap and allows the publisher to use the ISBN as identifier for the whole project.
  - tedivm 4y ago
    I have no idea why they assigned it this early, but that seems as likely a reason I can think of. It was only assigned the day we finalized the contract, and there was a lot of work before that working the proposal through the system and getting reviews from the target audience and people familiar with the topic. It's only now that I'm expecting to hand content over on a schedule that they assigned the number.
- lmm 4y ago
  It's a good unique key to use for tracking the book, even internally. You might change the title of the book at a late stage. You could use your own ID scheme, but what if your publisher merges with another while the book production is in process?

ZeroGravitas 4y ago

> extracting ISBNs from the actual book scans themselves (in the case of Z-Library/Libgen). OpenLibrary also uses book scans in Archive.org to extract ISBNs (and a few other bits of metadata, like urls in the text): https://blog.openlibrary.org/2021/08/23/gsoc-2021-making-books-lendable/ https://blog.openlibrary.org/2021/08/23/gsoc-2021-making-boo... And have a software pipeline for that kind of thing available.

pilimi_anna 4y ago
There's probably a lot of things that Open Library does that we can try to apply to shadow libraries!

Archelaos 4y ago

The problem with counting "books" is that the term is used in so many different ways, that one might end up with estimates that differ by several magnitudes depending how narrow or wide a definition or charactierization one adopts. How many books is a bible? One or around 80.[1] When there is a new minor edition, do we count no, one or 80 new books? Some of this 80 "books" are only letters and less than a page or only a few pages long. Shall we count them all as "books"? If we do so, should we than count each letter of a modern published correspondence as a single "book"? Poems were often published as very small booklets, but for prominent writers you may be able to purchase their "complete works" in a single more or less thick volumn, or the very same text in one thick volumn or a few more handy volumns. How should we count this? > Physical copies. Obviously this is not very helpful, since they’re just duplicates of the same material. Alas, this is quite often not the case, in particular for older books for various reasons, for example copies were bound from sheets of different print runs that used freshly assembled typesettings containing accidential or deliberate variations, sometimes sheets were missing or the order of pages is not correct, etc., etc.[2] For important "books" we should therefore digitize every available copy. As great it would be to have 129,864,880 "books" scanned, this would be just an initial phase. We would need a quality control: Is the resolution of the scans really always sufficient? Are the colours correctly represented (includes every scan a standard colour chart for comparison)? What about watermarks (they are extremly important for dating old books)? ... ... Besides, I personally prefer to speak of "making books digitally available" rather than of "preserving" them, because many features of a physical copy are impossible to preserve digitally: chemical coposition, (bio-)chemical traces, the DNA of parchment or animal bindings, their texture, how it feels to handle them, their visual appearance under different illuminations ... ... [1] The number varies from denomination to denomination. [2] And even renowned contemporary publishers sometimes silently correct errors without changing the numbering of the edition.

photochemsyn 4y ago

There are some interesting technologies in the pipeline for truly long-term data storage. Synthetic diamond is one option (light-sensitive, so perhaps susceptible to cosmic-ray degradation over time): https://theconversation.com/turning-diamonds-defects-into-long-term-3-d-data-storage-67685 https://theconversation.com/turning-diamonds-defects-into-lo... Another is microetching, i.e. ion-beam insertion of foreign atoms into crystalline materials, such as diamond or nickel, although the data density is lower than the above approach, it seems a lot less sensitive (i.e. light should have less effect): https://en.wikipedia.org/wiki/HD-Rosetta https://en.wikipedia.org/wiki/HD-Rosetta

Tomte 4y ago

ISBNs are supposed to be unique, but they aren't. Publishers reuse ISBNs, by mistake if they are following the rules, or sometimes intentionally. It's not super common, but common enough that I ran across that problem when scanning in my bookshelf years ago.