7 ms·

I don't understand why so many people seem so fascinated by constructions like the library of Babel. Yes it contains the answers to all your questions, but the

I don't understand why so many people seem so fascinated by constructions like the library of Babel. Yes it contains the answers to all your questions, but there are some significant drawbacks. * It has more wrong information than right information, with no way to tell the difference. * If you had an oracle that could tell you how to get to the book you need, the navigation instructions to get to the book will be at least as long as the book, on average.

Llamamoe 1y ago
I wonder if there is some way to create a latent-space Library of Babel in which you only find incoherent gibberish with extremely long keys, with the shortest ones pointing specifically to the most common/likely strings of text, in manageable computational complexity.
- recursive 1y ago
  Reproducing the text of a book in the library is a synonym for identifying the book. So this is really called "text compression", which is a well-studied field.
- samsartor 1y ago
  In a library of all possible strings, this is just text compression (as the other comment observes). But in a finite library it gets even simpler, in a cool way! We can treat each text as a unique symbol and use an entropy encoding (eg Huffman) to assign length-optimized key to each based on likelihood (eg from an LLM). Building the library is something like O(n log n), which isn't terrible. But adding new texts would change the IDs for existing texts (which is annoying). There might be a good way to reserve space for future entries probabilistically? Out of my depth at this point!
- lxgr 1y ago
  That's arguably just a regular library :)
cryzinger 1y ago
To your first bullet, I believe this is one of the central points of the original Borges story :)
- cantor_S_drug 1y ago
  I think Library of Babel by Borges is a static manifestation of Turing complete behaviour via the fact that some L-systems are Turing complete. or put another way. Where in the Library of Babel, does the real Hamlet reside? If we consider finding and replacing names with other names, is it still a Hamlet? And if we bring the full force of edit operations and do these in a reversible manner, then where does the actual Hamlet reside? An equivalence class of Hamlet?
a_shovel 1y ago
Another way of looking at it is that the library of Babel would be less useful than an equivalent quantity of blank paper. For example, you could use it to print books in English instead of gibberish. Multiple copies of those books, even.
bonoboTP 1y ago
The Library of Babel made me aware that choosing/finding is not super distinct from making/creating. Or discovery and invention. In math, there is distinction between "there exists" and "we can construct", but "we can construct" is similar to "we can find".
- matheusmoreira 1y ago
  I don't think they're equivalent. I think invention and creation aren't actually real. There is no "making" or "creating" when it comes to intellectual work. All computer files are sequences of bits. All sequences of bits are integers. All integers already exist in the infinite set of natural numbers. I can even calculate how big those numbers are given their bit count. digits(bits) = ceil(bits * log10(2)) digits(32) = 10 digits(64) = 20 digits(128) = 39 digits(256) = 78 digits(512) = 155 digits(1024) = 309 digits(20 KiB) = 49,321 digits(2 GiB) = 5,171,655,946 We are merely discovering numbers through convoluted mental and technological processes. All our mental exertions result in the discovery of a number. This comment is a number.
  - synctext 1y ago
    How to find a nice SHA1 hash? How do keyword search in this list? Search and discovery of quality are unsolved scientific challenges. Fascinating stuff. At our university lab we've been working on this for 25 years. Building a search engine is the easy part. Keeping a federated server with a billion users running is unsolved. Creating a fully -serverless- decentralised search engine is possible, you also need self-funding economy. Seems we're one of the few labs worldwide to still make actual operational prototypes of this stuff. More shameless self promotion: "SwarmSearch: Decentralized Search Engine with Self-Funding Economy" [0] Really handy to have s search engine to search this webpage with 45,671,926,166,590,716,193,865,151,022,383,844,364,247,891,968 pages and the rest of the web (no spyware, no tracking). [0] https://arxiv.org/abs/2505.07452 https://arxiv.org/abs/2505.07452
    - lurk2 1y ago
      If you’re interested in mass market adoption rather than just proving the theory, you will need to change the name. “LimeWire” is fun. “SwarmSearch” sounds like a biblical plague.
deleted 1y ago
[deleted]
AnthonyMouse 1y ago
> If you had an oracle that could tell you how to get to the book you need, the navigation instructions to get to the book will be at least as long as the book, on average. This isn't quite true. Natural language text compresses extremely well and you would only need length equivalent to the compressed form, not the original form. And if you wanted to go further, you could use a mapping where extremely short strings map to known popular books and only unknown works have longer encodings.
- recursive 1y ago
  I suppose this would work if the library was arranged such that comprehensible books were closer to the "origin". The workings of the "real" library of babel are supposed to be more inscrutable though. But if I built one, it would totally work that way.
- variadix 1y ago
  Kolmogorov’s library
0cf8612b2e1e 1y ago
I am reminded of this SMBC comic https://www.smbc-comics.com/comic/the-library-of-heaven https://www.smbc-comics.com/comic/the-library-of-heaven
megablast 1y ago
Thank you captain obvious.
- recursive 1y ago
  At your service.
Chinjut 1y ago
Everyone is aware of this. Sites like this aren't created to be useful. They are created to be an amusement, a joke.
kristianp 1y ago
> If you had an oracle that could tell you how to get to the book you need, the navigation instructions to get to the book will be at least as long as the book, on average. Only if the oracle has all books that could possibly exist. If you're trying to find a book that already exists, that set is infinitely smaller.
- recursive 1y ago
  The oracle doesn't have the books. The library does. And it has all of them. Directions to each book depend only on the layout and contents of the library.

Llamamoe 1y ago

I wonder if there is some way to create a latent-space Library of Babel in which you only find incoherent gibberish with extremely long keys, with the shortest ones pointing specifically to the most common/likely strings of text, in manageable computational complexity.

recursive 1y ago
Reproducing the text of a book in the library is a synonym for identifying the book. So this is really called "text compression", which is a well-studied field.
samsartor 1y ago
In a library of all possible strings, this is just text compression (as the other comment observes). But in a finite library it gets even simpler, in a cool way! We can treat each text as a unique symbol and use an entropy encoding (eg Huffman) to assign length-optimized key to each based on likelihood (eg from an LLM). Building the library is something like O(n log n), which isn't terrible. But adding new texts would change the IDs for existing texts (which is annoying). There might be a good way to reserve space for future entries probabilistically? Out of my depth at this point!
lxgr 1y ago
That's arguably just a regular library :)

cryzinger 1y ago

To your first bullet, I believe this is one of the central points of the original Borges story :)

cantor_S_drug 1y ago
I think Library of Babel by Borges is a static manifestation of Turing complete behaviour via the fact that some L-systems are Turing complete. or put another way. Where in the Library of Babel, does the real Hamlet reside? If we consider finding and replacing names with other names, is it still a Hamlet? And if we bring the full force of edit operations and do these in a reversible manner, then where does the actual Hamlet reside? An equivalence class of Hamlet?

a_shovel 1y ago

Another way of looking at it is that the library of Babel would be less useful than an equivalent quantity of blank paper. For example, you could use it to print books in English instead of gibberish. Multiple copies of those books, even.

bonoboTP 1y ago

The Library of Babel made me aware that choosing/finding is not super distinct from making/creating. Or discovery and invention. In math, there is distinction between "there exists" and "we can construct", but "we can construct" is similar to "we can find".

matheusmoreira 1y ago
I don't think they're equivalent. I think invention and creation aren't actually real. There is no "making" or "creating" when it comes to intellectual work. All computer files are sequences of bits. All sequences of bits are integers. All integers already exist in the infinite set of natural numbers. I can even calculate how big those numbers are given their bit count. digits(bits) = ceil(bits * log10(2)) digits(32) = 10 digits(64) = 20 digits(128) = 39 digits(256) = 78 digits(512) = 155 digits(1024) = 309 digits(20 KiB) = 49,321 digits(2 GiB) = 5,171,655,946 We are merely discovering numbers through convoluted mental and technological processes. All our mental exertions result in the discovery of a number. This comment is a number.
- synctext 1y ago
  How to find a nice SHA1 hash? How do keyword search in this list? Search and discovery of quality are unsolved scientific challenges. Fascinating stuff. At our university lab we've been working on this for 25 years. Building a search engine is the easy part. Keeping a federated server with a billion users running is unsolved. Creating a fully -serverless- decentralised search engine is possible, you also need self-funding economy. Seems we're one of the few labs worldwide to still make actual operational prototypes of this stuff. More shameless self promotion: "SwarmSearch: Decentralized Search Engine with Self-Funding Economy" [0] Really handy to have s search engine to search this webpage with 45,671,926,166,590,716,193,865,151,022,383,844,364,247,891,968 pages and the rest of the web (no spyware, no tracking). [0] https://arxiv.org/abs/2505.07452 https://arxiv.org/abs/2505.07452
  - lurk2 1y ago
    If you’re interested in mass market adoption rather than just proving the theory, you will need to change the name. “LimeWire” is fun. “SwarmSearch” sounds like a biblical plague.

deleted 1y ago

[deleted]

AnthonyMouse 1y ago

> If you had an oracle that could tell you how to get to the book you need, the navigation instructions to get to the book will be at least as long as the book, on average. This isn't quite true. Natural language text compresses extremely well and you would only need length equivalent to the compressed form, not the original form. And if you wanted to go further, you could use a mapping where extremely short strings map to known popular books and only unknown works have longer encodings.

recursive 1y ago
I suppose this would work if the library was arranged such that comprehensible books were closer to the "origin". The workings of the "real" library of babel are supposed to be more inscrutable though. But if I built one, it would totally work that way.
variadix 1y ago
Kolmogorov’s library

0cf8612b2e1e 1y ago

I am reminded of this SMBC comic https://www.smbc-comics.com/comic/the-library-of-heaven https://www.smbc-comics.com/comic/the-library-of-heaven

megablast 1y ago

Thank you captain obvious.

recursive 1y ago
At your service.

Chinjut 1y ago

Everyone is aware of this. Sites like this aren't created to be useful. They are created to be an amusement, a joke.

kristianp 1y ago

> If you had an oracle that could tell you how to get to the book you need, the navigation instructions to get to the book will be at least as long as the book, on average. Only if the oracle has all books that could possibly exist. If you're trying to find a book that already exists, that set is infinitely smaller.

recursive 1y ago
The oracle doesn't have the books. The library does. And it has all of them. Directions to each book depend only on the layout and contents of the library.