I'm not well-versed in information theory, so this may be a misinterpretation. But this strikes me as a beautiful truth that rewards careful pondering.
A library containing all possible books may contain all the gems of knowledge in the world, but it also contains far more nonsense: Books full of random letters that mean nothing, as well as real books with changes made to them, both unnoticeable and noticeable. Such a library, though containing all information, would not contain any information about which books contain real information and which do not. A single real book, though, while it may contain factual errors, can at least be guaranteed to contain the beliefs and assumptions of its author, which is more than you can say about the infinite library.
Look at it this way: an infinite list of strings containing all possible strings (or even a finite list of strings containing all possible strings shorter than a given length) contains less information than a program or essay that I wrote, because the infinite list gives no indication about which strings are "important", "informational" or "useful", while my one essay or program contains concise, easily-extractable information intended for a particular purpose.
One more example, just for fun: Imagine I'm a stupid bank robber who keeps all the plans for my heists and the names of my partners in one Word document on my hard drive. If the police arrest me, they have enough information to send me and my partners to jail. However, imagine that I've written a program that makes a million copies of my secret plan and changes the names and locations inside all of the copies to random values. Now, if I get caught, without knowing which document is real and which are fake, the authorities don't have any useful information.
And it would also contain an infinite number of books which claim to list which books contain real information, but actually contain some titles that contain nonsense & lies and/or are missing some titles that contain real information. So the chance of you finding a fake information-listings book is far greater than you finding the real one.
Hmm, sorry. I thought you were referring to the book being difficult to find because it would be mixed in with an infinite number of other books, not specifically that there would be an infinite number of fake versions of that book.
A library containing all possible books may contain all the gems of knowledge in the world, but it also contains far more nonsense: Books full of random letters that mean nothing, as well as real books with changes made to them, both unnoticeable and noticeable. Such a library, though containing all information, would not contain any information about which books contain real information and which do not. A single real book, though, while it may contain factual errors, can at least be guaranteed to contain the beliefs and assumptions of its author, which is more than you can say about the infinite library.
Look at it this way: an infinite list of strings containing all possible strings (or even a finite list of strings containing all possible strings shorter than a given length) contains less information than a program or essay that I wrote, because the infinite list gives no indication about which strings are "important", "informational" or "useful", while my one essay or program contains concise, easily-extractable information intended for a particular purpose.
One more example, just for fun: Imagine I'm a stupid bank robber who keeps all the plans for my heists and the names of my partners in one Word document on my hard drive. If the police arrest me, they have enough information to send me and my partners to jail. However, imagine that I've written a program that makes a million copies of my secret plan and changes the names and locations inside all of the copies to random values. Now, if I get caught, without knowing which document is real and which are fake, the authorities don't have any useful information.