3 ms·
So ... 30 years ago when we were starting Amazon (I was employee #2 there), I had an idea for a feature I wanted in the initial store that is related to what yo
by PaulDavisThe1st 3y ago
So ... 30 years ago when we were starting Amazon (I was employee #2 there), I had an idea for a feature I wanted in the initial store that is related to what you're trying to do here, but quite different.
While I value human curation, and it probably has to come first, what I wanted to do was to take the LoC classification for a book (typically a triple that has 3 levels of specificity) and build a multidimensional search "space" where you could find books close to the one you were looking at.
In 1994, alas, the LoC data was only available via an insanely slow modem-based download, which made it unusable. The main data source we were using back then (Baker & Taylor / Books in Print) sadly fucked up their record of the LoC classification to that it was impossible to infer where the gaps between the triple's parts were.
The crux of the idea was to leave book classification to librarians, who've been studying this for years and likely doing a much better job than anything us silly software types could come up with.
So to use an example from your list above, in my mental image of the interface I wanted: you'd pick "Sapiens" and see it surrounded by a cloud of other books sharing the same LoC triple (could use other library classification methods too). You could zoom out, and now see even more books sharing only the first two elements of the triple, or even further and end up in "History". Think of it as a "word cloud" made of books, rather than words. You could meander over to another book, and from that book zoom in (and out) of other aspects of the LoC's classification space.
I still wish I had been able to build this, since it mirrored the best part of my experience in large physical bookstores. But it was not to be.
Not sure if the dataset(s) you're using would permit this either, just wanted to mention it.
- bwb 3y agoYep, I am working on this, and it is a lot of how our system works under the surface. It just isn't accurate enough yet. I do use the LOC data, but only a very small % of books have that. I also use a NLP/ML tool called Wikifier that connects books to Wikidata topics as I want to use that data to help build knowledge graphs so readers can browse a timeline of WW2 and also see books on those subjects. What is the problem? With the topic system, it just isn't quite accurate enough; once I switch over to the newer AI systems later this year, it should work better. With the genre system, the big problem is that the BISAC publishing genre system is being abused by publishers. So, the data is not reliable because of abuse and ignorance. For example, I have thousands of books marked nonfiction and fiction because the publisher doesn't know how to use the system and picks nonfiction topics for a fiction book. Or, a publisher picks a book for a genre it doesn't belong in because they think it will get them more sales. It is really frustrating and I am spending a huge chunk of my time this year to look at how I can fix it. It probably means ignoring this system and using AI to classify it with other data we have.