7 ms·

A visual book recommender

410 points by squidhunter 3y ago

igaloly 3y ago
Nice! Is there a github repo?
qumpis 3y ago
This looks super cool, but why not use this tool as a non-visual tool to show similar books given a title? As far as I know there aren't many tools for this Also it would be super cool if we could import out goodreads reading lists and see them on the cluster
WaitWaitWha 3y ago
Have you presented this to LibraryThing?
FreeHugs 3y ago
Reminds me of https://www.literature-map.com https://www.literature-map.com Which is a map of all authors in the world sorted by overlap in readership. I found some of my favorite writers by browsing it. I wonder which approach is better suited to find something that is spot on to my interests. When I think of my favorite books, they usually are the most popular books of their authors. Are there any counterexamples, where an author wrote a book that is more profound than their biggest hit but got overlooked for some reason?
- xpil 3y ago
  Whoa, this is a great resource. Never heard of it before, it looks like I'm finally able to get some proper book recommendations. Absolutely loving it. Weird though that for certain cases it spells English author names with Cyrillic alphabet. Like for instance when I center the graph on "Stanisław Lem", I can see names like George Orwell or Terry Pratchett spelled in Cyrillic. I wonder why.
- lkbm 3y ago
  Oh man, Literature Map looks really great for finding recs. That said, I do think book-level might be much more valuable. My first thought for this was Night in the Lonesome October by Roger Zelazny. I haven't read anything else by him yet because my brother informs me his other stuff is entirely different. Looking at Goodreads, I think that qualifies as far from his biggest hit. Is it "more profound?"? Doubtful, but seems likely that you shouldn't group it with his others. I want recommendations based on the book I like, not the author I mostly might-not. A better example might be how Stephenie Meyer wrote the extremely popular Twilight books, and also The Host which is much less well-known, and better in many respects. Probably qualifies as more profound, too—it's told from the perspective of a parasitic alien. Picture the Yeerks from Animorphs if you read those.)
- Tyr42 3y ago
  I wish it was easier to see some of their books, or even copy and search the author from each page.
- aidenn0 3y ago
  > Are there any counterexamples, where an author wrote a book that is more profound than their biggest hit but got overlooked for some reason? Any reply is going to be somewhat subjective, but All Systems Red is not even in my top 5 books by Martha Wells...
- niccl 3y ago
  Stella Gibbons is basically only known for Cold Comfort Farm, but IMO many of her other things are better, although in a different way. My favourite is Pure Juliet, which is a beautifully gentle portrayal of someone who is on the Autism spectrum, written before Autism was really a thing that people talked about.
jvanderbot 3y ago
Id love to have https://same.energy https://same.energy for book contents.
- etra0 3y ago
  woah, didn't know about this site! it is super cool! It feels like what Pinterest would be without the annoying bits.
  - jvanderbot 3y ago
    Be careful, it'll become your defacto image search, and then you'll be really disappointed when you realize it hasn't been updated in ages
motoboi 3y ago
Really interesting that my favorite sci-fi book, Pandora's Star is in the middle of a void in the center of a large sci-fi cluster. It explains why I couldn't find anything like it. Also very explanatory the fact the Tolkien's The Two Towers is right by its side, because I also love that book. And now I'm already downloading the other "outliers" close to Pandora's Star.
- webmaven 3y ago
  Pandora's Star has a direct sequel: Judas Unchained, and Peter F. Hamilton has other books in that (The Commonwealth) millieu. Additional authors you might enjoy are Neal Asher, Iain M. Banks, Alistair Reynolds, and Jack McDevitt.
  - motoboi 3y ago
    Yeah, I've read the commonwealth. But nothing like the first book, although the two are actually a single book sold separately, at least for me. I just read a neighbor of Pandora's Star in the graph (Coyote) and enjoyed it. Note quite like pandora, but fine.
    - webmaven 3y ago
      Allen Steele's work isn't really all that similar, though I suppose Coyote and its sequels come closest. The authors I mentioned are much better bets.
nycdatasci 3y ago
Cool visualization, but the model needs work. The closest book to "Harry Potter and The Chamber of Secrets" is "The Victorian Lady's Guide to Sex, Marriage, and Manners".
- narcraft 3y ago
  Sounds about right to me!
- badcppdev 3y ago
  I have a theory (after having searched for The Diamond Age) that certain very popular books are ironically not going to be close to similar books because they appear on so many varied reading lists. There's probably a graph theory phenomena that describes what I'm thinking.
  - twosdai 3y ago
    Yeah basically what I think you're saying is that in a weighted graph, if there are edge weights which are orders of magnitude larger then the average it throws off certain models. Like nearest neighbor. Basically just prune the top and bottom %1 of weighted edges to get an appropriate average. Would be my guess for a fix.
  - bwb 3y ago
    Yep this is 100% true (i run shepherd.com). Within our data books like lord of the rings and harry potter are nearly impossible to map for "books like" because they are connected to so many other things. I am working now to fine tune our model, but it has been an interesting challenge.
  - aidenn0 3y ago
    IIRC you could do pretty okay in Netflix's recommendation contest by ignoring what an individual likes and just recommending the stuff that "everybody" likes.
- rossdavidh 3y ago
  Hypothesis: the sort of person who reads "The Victorian...", is likely to both like Harry Potter books, and also to review them enthusiastically online. The typical HP child reader, does not review books online. Just an hypothesis.
paweladamczuk 3y ago
This is really cool! I wish the accompanying article was longer. I can't fully grasp how it was done because I don't know enough about the concepts mentioned.
mhb 3y ago
I also like https://shepherd.com/ https://shepherd.com/. One of its interesting features is that authors list their five favorite books and say why they like them.
- pedrosbmartins 3y ago
  Which reminds me of https://fivebooks.com/ https://fivebooks.com/, where people from a particular field are asked their top five book recommendations for a given theme. The interview format is great, and I've picked up a few recommended books along the way.
  - scotty79 3y ago
    > The Best Apocalyptic Fiction, recommended by Elliot Ackerman > 1. The King James Bible
- cainxinth 3y ago
  Lol, I tried it out and put in one of my favorite books, "Liar's Poker" by Michael Lewis and it suggested me a list of "The best novels to help you understand the rich and dysfunctional." Spot on! Bookmarked!
- bwb 3y ago
  Thanks for the nice mention :) I am about a week away from launching genre pages, age pages, and filters for all those things. So on the hard-science-fiction page, you can filter to see books in a variety of fun ways and keep following your curiosity: Image showing how it works: https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc9275574-3e43-4fad-8241-48ff5eb8b41f_1138x868.png https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_pr... Hit me up at ben@shepherd.com if you want to try the early preview, just tell me your fav genre and ill send you a link.
Rediscover 3y ago
Nice article. Not what I was expecting... I was really hoping this would address "Visual Book" like in ふしぎの海のナディア, Fushigi no Umi no Nadia/The Secret of Blue Water. The LaserDiscs used to say "Visual Book" in every episode. Great animated series (to me), a mix of 20,000 Leagues Under the Sea and the Illuminati. Do NOT watch it with the english subs. Suffer with the Japanese, even if You don't speak it. You don't need the junk verbal translation, You will still get the main concepts. And Hanson's driving is so much more manaical in Japanese.
archydeb 3y ago
Love this. Bought a couple of books similar to "Midnight's Children' - author should definitely think about adding affiliate links!
bwb 3y ago
This is an awesome visualization, I am so impressed :). I've been working on (shepherd.com) with a similar goal to try to bring in human groupings to try to determine book connections via the many angles humans bring to the table. And more serendipity like wandering a local bookstore. I really love how you have done this. Dropping you an email in a few hours :)
tpttt 3y ago
Made an account just to comment on this. Look incredible.
Zufriedenheit 3y ago
Impressive Tool. I would love to have the same for movies.
eshnil 3y ago
> Only include reviews which came from users who had at least 10 reviews. Not sure if that's a good idea. It shrinks the set of genuine readers and overweights the set of professional spammers.
aroc 3y ago
This is great. I think visual tools like this are under utilized. They’re fun to use and can often reveal interesting insights.
stared 3y ago
For post-t-SNE processing to get non-overlapping items, see also: https://github.com/Quasimondo/RasterFairy https://github.com/Quasimondo/RasterFairy I also used more crude algorithms that sort by X, group elements in buckets, and within each, sort by Y. Then we get a grid of elements. The result is less high-quality than with iterative algorithms (and depends on if we sort by X or Y first), but it is hard to beat its simplicity.
gnewton77 3y ago
The visualizations remind me of those in a paper I co-authored a while ago (2009) visualizing ~2400 scientific journal / ~5.7M full-text articles: "Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library" https://nrc-publications.canada.ca/eng/view/accepted/?id=63e869d5-b589-456e-8459-33e1da3d35c8 https://nrc-publications.canada.ca/eng/view/accepted/?id=63e...
CSMastermind 3y ago
I'm surprised there are enough books published under the category of "Reverse-harem" to make it its own category along with things like "Horror", "Fantasy", "Business", etc.
scotty79 3y ago
Funny that most remote and isolated clusters ended up being m-m-romance and ... manga.
vrglvrglvrgl 3y ago
[dead]
thadk 3y ago
Anvaka's YASIV was an extremely strong tool in this space until Amazon discontinued the API it relied on. https://twitter.com/anvaka https://twitter.com/anvaka https://twitter.com/yasivcom https://twitter.com/yasivcom
- anvaka 3y ago
  Thank you. I agree :)
jasonshen 3y ago
Did I miss how we are supposed to get recommendations from what OP built?
- _dwt 3y ago
  Find something on the map that you know you like. Now look around it to see similar (i.e. if you like this, you're likely to like that) books.
- hombre_fatal 3y ago
  Use the magnifying glass on the visualization to type in a book.
internetter 3y ago
This is awesome. I only wish the author haden't waited years after scrapping! Many books I've loved have been released in the past couple years
camjohnson26 3y ago
Kind of hacky but I built something similar to apply the page rank algorithm to the authors referenced between books of various topics, here's the result for science: https://camjohnson26.github.io/author-graph/science/ https://camjohnson26.github.io/author-graph/science/ https://github.com/CamJohnson26/author-graph https://github.com/CamJohnson26/author-graph Clearly needs a lot of data clean up but still was very helpful for discovering important scientists and their approximate relative impact
skuxxlife 3y ago
This is super cool! I actually have been working on a new book recommendation site (https://braincandy.com https://braincandy.com) that has a similar-ish (but much smaller scale) visualization for book similarity. It is really interesting how certain genres tend to be much more insular than others and it can be a real challenge to break out of genre boxes when making recommendations. There's so many books out there on the edges and in-betweens that get lost when they don't fit neatly into an existing popular genre, and those indeed can be some of the most interesting.
butterNaN 3y ago
Something's wrong here. I was very excited to explore this, until I searched for "Getting Things Done" by David Allen. A nearby book - "Crippled America: How to Make America Great Again" by Donald J. Trump.
brubsby 3y ago
The gang of four design patterns book being two away from a MAGA book is funny.
mxfh 3y ago
similiar t-SNE visualisation, just for papers: https://static.nomic.ai/pubmed.html https://static.nomic.ai/pubmed.html running on their deepscatter visualization engine: https://github.com/nomic-ai/deepscatter https://github.com/nomic-ai/deepscatter that keeps things dynamic for rendering
crabkin 3y ago
I think that people in general don't read "enough", or maybe can't "enough." By that I mean, recommendations based on other people's reading habits aren't going to be as fine tuned as say, recommendations based on people's music listening habits.
larsiusprime 3y ago
Really cool! Reminds me a lot of some of the stuff I used to work on at Steam Labs: https://store.steampowered.com/labs https://store.steampowered.com/labs
ComposedPattern 3y ago
I'm astonished by how well this works. I looked for a bunch of books I'd read recently and kept finding other books I've read or want to read near them.
deleted 3y ago
[deleted]

410 points by squidhunter 3y ago

Nice! Is there a github repo?

This looks super cool, but why not use this tool as a non-visual tool to show similar books given a title? As far as I know there aren't many tools for this Also it would be super cool if we could import out goodreads reading lists and see them on the cluster

WaitWaitWha 3y ago

Have you presented this to LibraryThing?

FreeHugs 3y ago

Reminds me of https://www.literature-map.com https://www.literature-map.com Which is a map of all authors in the world sorted by overlap in readership. I found some of my favorite writers by browsing it. I wonder which approach is better suited to find something that is spot on to my interests. When I think of my favorite books, they usually are the most popular books of their authors. Are there any counterexamples, where an author wrote a book that is more profound than their biggest hit but got overlooked for some reason?

xpil 3y ago
Whoa, this is a great resource. Never heard of it before, it looks like I'm finally able to get some proper book recommendations. Absolutely loving it. Weird though that for certain cases it spells English author names with Cyrillic alphabet. Like for instance when I center the graph on "Stanisław Lem", I can see names like George Orwell or Terry Pratchett spelled in Cyrillic. I wonder why.
lkbm 3y ago
Oh man, Literature Map looks really great for finding recs. That said, I do think book-level might be much more valuable. My first thought for this was Night in the Lonesome October by Roger Zelazny. I haven't read anything else by him yet because my brother informs me his other stuff is entirely different. Looking at Goodreads, I think that qualifies as far from his biggest hit. Is it "more profound?"? Doubtful, but seems likely that you shouldn't group it with his others. I want recommendations based on the book I like, not the author I mostly might-not. A better example might be how Stephenie Meyer wrote the extremely popular Twilight books, and also The Host which is much less well-known, and better in many respects. Probably qualifies as more profound, too—it's told from the perspective of a parasitic alien. Picture the Yeerks from Animorphs if you read those.)
Tyr42 3y ago
I wish it was easier to see some of their books, or even copy and search the author from each page.
aidenn0 3y ago
> Are there any counterexamples, where an author wrote a book that is more profound than their biggest hit but got overlooked for some reason? Any reply is going to be somewhat subjective, but All Systems Red is not even in my top 5 books by Martha Wells...
niccl 3y ago
Stella Gibbons is basically only known for Cold Comfort Farm, but IMO many of her other things are better, although in a different way. My favourite is Pure Juliet, which is a beautifully gentle portrayal of someone who is on the Autism spectrum, written before Autism was really a thing that people talked about.

jvanderbot 3y ago

Id love to have https://same.energy https://same.energy for book contents.

etra0 3y ago
woah, didn't know about this site! it is super cool! It feels like what Pinterest would be without the annoying bits.
- jvanderbot 3y ago
  Be careful, it'll become your defacto image search, and then you'll be really disappointed when you realize it hasn't been updated in ages

motoboi 3y ago

Really interesting that my favorite sci-fi book, Pandora's Star is in the middle of a void in the center of a large sci-fi cluster. It explains why I couldn't find anything like it. Also very explanatory the fact the Tolkien's The Two Towers is right by its side, because I also love that book. And now I'm already downloading the other "outliers" close to Pandora's Star.

webmaven 3y ago
Pandora's Star has a direct sequel: Judas Unchained, and Peter F. Hamilton has other books in that (The Commonwealth) millieu. Additional authors you might enjoy are Neal Asher, Iain M. Banks, Alistair Reynolds, and Jack McDevitt.
- motoboi 3y ago
  Yeah, I've read the commonwealth. But nothing like the first book, although the two are actually a single book sold separately, at least for me. I just read a neighbor of Pandora's Star in the graph (Coyote) and enjoyed it. Note quite like pandora, but fine.
  - webmaven 3y ago
    Allen Steele's work isn't really all that similar, though I suppose Coyote and its sequels come closest. The authors I mentioned are much better bets.

nycdatasci 3y ago

Cool visualization, but the model needs work. The closest book to "Harry Potter and The Chamber of Secrets" is "The Victorian Lady's Guide to Sex, Marriage, and Manners".

narcraft 3y ago
Sounds about right to me!
badcppdev 3y ago
I have a theory (after having searched for The Diamond Age) that certain very popular books are ironically not going to be close to similar books because they appear on so many varied reading lists. There's probably a graph theory phenomena that describes what I'm thinking.
- twosdai 3y ago
  Yeah basically what I think you're saying is that in a weighted graph, if there are edge weights which are orders of magnitude larger then the average it throws off certain models. Like nearest neighbor. Basically just prune the top and bottom %1 of weighted edges to get an appropriate average. Would be my guess for a fix.
- bwb 3y ago
  Yep this is 100% true (i run shepherd.com). Within our data books like lord of the rings and harry potter are nearly impossible to map for "books like" because they are connected to so many other things. I am working now to fine tune our model, but it has been an interesting challenge.
- aidenn0 3y ago
  IIRC you could do pretty okay in Netflix's recommendation contest by ignoring what an individual likes and just recommending the stuff that "everybody" likes.
rossdavidh 3y ago
Hypothesis: the sort of person who reads "The Victorian...", is likely to both like Harry Potter books, and also to review them enthusiastically online. The typical HP child reader, does not review books online. Just an hypothesis.

paweladamczuk 3y ago

This is really cool! I wish the accompanying article was longer. I can't fully grasp how it was done because I don't know enough about the concepts mentioned.

mhb 3y ago

I also like https://shepherd.com/ https://shepherd.com/. One of its interesting features is that authors list their five favorite books and say why they like them.

pedrosbmartins 3y ago
Which reminds me of https://fivebooks.com/ https://fivebooks.com/, where people from a particular field are asked their top five book recommendations for a given theme. The interview format is great, and I've picked up a few recommended books along the way.
- scotty79 3y ago
  > The Best Apocalyptic Fiction, recommended by Elliot Ackerman > 1. The King James Bible
cainxinth 3y ago
Lol, I tried it out and put in one of my favorite books, "Liar's Poker" by Michael Lewis and it suggested me a list of "The best novels to help you understand the rich and dysfunctional." Spot on! Bookmarked!
bwb 3y ago
Thanks for the nice mention :) I am about a week away from launching genre pages, age pages, and filters for all those things. So on the hard-science-fiction page, you can filter to see books in a variety of fun ways and keep following your curiosity: Image showing how it works: https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc9275574-3e43-4fad-8241-48ff5eb8b41f_1138x868.png https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_pr... Hit me up at ben@shepherd.com if you want to try the early preview, just tell me your fav genre and ill send you a link.

Rediscover 3y ago

Nice article. Not what I was expecting... I was really hoping this would address "Visual Book" like in ふしぎの海のナディア, Fushigi no Umi no Nadia/The Secret of Blue Water. The LaserDiscs used to say "Visual Book" in every episode. Great animated series (to me), a mix of 20,000 Leagues Under the Sea and the Illuminati. Do NOT watch it with the english subs. Suffer with the Japanese, even if You don't speak it. You don't need the junk verbal translation, You will still get the main concepts. And Hanson's driving is so much more manaical in Japanese.

archydeb 3y ago

Love this. Bought a couple of books similar to "Midnight's Children' - author should definitely think about adding affiliate links!

bwb 3y ago

This is an awesome visualization, I am so impressed :). I've been working on (shepherd.com) with a similar goal to try to bring in human groupings to try to determine book connections via the many angles humans bring to the table. And more serendipity like wandering a local bookstore. I really love how you have done this. Dropping you an email in a few hours :)

tpttt 3y ago

Made an account just to comment on this. Look incredible.

Zufriedenheit 3y ago

Impressive Tool. I would love to have the same for movies.

eshnil 3y ago

> Only include reviews which came from users who had at least 10 reviews. Not sure if that's a good idea. It shrinks the set of genuine readers and overweights the set of professional spammers.

aroc 3y ago

This is great. I think visual tools like this are under utilized. They’re fun to use and can often reveal interesting insights.

stared 3y ago

For post-t-SNE processing to get non-overlapping items, see also: https://github.com/Quasimondo/RasterFairy https://github.com/Quasimondo/RasterFairy I also used more crude algorithms that sort by X, group elements in buckets, and within each, sort by Y. Then we get a grid of elements. The result is less high-quality than with iterative algorithms (and depends on if we sort by X or Y first), but it is hard to beat its simplicity.

gnewton77 3y ago

The visualizations remind me of those in a paper I co-authored a while ago (2009) visualizing ~2400 scientific journal / ~5.7M full-text articles: "Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library" https://nrc-publications.canada.ca/eng/view/accepted/?id=63e869d5-b589-456e-8459-33e1da3d35c8 https://nrc-publications.canada.ca/eng/view/accepted/?id=63e...

CSMastermind 3y ago

I'm surprised there are enough books published under the category of "Reverse-harem" to make it its own category along with things like "Horror", "Fantasy", "Business", etc.

scotty79 3y ago

Funny that most remote and isolated clusters ended up being m-m-romance and ... manga.

vrglvrglvrgl 3y ago

[dead]

thadk 3y ago

Anvaka's YASIV was an extremely strong tool in this space until Amazon discontinued the API it relied on. https://twitter.com/anvaka https://twitter.com/anvaka https://twitter.com/yasivcom https://twitter.com/yasivcom

anvaka 3y ago
Thank you. I agree :)

jasonshen 3y ago

Did I miss how we are supposed to get recommendations from what OP built?

_dwt 3y ago
Find something on the map that you know you like. Now look around it to see similar (i.e. if you like this, you're likely to like that) books.
hombre_fatal 3y ago
Use the magnifying glass on the visualization to type in a book.

internetter 3y ago

This is awesome. I only wish the author haden't waited years after scrapping! Many books I've loved have been released in the past couple years

camjohnson26 3y ago

Kind of hacky but I built something similar to apply the page rank algorithm to the authors referenced between books of various topics, here's the result for science: https://camjohnson26.github.io/author-graph/science/ https://camjohnson26.github.io/author-graph/science/ https://github.com/CamJohnson26/author-graph https://github.com/CamJohnson26/author-graph Clearly needs a lot of data clean up but still was very helpful for discovering important scientists and their approximate relative impact

skuxxlife 3y ago

This is super cool! I actually have been working on a new book recommendation site (https://braincandy.com https://braincandy.com) that has a similar-ish (but much smaller scale) visualization for book similarity. It is really interesting how certain genres tend to be much more insular than others and it can be a real challenge to break out of genre boxes when making recommendations. There's so many books out there on the edges and in-betweens that get lost when they don't fit neatly into an existing popular genre, and those indeed can be some of the most interesting.

butterNaN 3y ago

Something's wrong here. I was very excited to explore this, until I searched for "Getting Things Done" by David Allen. A nearby book - "Crippled America: How to Make America Great Again" by Donald J. Trump.

brubsby 3y ago

The gang of four design patterns book being two away from a MAGA book is funny.

mxfh 3y ago

similiar t-SNE visualisation, just for papers: https://static.nomic.ai/pubmed.html https://static.nomic.ai/pubmed.html running on their deepscatter visualization engine: https://github.com/nomic-ai/deepscatter https://github.com/nomic-ai/deepscatter that keeps things dynamic for rendering

crabkin 3y ago

I think that people in general don't read "enough", or maybe can't "enough." By that I mean, recommendations based on other people's reading habits aren't going to be as fine tuned as say, recommendations based on people's music listening habits.

larsiusprime 3y ago

Really cool! Reminds me a lot of some of the stuff I used to work on at Steam Labs: https://store.steampowered.com/labs https://store.steampowered.com/labs

ComposedPattern 3y ago

I'm astonished by how well this works. I looked for a bunch of books I'd read recently and kept finding other books I've read or want to read near them.

deleted 3y ago

[deleted]