3 ms·
> Would an LLM-driven "Personal Library" require manually annotated textual interpretation of each curated item, or could it derive personal interpretations fro
by _bramses 2y ago
> Would an LLM-driven "Personal Library" require manually annotated textual interpretation of each curated item, or could it derive personal interpretations from user history and the uniqueness of curated items/sets?
I’ve personally found that tagging is less robust than LLM embeddings (mainly due to dimensionality), but human appended thoughts about a source — also embedded — serve even better as tags.
Example: “this is a quote about dinosaurs…”
(Old way of doing things) Tags: dinosaurs, jurassic, history
Query: “dinosaurs” > results = 1…
(New way of doing things) Embedded Quote: [0.182…]
User Added Thought: “this dinosaur reminds me of a time i went to six flags with my cousins and…”
Embedded User Added Thought: [0.284…]
Query: “dinosaurs” > results = 2 (indexes = sources, thoughts)
The "thoughts" index can do a second layer cosine similarity search and serve as a tag on its own to fetch similar concepts. Basically a tree search created by similarity from user input/feedback loops.