4 ms·

Funny this has come up – I've literally just finished building a RAG search with Postgres this morning. I run a directory site for UX articles and tools, which

Funny this has come up – I've literally just finished building a RAG search with Postgres this morning. I run a directory site for UX articles and tools, which I've recently rebuilt in Django. It has links to over 5000 articles, making it hard to browse, so I thought it'd be fun to use RAG with citations to create a knowledge search tool. The site fetches new articles via RSS, which are chunked, embedded and added to the vector store. On conducting a search, the site returns a summary as well as links to the original articles. I'm using LlamaIndex, OpenAI and Supabase. It's taken a little while to figure out as I really didn't know what I was doing and there's loads of improvements to make, but you can try it out here: https://www.uxlift.org/search/ https://www.uxlift.org/search/ I'd love to hear what you think.

rjrodger 3y ago
Any chance you'd Open Source it?
- mattkevan 3y ago
  Yes, I'm planning to write an article showing the code, mostly so I don't forget how I did it but hopefully it'll also be useful to others. I relied on a lot of articles and code examples when putting it together so I'm happy to do the same.
  - WesleyJohnson 3y ago
    Looking forward to this. We're running a Django codebase and would like to pull in Confluence articles and run GPT queries against them.
gcanyon 3y ago
This is awesome! I just tried it out, and I have a few bits of feedback: Show the question I asked. I asked something like "What are the latest trends in UX for mobile?" and got a good answer that started with "The latest trends in UX design for mobile include..." but that's not the same as just listing the exact thing I asked. Either show a chat log or stuff it into the browser history or...something. There's no way to see the answer to the question I asked before this one. I just went back and did another search, and the results had 6 links, while the response (only) had cites labeled [7] [6] and [8] in that order. Again, great stuff!
- mattkevan 3y ago
  Hey thanks for trying it out and giving your feedback, much appreciated! Good idea about showing the question, I've just added that to the search result. Yes, I agree a chat history is really needed - it's on the list, along with a chat-style interface. Also need to figure out what's going on with the citations in the text - the numbers seem pretty random and don't link up to the cited articles. Thanks again!
netdur 3y ago
Could you share more about your strategy or the approach you took for chunking the articles? I'm curious about the criteria or methods you used to decide the boundaries of each chunk and how you ensured the chunks remained meaningful for the search functionality. Thanks!
- mattkevan 3y ago
  As this was my first attempt, I decided to take a pretty basic approach, see what the results were like and optimise it later. Content is stored in Django as posts, so I wrote a custom document reader that created a new LlamaIndex document for each post, attaching the post id, title, link and published date as metadata. This gave better results than just loading in all the content as a text or CSV file, which I tried first. I did try with a bunch of different techniques to split the chunks, including by sentence count and a larger and smaller number of tokens. In the end I decided to leave it to the LlamaIndex default just to get it working.
- drittich 3y ago
  I wrote a C# library to do this, which is similar to other chunking approaches that are common, like the way langchain does it: https://github.com/drittich/SemanticSlicer https://github.com/drittich/SemanticSlicer Given a list of separators (regexes), it goes through them in order and keeps splitting the text by them until the chunk fits within the desired size. By putting the higher level separators first (e.g., for HTML split by <h1> before <h2>), it's a pretty good proxy for maintaining context. Which chunk size you decide on largely depends on your data, so I typically eyeball a sample of the results to determine if the splitting is satisfactory. You can see the separators here: https://github.com/drittich/SemanticSlicer/blob/main/SemanticSlicer/Separators.cs https://github.com/drittich/SemanticSlicer/blob/main/Semanti...

rjrodger 3y ago

Any chance you'd Open Source it?

mattkevan 3y ago
Yes, I'm planning to write an article showing the code, mostly so I don't forget how I did it but hopefully it'll also be useful to others. I relied on a lot of articles and code examples when putting it together so I'm happy to do the same.
- WesleyJohnson 3y ago
  Looking forward to this. We're running a Django codebase and would like to pull in Confluence articles and run GPT queries against them.

gcanyon 3y ago

This is awesome! I just tried it out, and I have a few bits of feedback: Show the question I asked. I asked something like "What are the latest trends in UX for mobile?" and got a good answer that started with "The latest trends in UX design for mobile include..." but that's not the same as just listing the exact thing I asked. Either show a chat log or stuff it into the browser history or...something. There's no way to see the answer to the question I asked before this one. I just went back and did another search, and the results had 6 links, while the response (only) had cites labeled [7] [6] and [8] in that order. Again, great stuff!

mattkevan 3y ago
Hey thanks for trying it out and giving your feedback, much appreciated! Good idea about showing the question, I've just added that to the search result. Yes, I agree a chat history is really needed - it's on the list, along with a chat-style interface. Also need to figure out what's going on with the citations in the text - the numbers seem pretty random and don't link up to the cited articles. Thanks again!

netdur 3y ago

Could you share more about your strategy or the approach you took for chunking the articles? I'm curious about the criteria or methods you used to decide the boundaries of each chunk and how you ensured the chunks remained meaningful for the search functionality. Thanks!

mattkevan 3y ago
As this was my first attempt, I decided to take a pretty basic approach, see what the results were like and optimise it later. Content is stored in Django as posts, so I wrote a custom document reader that created a new LlamaIndex document for each post, attaching the post id, title, link and published date as metadata. This gave better results than just loading in all the content as a text or CSV file, which I tried first. I did try with a bunch of different techniques to split the chunks, including by sentence count and a larger and smaller number of tokens. In the end I decided to leave it to the LlamaIndex default just to get it working.
drittich 3y ago
I wrote a C# library to do this, which is similar to other chunking approaches that are common, like the way langchain does it: https://github.com/drittich/SemanticSlicer https://github.com/drittich/SemanticSlicer Given a list of separators (regexes), it goes through them in order and keeps splitting the text by them until the chunk fits within the desired size. By putting the higher level separators first (e.g., for HTML split by <h1> before <h2>), it's a pretty good proxy for maintaining context. Which chunk size you decide on largely depends on your data, so I typically eyeball a sample of the results to determine if the splitting is satisfactory. You can see the separators here: https://github.com/drittich/SemanticSlicer/blob/main/SemanticSlicer/Separators.cs https://github.com/drittich/SemanticSlicer/blob/main/Semanti...