4 ms·

Show HN: I made it fast and easy to launch your own RAG-powered AI chatbots

2 points by carlos_marcial 11mo ago

carlos_marcial 11mo ago
I built the tech stack behind ChatRAG to handle the increasing number of clients I started getting about a year ago who needed Retrieval Augmented Generation (RAG) powered chatbots. After a lot of trial and error, I settled on this tech stack for ChatRAG: Frontend - Next.js 16 (App Router) Latest React framework with server components and streaming - React 19 + React Compiler: Automatic memoization, no more useMemo/useCallback hell - Zustand: Lightweight state management (3kb vs Redux bloat) - Tailwind CSS + Framer Motion: Styling + buttery animations - Embed a chat widget version of your RAG chatbot on any web page, apart from creating a ChatGPT or Claude looking web UI AI / LLM Layer - Vercel AI SDK 5 – Unified streaming interface for all providers - OpenRouter – Single API for Claude, GPT-4, DeepSeek, Gemini, etc. - MCP (Model Context Protocol) – Tool use and function calling across models RAG Pipeline - Text chunking → documents split for optimal retrieval - OpenAI embeddings (1536 dim vectors) – Semantic search representation - pgvector with HNSW indexes – Fast approximate nearest neighbor search directly in Postgres Database & Auth - Supabase (PostgreSQL) – Database, auth, realtime, storage in one - GitHub & Google OAuth via Supabase – Third party sign in providers managed by Supabase - Row Level Security – Multi-tenant data isolation at the DB level Multi-Modal Generation - Use Fal.ai or Replicate.ai API keys for generating image, video and 3D assets inside of your RAG chatbot Integrations - WhatsApp via Baileys – Chat with your RAG from WhatsApp - Stripe / Polar – Payments and subscriptions Infra - Fly.io / Koyeb – Edge deployment for WhatsApp workers - Vercel – Frontend hosting with edge functions My special sauce: pgvector HNSW indexes (m=64, ef_construction=200) give you sub-100ms semantic search without leaving Postgres. No Pinecone/Weaviate vendor lock-in. Single-tenant vs Multi-tenant RAG setups: Why not both? ChatRAG supports both deployment modes depending on your use case: Single-tenant - One knowledge base → many users - Ideal for celebrity/expert AI clones or brand-specific agents - e.g., "Tony Robbins AI chatbot" or "Deepak Chopra AI" - All users interact with the same dataset and the same personality layer Multi-tenant - Users have workspace/project isolation — each with its own knowledge base, project-based system prompt and settings - Perfect for SaaS products or platform builders that want to offer AI chatbots to their customers - Every customer gets private data and their own RAG My long term vision is to keep evolving ChatRAG so I can eventually release a fully open-source version for everyone to build with.
cluckindan 11mo ago
How does it handle context curation in long conversations? Does it detect user intent for each message to see if it can act purely on existing context, or if it needs to do another retrieval and/or purge/summarize previous discussion and context?
- carlos_marcial 11mo ago
  Great questions! Here's how ChatRAG handles context curation: 1. Intent Detection Per Message Every message is classified to decide the retrieval strategy: - conversational (greetings, small talk) → skip RAG entirely - document_search (explicit knowledge queries) → always retrieve - complex/exploratory → adaptive retrieval - tool_required → route to MCP tools instead This prevents unnecessary retrieval on follow-ups like "thanks" or "can you explain that differently?" 2. Adaptive Retrieval When retrieval is needed, the system adjusts based on query characteristics: - Similarity thresholds: 0.45 for general queries → 0.75 for exact data requests - Result limits: 15 for specific queries → 30 for broad exploration - Strategy: semantic-only vs. hybrid vs. temporal-boosted "What was Q3 2024 revenue?" triggers exact_match mode at 0.75 threshold, while "Tell me about the company" uses standard semantic search at 0.45. 3. Conversation History Full conversation history is passed to the model without truncation—modern LLMs have 100K-200K token windows, and RAG context is freshly curated each turn rather than accumulated. 4. Not Implemented (Yet) - Conversation summarization/compression - Explicit context window management - "Forget previous context" mechanisms These would matter at scale or for very long sessions. That said, ChatRAG is a fully customizable boilerplate, and the idea is for developers to keep building on top of it, as I will too.
  - cluckindan 11mo ago
    Did you just copy-paste this answer from an LLM? It sounds exactly like a generative model, and there are obvious issues with the response. For example, ”RAG context is freshly curated each turn rather than accumulated” is nonsensical: it implies that context is curated instead of replaced, but also that the previous context is forgotten and replaced, and neither means that full conversation history (including context) is passed without truncation. Also, no mention of tool-based retrieval. I have a hunch this is not a real product, just AI slop. Prove me wrong.

I built the tech stack behind ChatRAG to handle the increasing number of clients I started getting about a year ago who needed Retrieval Augmented Generation (RAG) powered chatbots. After a lot of trial and error, I settled on this tech stack for ChatRAG: Frontend - Next.js 16 (App Router) Latest React framework with server components and streaming - React 19 + React Compiler: Automatic memoization, no more useMemo/useCallback hell - Zustand: Lightweight state management (3kb vs Redux bloat) - Tailwind CSS + Framer Motion: Styling + buttery animations - Embed a chat widget version of your RAG chatbot on any web page, apart from creating a ChatGPT or Claude looking web UI AI / LLM Layer - Vercel AI SDK 5 – Unified streaming interface for all providers - OpenRouter – Single API for Claude, GPT-4, DeepSeek, Gemini, etc. - MCP (Model Context Protocol) – Tool use and function calling across models RAG Pipeline - Text chunking → documents split for optimal retrieval - OpenAI embeddings (1536 dim vectors) – Semantic search representation - pgvector with HNSW indexes – Fast approximate nearest neighbor search directly in Postgres Database & Auth - Supabase (PostgreSQL) – Database, auth, realtime, storage in one - GitHub & Google OAuth via Supabase – Third party sign in providers managed by Supabase - Row Level Security – Multi-tenant data isolation at the DB level Multi-Modal Generation - Use Fal.ai or Replicate.ai API keys for generating image, video and 3D assets inside of your RAG chatbot Integrations - WhatsApp via Baileys – Chat with your RAG from WhatsApp - Stripe / Polar – Payments and subscriptions Infra - Fly.io / Koyeb – Edge deployment for WhatsApp workers - Vercel – Frontend hosting with edge functions My special sauce: pgvector HNSW indexes (m=64, ef_construction=200) give you sub-100ms semantic search without leaving Postgres. No Pinecone/Weaviate vendor lock-in. Single-tenant vs Multi-tenant RAG setups: Why not both? ChatRAG supports both deployment modes depending on your use case: Single-tenant - One knowledge base → many users - Ideal for celebrity/expert AI clones or brand-specific agents - e.g., "Tony Robbins AI chatbot" or "Deepak Chopra AI" - All users interact with the same dataset and the same personality layer Multi-tenant - Users have workspace/project isolation — each with its own knowledge base, project-based system prompt and settings - Perfect for SaaS products or platform builders that want to offer AI chatbots to their customers - Every customer gets private data and their own RAG My long term vision is to keep evolving ChatRAG so I can eventually release a fully open-source version for everyone to build with.

cluckindan 11mo ago

How does it handle context curation in long conversations? Does it detect user intent for each message to see if it can act purely on existing context, or if it needs to do another retrieval and/or purge/summarize previous discussion and context?

carlos_marcial 11mo ago
Great questions! Here's how ChatRAG handles context curation: 1. Intent Detection Per Message Every message is classified to decide the retrieval strategy: - conversational (greetings, small talk) → skip RAG entirely - document_search (explicit knowledge queries) → always retrieve - complex/exploratory → adaptive retrieval - tool_required → route to MCP tools instead This prevents unnecessary retrieval on follow-ups like "thanks" or "can you explain that differently?" 2. Adaptive Retrieval When retrieval is needed, the system adjusts based on query characteristics: - Similarity thresholds: 0.45 for general queries → 0.75 for exact data requests - Result limits: 15 for specific queries → 30 for broad exploration - Strategy: semantic-only vs. hybrid vs. temporal-boosted "What was Q3 2024 revenue?" triggers exact_match mode at 0.75 threshold, while "Tell me about the company" uses standard semantic search at 0.45. 3. Conversation History Full conversation history is passed to the model without truncation—modern LLMs have 100K-200K token windows, and RAG context is freshly curated each turn rather than accumulated. 4. Not Implemented (Yet) - Conversation summarization/compression - Explicit context window management - "Forget previous context" mechanisms These would matter at scale or for very long sessions. That said, ChatRAG is a fully customizable boilerplate, and the idea is for developers to keep building on top of it, as I will too.
- cluckindan 11mo ago
  Did you just copy-paste this answer from an LLM? It sounds exactly like a generative model, and there are obvious issues with the response. For example, ”RAG context is freshly curated each turn rather than accumulated” is nonsensical: it implies that context is curated instead of replaced, but also that the previous context is forgotten and replaced, and neither means that full conversation history (including context) is passed without truncation. Also, no mention of tool-based retrieval. I have a hunch this is not a real product, just AI slop. Prove me wrong.