From Zero to One: Building a RAG-Powered Smart Chat Application

With the rapid development of large AI models, building a practical chat application with advanced features has become a hot topic. This article shares the complete process of building a modern AI chat app from scratch, integrating RAG (Retrieval-Augmented Generation), multi-turn conversation management, and local vector storage.

Live Demo: https://chat.erishen.cn

Core Features

Intelligent Conversation

• Streaming responses: Real-time streaming via Vercel AI SDK
• Multi-turn dialogue: Complete conversation history and context retention
• Auto-titling: AI-generated conversation titles based on content
• Persistence: Conversation history survives page refreshes

RAG Document Retrieval

• Document upload: Supports TXT, MD formats
• Smart chunking: Auto-splits long documents into semantic chunks
• Client-side vectorization: Using @xenova/transformers for local embedding generation
• Semantic search: Cosine similarity-based intelligent document retrieval
• Context enhancement: Combines retrieved documents for more accurate responses

Modern UI/UX

• Responsive design: Adapts perfectly to desktop and mobile
• Theme switching: Light/dark/system themes
• Component architecture: Tailwind CSS-based design system
• Fluid animations: Polished interaction experience

Technical Architecture

Frontend Stack

Next.js 15 + React 19 + TypeScript, Tailwind CSS 4, Vercel AI SDK

RAG Architecture (Hybrid)

• Client side: Document processing + vectorization + local storage
• Server side: Semantic search + context generation

Core Libraries

@xenova/transformers (client ML), localStorage (vector DB), cosine similarity, React Hooks

Key Implementations

Multi-Turn Conversation Management

export function useMultiTurnChat() {
  const [currentConversationId, setCurrentConversationId] = useState<string | null>(null)
  const [conversations, setConversations] = useState<Conversation[]>([])

  useEffect(() => {
    if (!currentConversationId || messages.length === 0) return
    conversationManager.updateConversation(currentConversationId, {
      messages: messages.map(msg => ({
        id: msg.id,
        role: msg.role as 'user' | 'assistant',
        content: msg.content,
        timestamp: new Date(),
      }))
    })
  }, [messages, currentConversationId])

  return { messages, conversations, createNewConversation, switchConversation, deleteConversation }
}

RAG Document Processing Pipeline

class DocumentProcessor {
  async processDocument(file: File): Promise<ProcessedDocument> {
    const content = await this.readFileContent(file)
    const chunks = await this.chunkText(content, { chunkSize: 500, chunkOverlap: 50 })
    const embeddings = await this.generateEmbeddings(chunks)
    await vectorStore.addDocument({
      id: generateId(), title: file.name, content,
      chunks: chunks.map((chunk, index) => ({
        id: generateId(), content: chunk, embedding: embeddings[index]
      }))
    })
    return processedDocument
  }
}

Semantic Search with Cosine Similarity

class LocalStorageVectorStore {
  async search(queryEmbedding: number[], topK: number): Promise<SearchResult[]> {
    const allChunks = this.getAllChunks()
    const similarities = allChunks.map(chunk => ({
      ...chunk,
      similarity: this.cosineSimilarity(queryEmbedding, chunk.embedding)
    }))
    return similarities
      .sort((a, b) => b.similarity - a.similarity)
      .slice(0, topK)
      .filter(result => result.similarity > 0.5)
  }

  private cosineSimilarity(a: number[], b: number[]): number {
    const dotProduct = a.reduce((sum, val, i) => sum + val * b[i], 0)
    const magnitudeA = Math.sqrt(a.reduce((sum, val) => sum + val * val, 0))
    const magnitudeB = Math.sqrt(b.reduce((sum, val) => sum + val * val, 0))
    return dotProduct / (magnitudeA * magnitudeB)
  }
}

Performance Optimizations

• Client-side vectorization: Reduces server load, Web Workers prevent UI thread blocking, local cache avoids recomputation
• Smart chunking: 500 chars per chunk, 50 char overlap, preserves document structure
• Memory management: Lazy loading, LRU cache, proactive garbage collection

Problems Solved

• Page refresh data loss: Auto-save messages to localStorage on every change via useEffect
• Vector similarity precision: Normalized embeddings, dynamic thresholds, multi-tier ranking
• Large document performance: Web Workers for async processing, prevents blocking

Performance Metrics

• First load: < 2s
• Response time: < 500ms
• Document processing: < 3s for 1MB docs
• Search latency: < 100ms

Key Takeaways

Building this project gave me deep understanding of the complete AI application development pipeline — from UI/UX design to RAG system implementation, performance optimization to user experience. The RAG system implementation deepened my understanding of vector databases, semantic search, and context generation.

Links: GitHub | Live Demo

Related posts

• Full-stack knowledge base
• Production React SSR framework
• AI tool server Lobster

Source code navigation

The full project spans two repos: erishen/rag-platform (retrieval and indexing core) and erishen/langchain-llm-toolkit (LangChain RAG wrapper).

Retrieval core (rag-platform)

File	What it does
app/bm25.py	BM25 keyword retrieval implementation
app/db.py	Vector / document storage layer
app/config.py	Config (model / embedding / corpus path)
app/models.py	Data models
app/embed/st_backend.py	Embedding backend (Sentence-Transformers, etc.)

LangChain wrapper (langchain-llm-toolkit)

File	What it does
rag_query_web.py	RAG query web entry point
src/langchain_llm_toolkit/hybrid_retriever.py	BM25 + semantic hybrid retriever
src/langchain_llm_toolkit/llm_integration.py	LLM calls, retries, caching, rate-limiting
src/langchain_llm_toolkit/rag.py	Complete RAG system