This document updates the enterprise-level custom chatbot architecture for an organization with 20,000+ employees, incorporating Retrieval-Augmented Generation (RAG) with hybrid search (vector-based semantic search combined with raw text keyword search) and Optical Character Recognition (OCR) for document processing. It addresses the user's feedback to provide more details on the Data Ingestion Pipeline and DevOps, while retaining the recommended technologies: NGINX as API Gateway, Groq for LLM Inference, Jina Embeddings, and Pinecone as the vector database. The architecture is modular, scalable, and secure, designed to process user queries, leverage NLP, search SharePoint documents, handle smart prompts, and integrate within a robust system design. Additional concepts like monitoring and error handling ensure completeness.

Data Ingestion Pipeline Flow

+---------------------------------------------+
|               Apache Airflow                 |
|   (Orchestrates and schedules pipeline)      |
+---------------------------------------------+
                   |
                   v
+---------------------------------------------+
|           SharePoint (Data Source)          |
|   (Documents: PDFs, Images, Word, etc.)     |
+---------------------------------------------+
                   |
                   v
+---------------------------------------------+
|       Azure Document Intelligence (OCR)      |
|   (Extracts text, tables, structured data)  |
+---------------------------------------------+
                   |
                   v
+---------------------------------------------+
|        Data Cleaning (Python)               |
|   (Removes noise: headers, footers, etc.)   |
+---------------------------------------------+
                   |
                   v
+---------------------------------------------+
|         Chunking (Python/NLTK)              |
|   (Splits text into semantic chunks)        |
+---------------------------------------------+
                   |
                   v
+---------------------------------------------+       +---------------------------------------------+
|         Jina Embeddings                     | ----> |       Pinecone (Vector Database)            |
|   (Generates dense vector embeddings)       |       |   (Stores embeddings for semantic search)   |
+---------------------------------------------+       +---------------------------------------------+
                   |
                   v
+---------------------------------------------+
|       PostgreSQL (Relational Database)      |
|   (Stores raw text, metadata, tables)       |
+---------------------------------------------+
                   |
                   v
+---------------------------------------------+
|       Chatbot Retrieval (Hybrid Search)     |
|   (Combines keyword and semantic search)    |
+---------------------------------------------+

Table of Contents

1. Overview
2. Architecture Components
   • User Interface (UI - Web/Teams)
   • API Gateway (NGINX with Load Balancing)
   • LLM Inference (Groq)
   • Data Connector
   • Vector Store (Pinecone with Hybrid Search)
   • Memory Management
   • Data Ingestion Pipeline (Including OCR)
   • DevOps / Deployment
   • Security
3. Additional Concepts
   • Monitoring and Logging
   • Error Handling
4. System Design and Integration
5. Resources for Further Learning
6. Conclusion

Overview

The architecture is a cloud-native, RAG-based chatbot designed for large enterprises, integrating with Microsoft’s ecosystem (e.g., SharePoint, Teams). It addresses the limitations of traditional chatbots by using hybrid search (vector-based semantic search + raw text keyword search) to retrieve accurate, context-aware information from diverse internal knowledge bases. OCR enhances document processing by extracting text from scanned documents, making them searchable. The updated architecture incorporates:

• NGINX as API Gateway with load balancing.
• Groq for low-latency LLM inference (Llama3, Mixtral).
• Jina Embeddings for vector generation, with alternatives like Sentence Transformers.
• Pinecone for vector storage, supporting hybrid search.
• OCR using cloud-based services (e.g., Azure Document Intelligence) for digitizing documents.
• Enhanced Data Ingestion Pipeline and DevOps details for developer implementation.

The updated table is: