High-Level Architecture (Pebble Orchestrator)¶

This document describes the Technical Foundation and Data Flow for the Pebble Orchestrator (Pebble IQ). For a phase-by-phase delivery schedule, see the Product Delivery Roadmap.

System Overview¶

The Pebble Orchestrator (Pebble IQ) is a multi-entity, event-driven architecture built to unify chaotic communication (Email) into structured operational workflows (Kanban → CRM → ERP).

• Multi-Entity Architecture: Parallel data silos for legal entities (e.g., entity_A, entity_B) with cumulative reporting.
• Incremental Automation (Shadow-to-Partner): A dedicated learning loop that transitions from manual shadowing to autonomous partner over 12 months.
• Native Visual Kanban: Internal React-based drag-and-drop board for real-time workflow management.
• Unified Logic Layer: Centralized orchestration for CRM Master, Logistics, and Quality.
• AI-Augmented Logic: Intelligent stream classification (Sales vs Ops vs Tender) using a Dual-Database (Rules + Memory) architecture.
• Native Persistence: Single source of truth for all engagement and transaction data.
• Keycloak Identity Foundation: Standardized SSO and identity management for the enterprise.
• New Platform Evaluation: A coordinated assessment of n8n (Automations), Nextcloud/Mayan (EDMS), and Apache Guacamole (Secure VDI).

🧭 Architecture Documentation Roadmap¶

This document serves as the high-level entry point. For specific implementation details, refer to:

Orchestrator Diagram¶

flowchart TD
    %% Global Styles
    classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px,font-size:14px;
    classDef critical fill:#ffeded,stroke:#d32f2f,stroke-width:2px;
    classDef phase1 fill:#f9f,stroke:#333,stroke-dasharray: 5 5;

    subgraph Channels["Communication Channels"]
        A1["Outlook / Office 365"]
        A2["In-App Forms"]
        A3["Legacy Exports (Excel/D365)"]
    end

    subgraph Ingestion["Ingestion Layer"]
        B["Email Listener Service"]
        C["Legacy Ingestion Engine"]
    end

    subgraph Core["Native Orchestration Core"]
        D["AI Classifier / Intent Engine"]
        E["Native CRM Service"]
        F["Native Kanban Service"]
        O["CRM Workflow Service (Lead-to-Enquiry)"]
    end

    subgraph UI["Visual Workspace (Native)"]
        G["Dynamic Enquiry Board"]
        H["Sales Pipeline View"]
    end

    subgraph Phase1_Box["Phase 1: ERP Integration (Tally)"]
        T["Tally Prime (XML/HTTP Port 9000)"]
    end

    subgraph Persistence["Persistence & State"]
        K[("PostgreSQL - Unified Core")]
        L[("Redis - Activity Stream")]
        M[("Object Storage - Docs")]
        V[("Vector DB - Shadow Memory")]
    end

    subgraph Phase2_4["Long-term Infrastructure"]
        NX["Nextcloud / Mayan (EDMS)"]
        N8["n8n (SOP Automation)"]
        GU["Apache Guacamole (VDI)"]
    end

    A1 --> B
    A2 --> Core
    A3 --> C
    B --> D
    C --> E
    D --> F
    F --> G
    O <--> K
    E <--> K
    F <--> K
    G <--> L
    K <--> M

    %% Phase 1 Connections
    E -.-> T
    F -.-> T
    T -.-> G

    %% Future Phase Connections
    G <--> N8
    N8 <--> K
    K <--> NX
    G --- GU

    %% Apply Styles
    class T phase1;
    class D,O critical;

Integration Architecture Map¶

The Pebble IQ ecosystem connects specialized "Operational Engines" (Prelead, RFQ, etc.) to a foundational "Native CRM" (the screens and workflow).

graph TD
    %% Global Styles
    classDef source fill:#e1f5fe,stroke:#01579b,font-size:14px;
    classDef foundation fill:#fff3e0,stroke:#e65100,font-size:14px;
    classDef engine fill:#f3e5f5,stroke:#4a148c,font-size:14px;

    subgraph Sources["1. Signal Sources"]
        CP[Campaign Mgmt]
        EM[Email Ingestion]
        PL[Prelead Sheets]
    end

    subgraph Master["2. Native CRM & Master Data"]
        Scr["CRM Master (The Screens)"]
        Wfl["CRM Workflow (Status: New -> Qual -> Enq)"]
        MDM["MDM Engine (Prelead -> Prospect -> Cust)"]
    end

    subgraph Engines["3. Operational Engines"]
        RFQ[RFQ & Sourcing Engine]
        Cst[Costing & Pricing IQ]
        Quote[Quotation Engine]
        Order[Order Management]
        Field[Field Sales & Visits]
        Tender[Tender Management]
        Quality[Quality & LIMS]
        Grades["Product Grade Master"]
    end

    %% Linkages
    Sources -->|Feed Data| MDM
    MDM -->|Validated| Scr
    Sources -->|Set Initial State| Wfl

    Wfl -->|State = Pending RFQ| RFQ
    Grades -->|Filter Suppliers| RFQ
    RFQ -->|Rate Received| Cst

    Scr -->|Context: History/Losses| Cst
    Cst -->|Target Price| Quote
    Quote -->|Final PDF| Wfl

    Quote -->|Deal Won| Order
    Order -->|Invoice & Status| Wfl
    Wfl -->|Visit Planned| Field
    Sources -->|GEM Alerts| Tender
    Wfl -->|Sample Request| Quality

    EM -->|CC-Contact Extraction| MDM

    %% Apply Styles
    class CP,EM,PL source;
    class Scr,Wfl,MDM foundation;
    class RFQ,Cst,Quote,Order,Field,Tender,Quality,Grades engine;

Relationship Key:¶

• CRM Master (Screens): The "Static Container" for Company, Contact, and Engagement data.
• CRM Workflow (States): The "Horizontal Path" that a lead takes from identifying a signal to closing a deal.
• Specialized Engines: The "Vertical Logic" that performs complex tasks at specific workflow stages:
  • Campaign: Drives the "New" state (Top of Funnel).
  • Prelead: Cleans and validates data before it enters the formal CRM.
  • RFQ: Triggered when an Enquiry is in "Pending RFQ" status.
  • Costing: The logic required to move from Enquiry to Quotation.
  • Quotation: Template engine for generating multi-entity PDFs and HTML emails.
  • Order Mgmt: Handling the "Closed Won" to "Invoiced" lifecycle.
  • Field Sales: Routing and verification for physical customer visits.
  • Tender: Specialized compliance workflow for GEM ecosystem.
  • Quality: Barcode-based LIMS integration for sample approval.

Core Operational Components¶

1. Ingestion & Triage¶

Listen to sales@, logistics@, and tender@ mailboxes using the Pebble Listener Service.

• Action: Converts every raw email into a structured "Email Card" (JSONB Metadata).
• Deduplication: Deep-scan against CRM Master via GST/CIN/Email to prevent lead scatter.

2. Stream Classification (AI)¶

Route incoming cards to the correct functional board based on intent extraction.

• Streams:
  • CRM Stream: Sales enquiries, quote requests, meeting invites.
  • ERP Stream: Existing POs, logistics tracking, invoice queries.
  • Tender Stream: GEM portal alerts, NIT documents, bid deadlines.

3. Native Visual Kanban¶

The primary UI for sales and operations teams, built directly into the Pebble platform.

• Logic: Custom React components managing workflow state and transitions.
• State Machine: Column moves (e.g., Qualify → Quote) trigger backend Activity-Stream events.
• Deep Integration: Seamless linkage between Kanban cards and the Native CRM Master records.

4. CRM Workflow Engine (The State Machine)¶

The logic engine that manages the lifecycle of a lead from source identification to final enquiry conversion.

• FSM Logic: Finite State Machine (FSM) managing transitions: New → Qualified → Enquiry.
• Validation Gates: Automated CIN/PAN/GST checks must pass before a lead becomes "Qualified."
• Response Orchestration: Selects the appropriate "Response Playbook" (Price pushback, Sample Followup, etc.) based on customer interaction.

5. Logic & Persistence Layer¶

Handles all internal business rules and data storage.

• Engagement Master: Pebble is the sole source of truth for Engagement (Chatter, Timeline, CRM).
• Financial Baseline: Historical financials are ingested from legacy systems for context.
• Unified Core: A single multi-tenant database manages both CRM, Workflow state, and State Machine history.

6. DevOps & Observability¶

Infrastructure-level services ensuring production reliability.

• Backups: Nightly PostgreSQL/Object Storage dumps with automated restore testing.
• Monitoring: Grafana-based tracking of ingestion latency and system health.
• Provisioning: Automated user-provisioning with granular RBAC controls.

7. ERP Integration (Phase 1)¶

Connects the Native CRM/Kanban core to Tally Prime for transactional data exchange.

• Mechanism: Tally XML over HTTP (Port 9000) running on a local/gateway machine.
• Data Flow:
  • Outbound: Sales Order creation from Kanban cards.
  • Inbound: Live stock availability, ledger balances, and batch tracking for draft grounded replies.
• Dispatch Gap: Since Tally lacks native logistics/dispatch management, Pebble IQ acts as the Custom Dispatch Interface, managing batch allocation and logistics status before finalizing the transaction in Tally via XML.
• Identifier: Distinguished in diagrams by Dashed Lines and Magenta Coloring to signify Phase 1 scope.

8. Content foundation: Nextcloud & Mayan (Phase 2)¶

Transition from static object storage to a full Enterprise Document Management System (EDMS). - Nextcloud: Provides the "SharePoint-like" user experience for general file sharing. - Mayan EDMS: Manages the high-compliance, versioned industrial documents for ISO audits.

9. Process Orchestration: n8n (Phase 3)¶

Moves from hard-coded Python routines to a visual SOP Engine. - Role: n8n acts as the middleware that executes the "Series of Actions" defined in the Master Sheet (e.g., "On Invoice Received -> Store in Nextcloud -> Update Plane -> Notify Finance").

10. Secure Workplace: Apache Guacamole (Phase 4)¶

Provides a secure "walled garden" for sensitive corporate data. - Role: A virtual desktop interface that prevents unauthorized downloads and provides full audit logs of user interaction with sensitive financial screens.

Multi-Entity Data Strategy¶

To support multi-entity organizations (e.g., 3 legal entities), the architecture employs a Federated Master Data approach.

Technical Performance Targets¶

Security & Compliance¶

• Auth: JWT-based RBAC (Role-Based Access Control).
• Audit: Every card move and attribute change is logged for SLA tracking.
• Privacy: TLS 1.3 for all data in transit; AES-256 for credentials/tokens.

User Personas¶

1. Sales / Business Development Rep¶

• Goal: Convert email enquiries into won deals.
• Workflow: Kanban triage → CRM Sync → Quotation Generation.

2. Operations / Logistics Coordinator¶

• Goal: Move orders from "Closed Won" to "Delivered".
• Workflow: Order card movement → Driver dispatch → Route planning.

3. Tender Manager¶

• Goal: Track bid compliance and deadlines.
• Workflow: Tender screen → Entity selection → Bid status tracking.

4. IT / System Admin¶

• Goal: Monitor system health and integrations.
• Workflow: Connector status check → Error logs → API key management.

← Back to Use Cases | View Product Delivery Roadmap