Skip to content
Knowledge & Search6 min read

Knowledge Bases & Hybrid Retrieval (RAG)

How SCA parses SOP PDFs, blends pgvector semantic similarity with PostgreSQL full-text keyword search via RRF, and splits retrieval from viewing rights.

Logistics manuals mix prose policies with dense exact identifiers: pallet dimensions (1.85m), form codes (EX-1, EUR.1, CMR), UN hazmat classes, and customer SKU numbers. Pure vector search misses exact codes; pure keyword search misses operational phrasing.

SCA combines both via hybrid retrieval inside tenant-scoped PostgreSQL + pgvector.

Hybrid scoring pipeline

For every question routed to a specialist's Knowledge Collection:

  1. Chunk ingestion & page tracking: Uploaded handbooks are segmented using collection-level chunk size and overlap settings, preserving exact document filenames, section headings, and page indices (p. 16).
  2. Dual candidate retrieval: SCA runs cosine vector similarity (pgvector HNSW index) alongside PostgreSQL tsvector full-text ranking within the company's permitted documents.
  3. Reciprocal Rank Fusion (RRF): Both ranked candidate sets merge via configurable keyword vs meaning weighting (rag_weight_fts / rag_weight_vector), allowing ops managers to dial exact-code bias higher for customs tariff handbooks or semantic breadth higher for training guides.
  4. Live page citations: Top grounded snippets pass to the assistant and emit structured citation chips rendered inside the message's expandable Sources drawer.

Separate retrieval rights from document viewing

Commercial contracts and carrier rate sheets frequently restrict raw text browsing even when dispatchers need operational answers from them. SCA enforces two distinct Knowledge Collection permissions:

PermissionWhat it allowsTypical team setup
retrieval_useAssistant retrieves grounded passages at query time to synthesize a rule-compliant answer with source referencesFrontline dispatchers & night-shift planners
view_contentUser can open the source drawer, inspect verbatim extracted chunks, and download or browse raw filesOperations Managers, Quality Leads & Account Owners

If a user holds retrieval_use without view_content, SCA refuses raw chunk inspection cleanly while still answering operational policy questions safely.