Knowledge Bases & Hybrid Retrieval (RAG)
How SCA parses SOP PDFs, blends pgvector semantic similarity with PostgreSQL full-text keyword search via RRF, and splits retrieval from viewing rights.
Logistics manuals mix prose policies with dense exact identifiers: pallet dimensions (1.85m), form codes (EX-1, EUR.1, CMR), UN hazmat classes, and customer SKU numbers. Pure vector search misses exact codes; pure keyword search misses operational phrasing.
SCA combines both via hybrid retrieval inside tenant-scoped PostgreSQL + pgvector.
Hybrid scoring pipeline
For every question routed to a specialist's Knowledge Collection:
- Chunk ingestion & page tracking: Uploaded handbooks are segmented using collection-level chunk size and overlap settings, preserving exact document filenames, section headings, and page indices (
p. 16). - Dual candidate retrieval: SCA runs cosine vector similarity (
pgvectorHNSW index) alongside PostgreSQLtsvectorfull-text ranking within the company's permitted documents. - Reciprocal Rank Fusion (RRF): Both ranked candidate sets merge via configurable keyword vs meaning weighting (
rag_weight_fts/rag_weight_vector), allowing ops managers to dial exact-code bias higher for customs tariff handbooks or semantic breadth higher for training guides. - Live page citations: Top grounded snippets pass to the assistant and emit structured citation chips rendered inside the message's expandable Sources drawer.
Separate retrieval rights from document viewing
Commercial contracts and carrier rate sheets frequently restrict raw text browsing even when dispatchers need operational answers from them. SCA enforces two distinct Knowledge Collection permissions:
| Permission | What it allows | Typical team setup |
|---|---|---|
retrieval_use | Assistant retrieves grounded passages at query time to synthesize a rule-compliant answer with source references | Frontline dispatchers & night-shift planners |
view_content | User can open the source drawer, inspect verbatim extracted chunks, and download or browse raw files | Operations Managers, Quality Leads & Account Owners |
If a user holds retrieval_use without view_content, SCA refuses raw chunk inspection cleanly while still answering operational policy questions safely.