Changes for page Architecture
Last modified by Robert Schaub on 2025/12/24 18:26
Summary
-
Page properties (1 modified, 0 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -20,43 +20,114 @@ 20 20 ==== Processing Layer ==== 21 21 Core business logic and AI processing: 22 22 * **AKEL Pipeline**: AI-driven claim analysis (parallel processing) 23 - * Parse and extract claim components 24 - * Gather evidence from multiple sources 25 - * Check source track records 26 - * Extract scenarios from evidence 27 - * Synthesize verdicts 28 - * Calculate risk scores 23 + * Parse and extract claim components 24 + * Gather evidence from multiple sources 25 + * Check source track records 26 + * Extract scenarios from evidence 27 + * Synthesize verdicts 28 + * Calculate risk scores 29 + 30 +* **LLM Abstraction Layer**: Provider-agnostic AI access 31 + * Multi-provider support (Anthropic, OpenAI, Google, local models) 32 + * Automatic failover and rate limit handling 33 + * Per-stage model configuration 34 + * Cost optimization through provider selection 35 + * No vendor lock-in 29 29 * **Background Jobs**: Automated maintenance tasks 30 - 31 - 32 - 33 - 37 + * Source track record updates (weekly) 38 + * Cache warming and invalidation 39 + * Metrics aggregation 40 + * Data archival 34 34 * **Quality Monitoring**: Automated quality checks 35 - 36 - 37 - 42 + * Anomaly detection 43 + * Contradiction detection 44 + * Completeness validation 38 38 * **Moderation Detection**: Automated abuse detection 39 - 40 - 41 - 46 + * Spam identification 47 + * Manipulation detection 48 + * Flag suspicious activity 42 42 ==== Data & Storage Layer ==== 43 43 Persistent data storage and caching: 44 44 * **PostgreSQL**: Primary database for all core data 45 - 46 - 47 - 48 - 52 + * Claims, evidence, sources, users 53 + * Scenarios, edits, audit logs 54 + * Built-in full-text search 55 + * Time-series capabilities for metrics 49 49 * **Redis**: High-speed caching layer 50 - 51 - 52 - 57 + * Session data 58 + * Frequently accessed claims 59 + * API rate limiting 53 53 * **S3 Storage**: Long-term archival 54 - 55 - 56 - 61 + * Old edit history (90+ days) 62 + * AKEL processing logs 63 + * Backup snapshots 57 57 **Optional future additions** (add only when metrics prove necessary): 58 58 * **Elasticsearch**: If PostgreSQL full-text search becomes slow 59 59 * **TimescaleDB**: If metrics queries become a bottleneck 67 + 68 + 69 +=== 2.2 LLM Abstraction Layer === 70 + 71 +{{include reference="Test.FactHarbor.Specification.Diagrams.LLM Abstraction Architecture.WebHome"/}} 72 + 73 +**Purpose:** FactHarbor uses a provider-agnostic abstraction layer for all AI interactions, avoiding vendor lock-in and enabling flexible provider selection. 74 + 75 +**Multi-Provider Support:** 76 +* **Primary:** Anthropic Claude API (Haiku for extraction, Sonnet for analysis) 77 +* **Secondary:** OpenAI GPT API (automatic failover) 78 +* **Tertiary:** Google Vertex AI / Gemini 79 +* **Future:** Local models (Llama, Mistral) for on-premises deployments 80 + 81 +**Provider Interface:** 82 +* Abstract `LLMProvider` interface with `complete()`, `stream()`, `getName()`, `getCostPer1kTokens()`, `isAvailable()` methods 83 +* Per-stage model configuration (Stage 1: Haiku, Stage 2 & 3: Sonnet) 84 +* Environment variable and database configuration 85 +* Adapter pattern implementation (AnthropicProvider, OpenAIProvider, GoogleProvider) 86 + 87 +**Configuration:** 88 +* Runtime provider switching without code changes 89 +* Admin API for provider management (`POST /admin/v1/llm/configure`) 90 +* Per-stage cost optimization (use cheaper models for extraction, quality models for analysis) 91 +* Support for rate limit handling and cost tracking 92 + 93 +**Failover Strategy:** 94 +* Automatic fallback: Primary → Secondary → Tertiary 95 +* Circuit breaker pattern for unavailable providers 96 +* Health checking and provider availability monitoring 97 +* Graceful degradation when all providers unavailable 98 + 99 +**Cost Optimization:** 100 +* Track and compare costs across providers per request 101 +* Enable A/B testing of different models for quality/cost tradeoffs 102 +* Per-stage provider selection for optimal cost-efficiency 103 +* Cost comparison: Anthropic ($0.114), OpenAI ($0.065), Google ($0.072) per article at 0% cache 104 + 105 +**Architecture Pattern:** 106 + 107 +{{code}} 108 +AKEL Stages LLM Abstraction Providers 109 +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 110 +Stage 1 Extract ──→ Provider Interface ──→ Anthropic (PRIMARY) 111 +Stage 2 Analyze ──→ Configuration ──→ OpenAI (SECONDARY) 112 +Stage 3 Holistic ──→ Failover Handler ──→ Google (TERTIARY) 113 + └→ Local Models (FUTURE) 114 +{{/code}} 115 + 116 +**Benefits:** 117 +* **No Vendor Lock-In:** Switch providers based on cost, quality, or availability without code changes 118 +* **Resilience:** Automatic failover ensures service continuity during provider outages 119 +* **Cost Efficiency:** Use optimal provider per task (cheap for extraction, quality for analysis) 120 +* **Quality Assurance:** Cross-provider output verification for critical claims 121 +* **Regulatory Compliance:** Use specific providers for data residency requirements 122 +* **Future-Proofing:** Easy integration of new models as they become available 123 + 124 +**Cross-References:** 125 +* [[Requirements>>FactHarbor.Specification.Requirements.WebHome#NFR-14]]: NFR-14 (formal requirement) 126 +* [[POC Requirements>>FactHarbor.Specification.POC.Requirements#NFR-POC-11]]: NFR-POC-11 (POC1 implementation) 127 +* [[API Specification>>FactHarbor.Specification.POC.API-and-Schemas.WebHome#Section-6]]: Section 6 (implementation details) 128 +* [[Design Decisions>>FactHarbor.Specification.Design-Decisions#Section-9]]: Section 9 (design rationale) 129 + 130 + 60 60 === 2.2 Design Philosophy === 61 61 **Start Simple, Evolve Based on Metrics** 62 62 The architecture deliberately starts simple: ... ... @@ -120,7 +120,7 @@ 120 120 * Process 100 claims with ~3x latency of single claim 121 121 * Parallel processing across independent claims 122 122 * Linear cost scaling with claim count 123 - 194 +=== 2.3 Design Philosophy === 124 124 **Quality:** 125 125 * Validation gates between phases 126 126 * Errors isolated to individual claims ... ... @@ -131,7 +131,6 @@ 131 131 * Can use different model sizes per phase 132 132 * Easy to add human review at decision points 133 133 134 - 135 135 == 4. Storage Architecture == 136 136 {{include reference="FactHarbor.Specification.Diagrams.Storage Architecture.WebHome"/}} 137 137 See [[Storage Strategy>>FactHarbor.Specification.Architecture.WebHome]] for detailed information. ... ... @@ -181,17 +181,17 @@ 181 181 === 5.3 Quality Monitoring === 182 182 **Automated checks run continuously**: 183 183 * **Anomaly Detection**: Flag unusual patterns 184 - 185 - 186 - 254 + * Sudden confidence score changes 255 + * Unusual evidence distributions 256 + * Suspicious source patterns 187 187 * **Contradiction Detection**: Identify conflicts 188 - 189 - 190 - 258 + * Evidence that contradicts other evidence 259 + * Claims with internal contradictions 260 + * Source track record anomalies 191 191 * **Completeness Validation**: Ensure thoroughness 192 - 193 - 194 - 262 + * Sufficient evidence gathered 263 + * Multiple source types represented 264 + * Key scenarios identified 195 195 === 5.4 Moderation Detection === 196 196 **Automated abuse detection**: 197 197 * **Spam Identification**: Pattern matching for spam claims ... ... @@ -280,7 +280,7 @@ 280 280 === 10.1 When to Add Complexity === 281 281 See [[When to Add Complexity>>FactHarbor.Specification.When-to-Add-Complexity]] for specific triggers. 282 282 **Elasticsearch**: When PostgreSQL search consistently >500ms 283 -**TimescaleDB**: When metrics queries consistently >1s 353 +**TimescaleDB**: When metrics queries consistently >1s 284 284 **Federation**: When 10,000+ users and explicit demand 285 285 **Complex Reputation**: When 100+ active contributors 286 286 === 10.2 Federation (V2.0+) ===