Skip to Content

Changes for page Automation

Last modified by Robert Schaub on 2025/12/24 21:46
From version 2.1
edited by Robert Schaub
on 2025/12/18 12:54
Change comment: Imported from XAR
To version 3.2
edited by Robert Schaub
on 2025/12/24 21:46
Change comment: Update document after refactoring.

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -FactHarbor.Specification.WebHome
1 +Archive.FactHarbor V0\.9\.50 Plus (Prev Rel).Specification.WebHome
Content
... ... @@ -37,6 +37,78 @@
37 37  * Detected manipulation attempt
38 38  * Unusual pattern
39 39  * Moderator reviews and may take action
40 +
41 +== 2.5 LLM-Based Processing Architecture ==
42 +
43 +FactHarbor delegates complex reasoning and analysis tasks to Large Language Models (LLMs). The architecture evolves from POC to production:
44 +
45 +=== POC: Two-Phase Approach ===
46 +
47 +**Phase 1: Claim Extraction**
48 +* Single LLM call to extract all claims from submitted content
49 +* Light structure, focused on identifying distinct verifiable claims
50 +* Output: List of claims with context
51 +
52 +**Phase 2: Claim Analysis (Parallel)**
53 +* Single LLM call per claim (parallelizable)
54 +* Full structured output: Evidence, Scenarios, Sources, Verdict, Risk
55 +* Each claim analyzed independently
56 +
57 +**Advantages:**
58 +* Fast to implement (2-4 weeks to working POC)
59 +* Only 2-3 API calls total (1 + N claims)
60 +* Simple to debug (claim-level isolation)
61 +* Proves concept viability
62 +
63 +=== Production: Three-Phase Approach ===
64 +
65 +**Phase 1: Claim Extraction + Validation**
66 +* Extract distinct verifiable claims
67 +* Validate claim clarity and uniqueness
68 +* Remove duplicates and vague claims
69 +
70 +**Phase 2: Evidence Gathering (Parallel)**
71 +* For each claim independently:
72 + * Find supporting and contradicting evidence
73 + * Identify authoritative sources
74 + * Generate test scenarios
75 +* Validation: Check evidence quality and source validity
76 +* Error containment: Issues in one claim don't affect others
77 +
78 +**Phase 3: Verdict Generation (Parallel)**
79 +* For each claim:
80 + * Generate verdict based on validated evidence
81 + * Assess confidence and risk level
82 + * Flag low-confidence results for human review
83 +* Validation: Check verdict consistency with evidence
84 +
85 +**Advantages:**
86 +* Error containment between phases
87 +* Clear quality gates and validation
88 +* Observable metrics per phase
89 +* Scalable (parallel processing across claims)
90 +* Adaptable (can optimize each phase independently)
91 +
92 +=== LLM Task Delegation ===
93 +
94 +All complex cognitive tasks are delegated to LLMs:
95 +* **Claim Extraction**: Understanding context, identifying distinct claims
96 +* **Evidence Finding**: Analyzing sources, assessing relevance
97 +* **Scenario Generation**: Creating testable hypotheses
98 +* **Source Evaluation**: Assessing reliability and authority
99 +* **Verdict Generation**: Synthesizing evidence into conclusions
100 +* **Risk Assessment**: Evaluating potential impact
101 +
102 +=== Error Mitigation ===
103 +
104 +Research shows sequential LLM calls face compound error risks. FactHarbor mitigates this through:
105 +* **Validation gates** between phases
106 +* **Confidence thresholds** for quality control
107 +* **Parallel processing** to avoid error propagation across claims
108 +* **Human review queue** for low-confidence verdicts
109 +* **Independent claim processing** - errors in one claim don't cascade to others
110 +
111 +
40 40  == 3. Risk Tiers ==
41 41  Risk tiers classify claims by potential impact and guide audit sampling rates.
42 42  === 3.1 Tier A (High Risk) ===