Skip to Content
Last modified by Robert Schaub on 2025/12/24 20:35
From version 1.7
edited by Robert Schaub
on 2025/12/24 20:35
Change comment: Renamed back-links.
To version 1.1
edited by Robert Schaub
on 2025/12/21 11:25
Change comment: Imported from XAR

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -FactHarbor.Archive.FactHarbor delta for V0\.9\.70.Roadmap.WebHome
1 +Test.FactHarbor.Roadmap.WebHome
Content
... ... @@ -4,7 +4,7 @@
4 4  
5 5  **Success Metric:** <5% hallucination rate, all 4 quality gates operational
6 6  
7 -----
7 +---
8 8  
9 9  == 1. Overview ==
10 10  
... ... @@ -13,13 +13,12 @@
13 13  **Key Innovation:** Complete quality validation pipeline catches all categories of errors
14 14  
15 15  **What We're Proving:**
16 -
17 17  * All 4 quality gates work together effectively
18 18  * Evidence deduplication prevents artificial inflation
19 19  * System maintains quality at larger scale
20 20  * Quality metrics dashboard provides actionable insights
21 21  
22 -----
21 +---
23 23  
24 24  == 2. New Requirements ==
25 25  
... ... @@ -32,13 +32,11 @@
32 32  **Purpose:** Ensure AI-linked evidence actually relates to the claim
33 33  
34 34  **Validation Checks:**
35 -
36 36  1. **Semantic Similarity:** Cosine similarity between claim and evidence embeddings ≥ 0.6
37 37  2. **Entity Overlap:** At least 1 shared named entity between claim and evidence
38 38  3. **Topic Relevance:** Evidence discusses the claim's subject matter (score ≥ 0.5)
39 39  
40 40  **Action if Failed:**
41 -
42 42  * Discard irrelevant evidence (don't count it)
43 43  * If <2 relevant evidence items remain → "Insufficient Evidence" verdict
44 44  * Log discarded evidence for quality review
... ... @@ -45,7 +45,7 @@
45 45  
46 46  **Target:** 0% of evidence cited is off-topic
47 47  
48 -----
45 +---
49 49  
50 50  ==== Gate 3: Scenario Coherence Check ====
51 51  
... ... @@ -52,7 +52,6 @@
52 52  **Purpose:** Validate scenarios are logical, complete, and meaningfully different
53 53  
54 54  **Validation Checks:**
55 -
56 56  1. **Completeness:** All required fields populated (assumptions, scope, evidence context)
57 57  2. **Internal Consistency:** Assumptions don't contradict each other (score <0.3)
58 58  3. **Distinctiveness:** Scenarios are meaningfully different (similarity <0.8)
... ... @@ -59,7 +59,6 @@
59 59  4. **Minimum Detail:** At least 1 specific assumption per scenario
60 60  
61 61  **Action if Failed:**
62 -
63 63  * Merge duplicate scenarios
64 64  * Flag contradictory assumptions for review
65 65  * Reduce confidence score by 20%
... ... @@ -67,7 +67,7 @@
67 67  
68 68  **Target:** 0% duplicate scenarios, all scenarios internally consistent
69 69  
70 -----
65 +---
71 71  
72 72  === 2.2 FR54: Evidence Deduplication (NEW) ===
73 73  
... ... @@ -77,7 +77,6 @@
77 77  **Purpose:** Prevent counting the same evidence multiple times when cited by different sources
78 78  
79 79  **Problem:**
80 -
81 81  * Wire services (AP, Reuters) redistribute same content
82 82  * Different sites cite the same original study
83 83  * Aggregators copy primary sources
... ... @@ -84,7 +84,6 @@
84 84  * AKEL might count this as "5 sources" when it's really 1
85 85  
86 86  **Solution: Content Fingerprinting**
87 -
88 88  * Generate SHA-256 hash of normalized text
89 89  * Detect near-duplicates (≥85% similarity) using fuzzy matching
90 90  * Track which sources cited each unique piece of evidence
... ... @@ -92,7 +92,7 @@
92 92  
93 93  **Target:** Duplicate detection >95% accurate, evidence counts reflect reality
94 94  
95 -----
88 +---
96 96  
97 97  === 2.3 NFR13: Quality Metrics Dashboard (Internal) ===
98 98  
... ... @@ -100,7 +100,6 @@
100 100  **Fulfills:** Real-time quality monitoring during development
101 101  
102 102  **Dashboard Metrics:**
103 -
104 104  * Claim processing statistics
105 105  * Gate performance (pass/fail rates for each gate)
106 106  * Evidence quality metrics
... ... @@ -109,12 +109,11 @@
109 109  
110 110  **Target:** Dashboard functional, all metrics tracked, exportable
111 111  
112 -----
104 +---
113 113  
114 114  == 3. Success Criteria ==
115 115  
116 116  **✅ Quality:**
117 -
118 118  * Hallucination rate <5% (target: <3%)
119 119  * Average quality rating ≥8.0/10
120 120  * 0 critical failures (publishable falsities)
... ... @@ -121,7 +121,6 @@
121 121  * Gates correctly identify >95% of low-quality outputs
122 122  
123 123  **✅ All 4 Gates Operational:**
124 -
125 125  * Gate 1: Claim validation working
126 126  * Gate 2: Evidence relevance filtering working
127 127  * Gate 3: Scenario coherence checking working
... ... @@ -128,18 +128,16 @@
128 128  * Gate 4: Verdict confidence assessment working
129 129  
130 130  **✅ Evidence Deduplication:**
131 -
132 132  * Duplicate detection >95% accurate
133 133  * Evidence counts reflect reality
134 134  * Provenance tracked correctly
135 135  
136 136  **✅ Metrics Dashboard:**
137 -
138 138  * All metrics implemented and tracking
139 139  * Dashboard functional and useful
140 140  * Alerts trigger appropriately
141 141  
142 -----
130 +---
143 143  
144 144  == 4. Architecture Notes ==
145 145  
... ... @@ -154,7 +154,6 @@
154 154  {{/code}}
155 155  
156 156  **Key Additions from POC1:**
157 -
158 158  * Scenario generation component
159 159  * Evidence deduplication system
160 160  * Gates 2 & 3 implementation
... ... @@ -161,23 +161,23 @@
161 161  * Quality metrics collection
162 162  
163 163  **Still Simplified vs. Full System:**
164 -
165 165  * Single AKEL orchestration (not multi-component pipeline)
166 166  * No review queue
167 167  * No federation architecture
168 168  
169 -**See:** [[Architecture>>FactHarbor.Archive.FactHarbor delta for V0\.9\.70.Specification.Architecture.WebHome]] for details
155 +**See:** [[Architecture>>Test.FactHarbor.Specification.Architecture.WebHome]] for details
170 170  
171 -----
157 +---
172 172  
173 173  == Related Pages ==
174 174  
175 -* [[POC1>>FactHarbor.Archive.FactHarbor delta for V0\.9\.70.Roadmap.POC1.WebHome]] - Previous phase
176 -* [[Beta 0>>Archive.FactHarbor delta for V0\.9\.70.Roadmap.Beta0.WebHome]] - Next phase
177 -* [[Roadmap Overview>>FactHarbor.Archive.FactHarbor delta for V0\.9\.70.Roadmap.WebHome]]
178 -* [[Architecture>>FactHarbor.Archive.FactHarbor delta for V0\.9\.70.Specification.Architecture.WebHome]]
161 +* [[POC1>>Test.FactHarbor.Roadmap.POC1.WebHome]] - Previous phase
162 +* [[Beta 0>>Test.FactHarbor.Roadmap.Beta0.WebHome]] - Next phase
163 +* [[Roadmap Overview>>Test.FactHarbor.Roadmap.WebHome]]
164 +* [[Architecture>>Test.FactHarbor.Specification.Architecture.WebHome]]
179 179  
180 -----
166 +---
181 181  
182 182  **Document Status:** ✅ POC2 Specification Complete - Waiting for POC1 Completion
183 183  **Version:** V0.9.70
170 +