Skip to Content
Last modified by Robert Schaub on 2025/12/24 18:26
From version 3.1
edited by Robert Schaub
on 2025/12/24 17:59
Change comment: Imported from XAR
To version 1.1
edited by Robert Schaub
on 2025/12/24 11:54
Change comment: Imported from XAR

Summary

Details

Page properties
Content
... ... @@ -4,6 +4,7 @@
4 4  
5 5  **Success Metric:** <10% hallucination rate, quality gates prevent low-confidence publications
6 6  
7 +
7 7  == 1. Overview ==
8 8  
9 9  POC1 validates that the core AKEL workflow (Article → Claims → Verdicts) can produce trustworthy fact-checking analyses automatically. This phase implements **2 critical quality gates** to prevent low-quality outputs from being published.
... ... @@ -40,6 +40,7 @@
40 40  * A/B testing
41 41  * Gates 2 & 3 (Evidence relevance, Scenario coherence)
42 42  
44 +
43 43  === Experimental Features (POC1) ===
44 44  
45 45  **Context-Aware Analysis** (Approach 1: Single-Pass Holistic)
... ... @@ -57,7 +57,7 @@
57 57  * Success criteria: ≥70% accuracy on misleading articles
58 58  * Marked as experimental - doesn't block POC1 success
59 59  
60 -**See:** [[Article Verdict Problem>>FactHarbor.Specification.POC.Article-Verdict-Problem]] for complete analysis
62 +**See:** [[Article Verdict Problem>>Test.FactHarbor.Specification.POC.Article-Verdict-Problem]] for complete analysis
61 61  
62 62  **Decision:**
63 63  * If ≥70% accuracy → ship in POC2
... ... @@ -68,7 +68,7 @@
68 68  
69 69  === 3.1 NFR11: Quality Assurance Framework (POC1 Lite Version) ===
70 70  
71 -**Importance:** CRITICAL - Core POC1 Requirement
73 +**Importance:** CRITICAL - Core POC1 Requirement
72 72  **Fulfills:** AI safety, credibility, prevents embarrassing failures
73 73  
74 74  **Specification:**
... ... @@ -100,6 +100,7 @@
100 100  
101 101  **Target:** 0% opinion statements processed as facts
102 102  
105 +
103 103  ==== Gate 4: Verdict Confidence Assessment ====
104 104  
105 105  **Purpose:** Only publish verdicts with sufficient evidence and confidence
... ... @@ -113,20 +113,20 @@
113 113  
114 114  **Confidence Tiers:**
115 115  {{code}}HIGH (80-100%):
116 - - ≥3 sources
117 - - ≥0.7 average quality
118 - - ≥80% agreement
119 -
119 + - ≥3 sources
120 + - ≥0.7 average quality
121 + - ≥80% agreement
122 +
120 120  MEDIUM (50-79%):
121 - - ≥2 sources
122 - - ≥0.6 average quality
123 - - ≥60% agreement
124 -
124 + - ≥2 sources
125 + - ≥0.6 average quality
126 + - ≥60% agreement
127 +
125 125  LOW (0-49%):
126 - - ≥2 sources BUT low quality/agreement
127 -
129 + - ≥2 sources BUT low quality/agreement
130 +
128 128  INSUFFICIENT:
129 - - <2 sources → DO NOT PUBLISH{{/code}}
132 + - <2 sources → DO NOT PUBLISH{{/code}}
130 130  
131 131  **POC1 Publication Rule:**
132 132  
... ... @@ -135,6 +135,7 @@
135 135  
136 136  **Target:** 0% verdicts published with <2 sources
137 137  
141 +
138 138  === 3.2 Modified FR7: Automated Verdicts (Enhanced) ===
139 139  
140 140  **Enhancement for POC1:**
... ... @@ -145,13 +145,13 @@
145 145  AKEL Workflow (POC1):
146 146  
147 147  1. Extract claims from article
148 - ↓
152 + ↓
149 149  2. [GATE 1] Validate each claim is fact-checkable
150 - ↓ (pass claims only)
154 + ↓ (pass claims only)
151 151  3. Generate verdicts for each claim
152 - ↓
156 + ↓
153 153  4. [GATE 4] Validate verdict has sufficient evidence
154 - ↓ (pass verdicts only)
158 + ↓ (pass verdicts only)
155 155  5. Display to user
156 156  
157 157  Failed claims/verdicts:
... ... @@ -176,22 +176,23 @@
176 176  
177 177  {{code}}
178 178  Analysis Summary:
179 - Total Claims Found: 5
180 - Verifiable Claims: 3
181 - Non-verifiable (Opinion): 1
182 - Non-verifiable (Prediction): 1
183 -
184 - Verdicts Generated: 3
185 - High Confidence: 1
186 - Medium Confidence: 2
187 - Insufficient Evidence: 0
188 -
189 - Evidence Sources: 12 total
190 - Average Source Quality: 0.73
191 -
192 - Quality Score: 8.5/10
183 + Total Claims Found: 5
184 + Verifiable Claims: 3
185 + Non-verifiable (Opinion): 1
186 + Non-verifiable (Prediction): 1
187 +
188 + Verdicts Generated: 3
189 + High Confidence: 1
190 + Medium Confidence: 2
191 + Insufficient Evidence: 0
192 +
193 + Evidence Sources: 12 total
194 + Average Source Quality: 0.73
195 +
196 + Quality Score: 8.5/10
193 193  {{/code}}
194 194  
199 +
195 195  == 4. Success Criteria ==
196 196  
197 197  POC1 is considered **SUCCESSFUL** if:
... ... @@ -234,6 +234,7 @@
234 234  
235 235  **Only proceed to POC2 if all success criteria met**
236 236  
242 +
237 237  == 6. Architecture Notes ==
238 238  
239 239  **POC1 Simplified Architecture:**
... ... @@ -240,8 +240,8 @@
240 240  
241 241  {{code}}
242 242  User Input → AKEL Processing → Quality Gates → Display
243 - (claim extraction (Gates 1 & 4)
244 - + verdicts)
249 + (claim extraction (Gates 1 & 4)
250 + + verdicts)
245 245  {{/code}}
246 246  
247 247  **vs. Full System (Future):**
... ... @@ -259,15 +259,16 @@
259 259  * 2 gates instead of 4
260 260  * No review queue
261 261  
262 -**See:** [[Architecture>>FactHarbor pre10 V0\.9\.70.Specification.Architecture.WebHome]] for details
268 +**See:** [[Architecture>>Test.FactHarbor pre10 V0\.9\.70.Specification.Architecture.WebHome]] for details
263 263  
270 +
264 264  == Related Pages ==
265 265  
266 -* [[Roadmap Overview>>FactHarbor pre10 V0\.9\.70.Roadmap.WebHome]] - All phases
267 -* [[POC2 Requirements>>FactHarbor pre10 V0\.9\.70.Roadmap.POC2.WebHome]] - Next phase
268 -* [[Requirements>>FactHarbor pre10 V0\.9\.70.Specification.Requirements.WebHome]] - Full system requirements
269 -* [[Architecture>>FactHarbor pre10 V0\.9\.70.Specification.Architecture.WebHome]] - System architecture
270 -* [[NFR11 Full Specification>>FactHarbor.Specification.Requirements.WebHome#NFR11]] - Complete quality framework
273 +* [[Roadmap Overview>>Test.FactHarbor pre10 V0\.9\.70.Roadmap.WebHome]] - All phases
274 +* [[POC2 Requirements>>Test.FactHarbor pre10 V0\.9\.70.Roadmap.POC2.WebHome]] - Next phase
275 +* [[Requirements>>Test.FactHarbor pre10 V0\.9\.70.Specification.Requirements.WebHome]] - Full system requirements
276 +* [[Architecture>>Test.FactHarbor pre10 V0\.9\.70.Specification.Architecture.WebHome]] - System architecture
277 +* [[NFR11 Full Specification>>Test.FactHarbor.Specification.Requirements.WebHome#NFR11]] - Complete quality framework
271 271  
272 -**Document Status:** ✅ POC1 Specification Complete - Ready for Implementation
279 +**Document Status:** ✅ POC1 Specification Complete - Ready for Implementation
273 273  **Version:** V0.9.70