Skip to Content

Changes for page POC Requirements

Last modified by Robert Schaub on 2026/02/08 08:25
From version 1.2
edited by Robert Schaub
on 2025/12/22 14:33
Change comment: Update document after refactoring.
To version 1.8
edited by Robert Schaub
on 2026/02/08 08:25
Change comment: Renamed back-links.

Summary

Details

Page properties
Content
... ... @@ -1,7 +1,9 @@
1 -= POC Requirements = **Status:** ✅ Approved for Development **Version:** 3.0 (Aligned with Main Requirements) **Goal:** Prove that AI can extract claims and determine verdicts automatically without human intervention {{info}}
2 -**Core Philosophy:** POC validates the [[Main Requirements>>FactHarbor.Specification.Requirements.WebHome]] through simplified implementation. All POC features map to formal FR/NFR requirements.
3 -{{/info}} == 1. POC Overview == === 1.1 What POC Tests === **Core Question:**
1 += POC Requirements =
2 +
3 + **Status:** ✅ Approved for Development **Version:** 3.0 (Aligned with Main Requirements) **Goal:** Prove that AI can extract claims and determine verdicts automatically without human intervention {{info}}**Core Philosophy:** POC validates the [[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]] through simplified implementation. All POC features map to formal FR/NFR requirements.{{/info}} == 1. POC Overview == === 1.1 What POC Tests === **Core Question:**
4 +
4 4  > Can AI automatically extract factual claims from articles and evaluate them with reasonable verdicts? **What we're proving:**
6 +
5 5  * AI can identify factual claims from text
6 6  * AI can evaluate those claims with structured evidence
7 7  * Quality gates can filter unreliable outputs
... ... @@ -8,13 +8,12 @@
8 8  * The core workflow is technically feasible **What we're NOT proving:**
9 9  * Production-ready reliability (that's POC2)
10 10  * User-facing features (that's Beta 0)
11 -* Full IFCN compliance (that's V1.0) === 1.2 Requirements Mapping === POC1 implements a **subset** of the full system requirements defined in [[Main Requirements>>FactHarbor.Specification.Requirements.WebHome]]. **Scope Summary:**
13 +* Full IFCN compliance (that's V1.0) === 1.2 Requirements Mapping === POC1 implements a **subset** of the full system requirements defined in [[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]]. **Scope Summary:**
12 12  * **In Scope:** 8 requirements (7 FRs + 1 NFR)
13 13  * **Partial:** 3 NFRs (simplified versions)
14 -* **Out of Scope:** 19 requirements (deferred to later phases) == 2. Requirements Scope Matrix == {{success}}
15 -**Requirements Traceability:** This matrix shows which [[Main Requirements>>FactHarbor.Specification.Requirements.WebHome]] are implemented in POC1, providing full traceability between POC and system requirements.
16 -{{/success}} |=Requirement|=POC1 Status|=Implementation Level|=Notes
17 -|**CORE WORKFLOW**||||
16 +* **Out of Scope:** 19 requirements (deferred to later phases) == 2. Requirements Scope Matrix == {{success}}**Requirements Traceability:** This matrix shows which [[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]] are implemented in POC1, providing full traceability between POC and system requirements.{{/success}} |=Requirement|=POC1 Status|=Implementation Level|=Notes
17 +
18 +|**CORE WORKFLOW**||||\\
18 18  |FR1: Claim Extraction|✅ **In Scope**|Full|AKEL extracts claims from text
19 19  |FR2: Claim Context|✅ **In Scope**|Basic|Context preserved with claim
20 20  |FR3: Multiple Scenarios|✅ **In Scope**|Full|AKEL generates interpretation scenarios
... ... @@ -22,12 +22,12 @@
22 22  |FR5: Evidence Collection|✅ **In Scope**|Full|AKEL searches for evidence
23 23  |FR6: Evidence Evaluation|✅ **In Scope**|Full|AKEL evaluates source reliability
24 24  |FR7: Automated Verdicts|✅ **In Scope**|Full|AKEL computes verdicts with uncertainty
25 -|**QUALITY & RELIABILITY**||||
26 +|**QUALITY & RELIABILITY**||||\\
26 26  |NFR11: Quality Assurance|✅ **In Scope**|**Lite**|**2 gates only** (Gate 1 & 4)
27 27  |NFR1: Performance|⚠️ **Partial**|Basic|Response time monitored, not optimized
28 28  |NFR2: Scalability|⚠️ **Partial**|Single-thread|No concurrent processing
29 29  |NFR3: Reliability|⚠️ **Partial**|Basic|Error handling, no retry logic
30 -|**DEFERRED TO LATER**||||
31 +|**DEFERRED TO LATER**||||\\
31 31  |FR8-FR13|❌ Out of Scope|N/A|User accounts, corrections, publishing
32 32  |FR44-FR53|❌ Out of Scope|N/A|Advanced features (V1.0+)
33 33  |NFR4: Security|❌ Out of Scope|N/A|POC2
... ... @@ -34,6 +34,7 @@
34 34  |NFR5: Maintainability|❌ Out of Scope|N/A|POC2
35 35  |NFR12: Security Controls|❌ Out of Scope|N/A|Beta 0
36 36  |NFR13: Monitoring|❌ Out of Scope|N/A|POC2 == 3. POC Simplifications == === 3.1 FR1: Claim Extraction (Full Implementation) === **Main Requirement:** AI extracts factual claims from input text **POC Implementation:**
38 +
37 37  * ✅ AKEL extracts claims using LLM
38 38  * ✅ Each claim includes original text reference
39 39  * ✅ Claims are identified as factual/non-factual
... ... @@ -101,9 +101,7 @@
101 101  * ⚠️ **Basic error handling** (no advanced retry logic) **Rationale:** POC proves functionality. Performance optimization happens in POC2. **POC Acceptance:**
102 102  * Analysis completes (no timeout requirement)
103 103  * Errors don't crash system
104 -* Basic logging in place == 4. What's NOT in POC Scope == === 4.1 User-Facing Features (Beta 0+) === {{warning}}
105 -**Deferred to Beta 0:**
106 -{{/warning}} **Out of Scope:**
106 +* Basic logging in place == 4. What's NOT in POC Scope == === 4.1 User-Facing Features (Beta 0+) === {{warning}}**Deferred to Beta 0:**{{/warning}} **Out of Scope:**
107 107  * ❌ User accounts and authentication (FR8)
108 108  * ❌ User corrections system (FR9, FR45-46)
109 109  * ❌ Public publishing interface (FR10)
... ... @@ -136,12 +136,9 @@
136 136  * Confidence level (LOW/MEDIUM/HIGH)
137 137  * Reasoning chain **5. Quality Status**
138 138  * Which gates passed/failed
139 -* If failed: Clear explanation why === 5.2 Example POC Output === {{code language="json"}}
140 -{ "claim": { "text": "Switzerland has the highest life expectancy in Europe", "type": "factual", "gate1_status": "PASS" }, "scenarios": [ "Switzerland's overall life expectancy is highest", "Switzerland ranks highest for specific age groups" ], "evidence": { "supporting": [ { "source": "WHO Report 2023", "reliability": 0.95, "excerpt": "Switzerland: 83.4 years average..." } ], "opposing": [ { "source": "Eurostat 2024", "reliability": 0.90, "excerpt": "Spain leads at 83.5 years..." } ] }, "verdict": { "probability": 0.65, "uncertainty": 0.15, "confidence": "MEDIUM", "reasoning": "WHO and Eurostat show similar but conflicting data...", "gate4_status": "PASS" }
141 -}
142 -{{/code}} == 6. Success Criteria == {{success}}
143 -**POC Success Definition:** POC validates that AI can extract claims, find balanced evidence, and compute reasonable verdicts with quality gates improving output quality.
144 -{{/success}} === 6.1 Functional Success === POC is successful if: ✅ **FR1-FR7 Requirements Met:**
139 +* If failed: Clear explanation why === 5.2 Example POC Output === {{code language="json"}}{ "claim": { "text": "Switzerland has the highest life expectancy in Europe", "type": "factual", "gate1_status": "PASS" }, "scenarios": [ "Switzerland's overall life expectancy is highest", "Switzerland ranks highest for specific age groups" ], "evidence": { "supporting": [ { "source": "WHO Report 2023", "reliability": 0.95, "excerpt": "Switzerland: 83.4 years average..." } ], "opposing": [ { "source": "Eurostat 2024", "reliability": 0.90, "excerpt": "Spain leads at 83.5 years..." } ] }, "verdict": { "probability": 0.65, "uncertainty": 0.15, "confidence": "MEDIUM", "reasoning": "WHO and Eurostat show similar but conflicting data...", "gate4_status": "PASS" }
140 +}{{/code}} == 6. Success Criteria == {{success}}**POC Success Definition:** POC validates that AI can extract claims, find balanced evidence, and compute reasonable verdicts with quality gates improving output quality.{{/success}} === 6.1 Functional Success === POC is successful if: ✅ **FR1-FR7 Requirements Met:**
141 +
145 145  1. Extracts 3-5 factual claims from test articles
146 146  2. Generates 2-3 scenarios per ambiguous claim
147 147  3. Finds supporting AND opposing evidence
... ... @@ -153,6 +153,7 @@
153 153  1. Quality gates reduce hallucination rate
154 154  2. Blocked outputs have clear explanations
155 155  3. Quality metrics are logged === 6.2 Quality Thresholds === **Minimum Acceptable:**
153 +
156 156  * ≥70% of test claims correctly classified (factual/non-factual)
157 157  * ≥60% of verdicts are reasonable (human evaluation)
158 158  * Gate 1 blocks 100% of non-factual claims
... ... @@ -172,7 +172,7 @@
172 172  * Quality gates insufficient
173 173  * Alternative approach needed == 7. Test Cases == === 7.1 Happy Path === **Test 1: Simple Factual Claim**
174 174  * Input: "Paris is the capital of France"
175 -* Expected: Factual, 1 scenario, verdict ~95% true **Test 2: Ambiguous Claim**
173 +* Expected: Factual, 1 scenario, verdict 95% true **Test 2: Ambiguous Claim**
176 176  * Input: "Switzerland has the highest income in Europe"
177 177  * Expected: Factual, 2-3 scenarios, verdict with uncertainty **Test 3: Statistical Claim**
178 178  * Input: "10% of people have condition X"
... ... @@ -201,15 +201,13 @@
201 201  * Async API calls
202 202  * Evidence caching
203 203  * Advanced error handling with retry
204 -* Structured logging + monitoring == 9. POC Philosophy == {{info}}
205 -**Important:** POC validates concept, not production readiness. Focus is on proving AI can do the job, with production quality coming in later phases.
206 -{{/info}} === 9.1 Core Principles === **1. Prove Concept, Not Production**
202 +* Structured logging + monitoring == 9. POC Philosophy == {{info}}**Important:** POC validates concept, not production readiness. Focus is on proving AI can do the job, with production quality coming in later phases.{{/info}} === 9.1 Core Principles === **1. Prove Concept, Not Production**
207 207  * POC validates AI can do the job
208 208  * Production quality comes in POC2 and Beta 0
209 209  * Focus on "does it work?" not "is it perfect?" **2. Implement Subset of Requirements**
210 210  * POC covers FR1-7, NFR11 (lite)
211 211  * All other requirements deferred
212 -* Clear mapping to [[Main Requirements>>FactHarbor.Specification.Requirements.WebHome]] **3. Quality Gates Validate Approach**
208 +* Clear mapping to [[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]] **3. Quality Gates Validate Approach**
213 213  * 2 gates prove the concept
214 214  * Remaining 5 gates added in POC2
215 215  * Gates must demonstrably improve quality **4. Iterate Based on Results**
... ... @@ -224,12 +224,11 @@
224 224  * Is the gate approach scalable? ✅ **Production Path:**
225 225  * Is the core architecture sound?
226 226  * What needs improvement for production?
227 -* Is POC2 the right next step? == 10. Related Pages == * **[[Main Requirements>>FactHarbor.Specification.Requirements.WebHome]]** - Full system requirements (this POC implements a subset)
223 +* Is POC2 the right next step? == 10. Related Pages == * **[[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]]** - Full system requirements (this POC implements a subset)
228 228  * **[[POC1 Specification (Detailed)>>FactHarbor.Specification.POC.Specification]]** - Detailed POC1 technical specs
229 229  * **[[POC Summary>>FactHarbor.Specification.POC.Summary]]** - High-level POC overview
230 -* **[[Implementation Roadmap>>FactHarbor.Roadmap.WebHome]]** - POC1, POC2, Beta 0, V1.0 phases
231 -* **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need (drives requirements) **Document Owner:** Technical Team **Review Frequency:** After each POC iteration **Version History:**
226 +* **[[Implementation Roadmap>>Archive.FactHarbor 2026\.01\.20.Roadmap.WebHome]]** - POC1, POC2, Beta 0, V1.0 phases
227 +* **[[User Needs>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.User Needs.WebHome]]** - What users need (drives requirements) **Document Owner:** Technical Team **Review Frequency:** After each POC iteration **Version History:**
232 232  * v1.0 - Initial POC requirements
233 233  * v2.0 - Updated after specification cross-check
234 234  * v3.0 - Aligned with Main Requirements (FR/NFR IDs added)
235 -