Wiki source code of POC Requirements

Last modified by Robert Schaub on 2026/02/08 08:25

version	line-number	content
1.1	1	= POC Requirements =
	2
	3	Status: ✅ Approved for Development
	4	Version: 3.0 (Aligned with Main Requirements)
	5	Goal: Prove that AI can extract claims and determine verdicts automatically without human intervention
	6
	7	{{info}}
1.7	8	Core Philosophy: POC validates the [[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]] through simplified implementation. All POC features map to formal FR/NFR requirements.
1.1	9	{{/info}}
	10
	11
	12	== 1. POC Overview ==
	13
	14	=== 1.1 What POC Tests ===
	15
	16	Core Question:
1.2	17
1.1	18	> Can AI automatically extract factual claims from articles and evaluate them with reasonable verdicts?
	19
	20	What we're proving:
1.2	21
1.1	22	* AI can identify factual claims from text
	23	* AI can evaluate those claims with structured evidence
	24	* Quality gates can filter unreliable outputs
	25	* The core workflow is technically feasible
	26
	27	What we're NOT proving:
1.2	28
1.1	29	* Production-ready reliability (that's POC2)
	30	* User-facing features (that's Beta 0)
	31	* Full IFCN compliance (that's V1.0)
	32
	33	=== 1.2 Requirements Mapping ===
	34
1.7	35	POC1 implements a subset of the full system requirements defined in [[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]].
1.1	36
	37	Scope Summary:
1.2	38
1.1	39	* In Scope: 8 requirements (7 FRs + 1 NFR)
	40	* Partial: 3 NFRs (simplified versions)
	41	* Out of Scope: 19 requirements (deferred to later phases)
	42
	43	== 2. Requirements Scope Matrix ==
	44
	45	{{success}}
1.7	46	Requirements Traceability: This matrix shows which [[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]] are implemented in POC1, providing full traceability between POC and system requirements.
1.1	47	{{/success}}
	48
	49	\|=Requirement\|=POC1 Status\|=Implementation Level\|=Notes
1.2	50	\|CORE WORKFLOW\|\|\|\|\\
1.1	51	\|FR1: Claim Extraction\|✅ In Scope\|Full\|AKEL extracts claims from text
	52	\|FR2: Claim Context\|✅ In Scope\|Basic\|Context preserved with claim
	53	\|FR3: Multiple Scenarios\|✅ In Scope\|Full\|AKEL generates interpretation scenarios
	54	\|FR4: Analysis Summary\|✅ In Scope\|Basic\|Simple summary format
	55	\|FR5: Evidence Collection\|✅ In Scope\|Full\|AKEL searches for evidence
	56	\|FR6: Evidence Evaluation\|✅ In Scope\|Full\|AKEL evaluates source reliability
	57	\|FR7: Automated Verdicts\|✅ In Scope\|Full\|AKEL computes verdicts with uncertainty
1.2	58	\|QUALITY & RELIABILITY\|\|\|\|\\
1.1	59	\|NFR11: Quality Assurance\|✅ In Scope\|Lite\|2 gates only (Gate 1 & 4)
	60	\|NFR1: Performance\|⚠️ Partial\|Basic\|Response time monitored, not optimized
	61	\|NFR2: Scalability\|⚠️ Partial\|Single-thread\|No concurrent processing
	62	\|NFR3: Reliability\|⚠️ Partial\|Basic\|Error handling, no retry logic
1.2	63	\|DEFERRED TO LATER\|\|\|\|\\
1.1	64	\|FR8-FR13\|❌ Out of Scope\|N/A\|User accounts, corrections, publishing
	65	\|FR44-FR53\|❌ Out of Scope\|N/A\|Advanced features (V1.0+)
	66	\|NFR4: Security\|❌ Out of Scope\|N/A\|POC2
	67	\|NFR5: Maintainability\|❌ Out of Scope\|N/A\|POC2
	68	\|NFR12: Security Controls\|❌ Out of Scope\|N/A\|Beta 0
	69	\|NFR13: Monitoring\|❌ Out of Scope\|N/A\|POC2
	70
	71	== 3. POC Simplifications ==
	72
	73	=== 3.1 FR1: Claim Extraction (Full Implementation) ===
	74
	75	Main Requirement: AI extracts factual claims from input text
	76
	77	POC Implementation:
1.2	78
1.1	79	* ✅ AKEL extracts claims using LLM
	80	* ✅ Each claim includes original text reference
	81	* ✅ Claims are identified as factual/non-factual
	82	* ❌ No advanced claim parsing (added in POC2)
	83
	84	Acceptance Criteria:
1.2	85
1.1	86	* Extracts 3-5 claims from typical article
	87	* Identifies factual vs non-factual claims
	88	* Quality Gate 1 validates extraction
	89
	90	=== 3.2 FR3: Multiple Scenarios (Full Implementation) ===
	91
	92	Main Requirement: Generate multiple interpretation scenarios for ambiguous claims
	93
	94	POC Implementation:
1.2	95
1.1	96	* ✅ AKEL generates 2-3 scenarios per claim
	97	* ✅ Scenarios capture different interpretations
	98	* ✅ Each scenario is evaluated separately
	99	* ✅ Verdict considers all scenarios
	100
	101	Acceptance Criteria:
1.2	102
1.1	103	* Generates 2+ scenarios for ambiguous claims
	104	* Scenarios are meaningfully different
	105	* All scenarios are evaluated
	106
	107	=== 3.3 FR4: Analysis Summary (Basic Implementation) ===
	108
	109	Main Requirement: Provide user-friendly summary of analysis
	110
	111	POC Implementation:
1.2	112
1.1	113	* ✅ Simple text summary generated
	114	* ❌ No rich formatting (added in Beta 0)
	115	* ❌ No visual elements (added in Beta 0)
	116	* ❌ No interactive features (added in Beta 0)
	117
	118	POC Format:
	119	```
	120	Claim: [extracted claim]
	121	Scenarios: [list of scenarios]
	122	Evidence: [supporting/opposing evidence]
	123	Verdict: [probability with uncertainty]
	124	```
	125
	126
	127	=== 3.4 FR5-FR6: Evidence Collection & Evaluation (Full Implementation) ===
	128
	129	Main Requirements:
1.2	130
1.1	131	* FR5: Collect supporting and opposing evidence
	132	* FR6: Evaluate evidence source reliability
	133
	134	POC Implementation:
1.2	135
1.1	136	* ✅ AKEL searches for evidence (web/knowledge base)
	137	* ✅ Mandatory contradiction search (finds opposing evidence)
	138	* ✅ Source reliability scoring
	139	* ❌ No evidence deduplication (added in POC2)
	140	* ❌ No advanced source verification (added in POC2)
	141
	142	Acceptance Criteria:
1.2	143
1.1	144	* Finds 2+ supporting evidence items
	145	* Finds 1+ opposing evidence (if exists)
	146	* Sources scored for reliability
	147
	148	=== 3.5 FR7: Automated Verdicts (Full Implementation) ===
	149
	150	Main Requirement: AI computes verdicts with uncertainty quantification
	151
	152	POC Implementation:
1.2	153
1.1	154	* ✅ Probabilistic verdicts (0-100% confidence)
	155	* ✅ Uncertainty explicitly stated
	156	* ✅ Reasoning chain provided
	157	* ✅ Quality Gate 4 validates verdict confidence
	158
	159	POC Output:
	160	```
	161	Verdict: 70% likely true
	162	Uncertainty: ±15% (moderate confidence)
	163	Reasoning: Based on 3 high-quality sources...
	164	Confidence Level: MEDIUM
	165	```
	166
	167	Acceptance Criteria:
1.2	168
1.1	169	* Verdicts include probability (0-100%)
	170	* Uncertainty explicitly quantified
	171	* Reasoning chain explains verdict
	172
	173	=== 3.6 NFR11: Quality Assurance Framework (LITE VERSION) ===
	174
	175	Main Requirement: Complete quality assurance with 7 quality gates
	176
	177	POC Implementation: 2 gates only
	178
	179	Quality Gate 1: Claim Validation
1.2	180
1.1	181	* ✅ Validates claim is factual and verifiable
	182	* ✅ Blocks non-factual claims (opinion/prediction/ambiguous)
	183	* ✅ Provides clear rejection reason
	184
	185	Quality Gate 4: Verdict Confidence Assessment
1.2	186
1.1	187	* ✅ Validates ≥2 sources found
	188	* ✅ Validates quality score ≥0.6
	189	* ✅ Blocks low-confidence verdicts
	190	* ✅ Provides clear rejection reason
	191
	192	Out of Scope (POC2+):
1.2	193
1.1	194	* ❌ Gate 2: Evidence Relevance
	195	* ❌ Gate 3: Scenario Coherence
	196	* ❌ Gate 5: Source Diversity
	197	* ❌ Gate 6: Reasoning Validity
	198	* ❌ Gate 7: Output Completeness
	199
	200	Rationale: Prove gate concept works. Add remaining gates in POC2 after validating approach.
	201
	202
	203	=== 3.7 NFR1-3: Performance, Scalability, Reliability (Basic) ===
	204
	205	Main Requirements:
1.2	206
1.1	207	* NFR1: Response time < 30 seconds
	208	* NFR2: Handle 1000+ concurrent users
	209	* NFR3: 99.9% uptime
	210
	211	POC Implementation:
1.2	212
1.1	213	* ⚠️ Response time monitored (not optimized)
	214	* ⚠️ Single-threaded processing (no concurrency)
	215	* ⚠️ Basic error handling (no advanced retry logic)
	216
	217	Rationale: POC proves functionality. Performance optimization happens in POC2.
	218
	219	POC Acceptance:
1.2	220
1.1	221	* Analysis completes (no timeout requirement)
	222	* Errors don't crash system
	223	* Basic logging in place
	224
	225	== 4. What's NOT in POC Scope ==
	226
	227	=== 4.1 User-Facing Features (Beta 0+) ===
	228
	229	{{warning}}
	230	Deferred to Beta 0:
	231	{{/warning}}
	232
	233	Out of Scope:
1.2	234
1.1	235	* ❌ User accounts and authentication (FR8)
	236	* ❌ User corrections system (FR9, FR45-46)
	237	* ❌ Public publishing interface (FR10)
	238	* ❌ Social sharing (FR11)
	239	* ❌ Email notifications (FR12)
	240	* ❌ API access (FR13)
	241
	242	Rationale: POC validates AI capabilities. User features added in Beta 0.
	243
	244
	245	=== 4.2 Advanced Features (V1.0+) ===
	246
	247	Out of Scope:
1.2	248
1.1	249	* ❌ IFCN compliance (FR47)
	250	* ❌ ClaimReview schema (FR48)
	251	* ❌ Archive.org integration (FR49)
	252	* ❌ OSINT toolkit (FR50)
	253	* ❌ Video verification (FR51)
	254	* ❌ Deepfake detection (FR52)
	255	* ❌ Cross-org sharing (FR53)
	256
	257	Rationale: Advanced features require proven platform. Added post-V1.0.
	258
	259
	260	=== 4.3 Production Requirements (POC2, Beta 0) ===
	261
	262	Out of Scope:
1.2	263
1.1	264	* ❌ Security controls (NFR4, NFR12)
	265	* ❌ Code maintainability (NFR5)
	266	* ❌ System monitoring (NFR13)
	267	* ❌ Evidence deduplication
	268	* ❌ Advanced source verification
	269	* ❌ Full 7-gate quality framework
	270
	271	Rationale: POC proves concept. Production hardening happens in POC2 and Beta 0.
	272
	273
	274	== 5. POC Output Specification ==
	275
	276	=== 5.1 Required Output Elements ===
	277
	278	For each analyzed claim, POC must produce:
	279
1.3	280	* \\
	281	** \\
1.2	282	**1. Claim
1.1	283	* Original text
	284	* Classification (factual/non-factual/ambiguous)
	285	* If non-factual: Clear reason why
	286
	287	2. Scenarios (if factual)
1.2	288
1.1	289	* 2-3 interpretation scenarios
	290	* Each scenario clearly described
	291
	292	3. Evidence (if factual)
1.2	293
1.1	294	* Supporting evidence (2+ items)
	295	* Opposing evidence (if exists)
	296	* Source URLs and reliability scores
	297
	298	4. Verdict (if factual)
1.2	299
1.1	300	* Probability (0-100%)
	301	* Uncertainty quantification
	302	* Confidence level (LOW/MEDIUM/HIGH)
	303	* Reasoning chain
	304
	305	5. Quality Status
1.2	306
1.1	307	* Which gates passed/failed
	308	* If failed: Clear explanation why
	309
	310	=== 5.2 Example POC Output ===
	311
	312	{{code language="json"}}
	313	{
	314	"claim": {
	315	"text": "Switzerland has the highest life expectancy in Europe",
	316	"type": "factual",
	317	"gate1_status": "PASS"
	318	},
	319	"scenarios": [
	320	"Switzerland's overall life expectancy is highest",
	321	"Switzerland ranks highest for specific age groups"
	322	],
	323	"evidence": {
	324	"supporting": [
	325	{
	326	"source": "WHO Report 2023",
	327	"reliability": 0.95,
	328	"excerpt": "Switzerland: 83.4 years average..."
	329	}
	330	],
	331	"opposing": [
	332	{
	333	"source": "Eurostat 2024",
	334	"reliability": 0.90,
	335	"excerpt": "Spain leads at 83.5 years..."
	336	}
	337	]
	338	},
	339	"verdict": {
	340	"probability": 0.65,
	341	"uncertainty": 0.15,
	342	"confidence": "MEDIUM",
	343	"reasoning": "WHO and Eurostat show similar but conflicting data...",
	344	"gate4_status": "PASS"
	345	}
	346	}
	347	{{/code}}
	348
	349
	350	== 6. Success Criteria ==
	351
	352	{{success}}
	353	POC Success Definition: POC validates that AI can extract claims, find balanced evidence, and compute reasonable verdicts with quality gates improving output quality.
	354	{{/success}}
	355
	356	=== 6.1 Functional Success ===
	357
	358	POC is successful if:
	359
	360	✅ FR1-FR7 Requirements Met:
1.2	361
1.1	362	1. Extracts 3-5 factual claims from test articles
	363	2. Generates 2-3 scenarios per ambiguous claim
	364	3. Finds supporting AND opposing evidence
	365	4. Computes probabilistic verdicts with uncertainty
	366	5. Provides clear reasoning chains
	367
	368	✅ Quality Gates Work:
1.2	369
1.1	370	1. Gate 1 blocks non-factual claims (100% block rate)
	371	2. Gate 4 blocks low-quality verdicts (blocks if <2 sources or quality <0.6)
	372	3. Clear rejection reasons provided
	373
	374	✅ NFR11 Met:
1.2	375
1.1	376	1. Quality gates reduce hallucination rate
	377	2. Blocked outputs have clear explanations
	378	3. Quality metrics are logged
	379
	380	=== 6.2 Quality Thresholds ===
	381
	382	Minimum Acceptable:
1.2	383
1.1	384	* ≥70% of test claims correctly classified (factual/non-factual)
	385	* ≥60% of verdicts are reasonable (human evaluation)
	386	* Gate 1 blocks 100% of non-factual claims
	387	* Gate 4 blocks verdicts with <2 sources
	388
	389	Target:
1.2	390
1.1	391	* ≥80% claims correctly classified
	392	* ≥75% verdicts are reasonable
	393	* <10% false positives (blocking good claims)
	394
	395	=== 6.3 POC Decision Gate ===
	396
	397	After POC1, we decide:
	398
	399	✅ PROCEED to POC2 if:
1.2	400
1.1	401	* Success criteria met
	402	* Quality gates demonstrably improve output
	403	* Core workflow is technically sound
	404	* Clear path to production quality
	405
	406	⚠️ ITERATE POC1 if:
1.2	407
1.1	408	* Success criteria partially met
	409	* Gates work but need tuning
	410	* Core issues identified but fixable
	411
	412	❌ PIVOT APPROACH if:
1.2	413
1.1	414	* Success criteria not met
	415	* Fundamental AI limitations discovered
	416	* Quality gates insufficient
	417	* Alternative approach needed
	418
	419	== 7. Test Cases ==
	420
	421	=== 7.1 Happy Path ===
	422
	423	Test 1: Simple Factual Claim
1.2	424
1.1	425	* Input: "Paris is the capital of France"
1.2	426	* Expected: Factual, 1 scenario, verdict 95% true
1.1	427
	428	Test 2: Ambiguous Claim
1.2	429
1.1	430	* Input: "Switzerland has the highest income in Europe"
	431	* Expected: Factual, 2-3 scenarios, verdict with uncertainty
	432
	433	Test 3: Statistical Claim
1.2	434
1.1	435	* Input: "10% of people have condition X"
	436	* Expected: Factual, evidence with numbers, probabilistic verdict
	437
	438	=== 7.2 Edge Cases ===
	439
	440	Test 4: Opinion
1.2	441
1.1	442	* Input: "Paris is the best city"
	443	* Expected: Non-factual (opinion), blocked by Gate 1
	444
	445	Test 5: Prediction
1.2	446
1.1	447	* Input: "Bitcoin will reach $100,000 next year"
	448	* Expected: Non-factual (prediction), blocked by Gate 1
	449
	450	Test 6: Insufficient Evidence
1.2	451
1.1	452	* Input: Obscure factual claim with no sources
	453	* Expected: Blocked by Gate 4 (<2 sources)
	454
	455	=== 7.3 Quality Gate Tests ===
	456
	457	Test 7: Gate 1 Effectiveness
1.2	458
1.1	459	* Input: Mix of 10 factual + 10 non-factual claims
	460	* Expected: Gate 1 blocks all 10 non-factual (100% precision)
	461
	462	Test 8: Gate 4 Effectiveness
1.2	463
1.1	464	* Input: Claims with varying evidence availability
	465	* Expected: Gate 4 blocks low-confidence verdicts
	466
	467	== 8. Technical Architecture (POC) ==
	468
	469	=== 8.1 Simplified Architecture ===
	470
	471	POC Tech Stack:
1.2	472
1.1	473	* Frontend: Simple web interface (Next.js + TypeScript)
	474	* Backend: Single API endpoint
	475	* AI: Claude API (Sonnet 4.5)
	476	* Database: Local JSON files (no database)
	477	* Deployment: Single server
	478
	479	Architecture Diagram: See [[POC1 Specification>>FactHarbor.Specification.POC.Specification]]
	480
	481
	482	=== 8.2 AKEL Implementation ===
	483
	484	POC AKEL:
1.2	485
1.1	486	* Single-threaded processing
	487	* Synchronous API calls
	488	* No caching
	489	* Basic error handling
	490	* Console logging
	491
	492	Full AKEL (POC2+):
1.2	493
1.1	494	* Multi-threaded processing
	495	* Async API calls
	496	* Evidence caching
	497	* Advanced error handling with retry
	498	* Structured logging + monitoring
	499
	500	== 9. POC Philosophy ==
	501
	502	{{info}}
	503	Important: POC validates concept, not production readiness. Focus is on proving AI can do the job, with production quality coming in later phases.
	504	{{/info}}
	505
	506	=== 9.1 Core Principles ===
	507
1.3	508	* \\
	509	** \\
1.2	510	**1. Prove Concept, Not Production
1.1	511	* POC validates AI can do the job
	512	* Production quality comes in POC2 and Beta 0
	513	* Focus on "does it work?" not "is it perfect?"
	514
	515	2. Implement Subset of Requirements
1.2	516
1.1	517	* POC covers FR1-7, NFR11 (lite)
	518	* All other requirements deferred
1.7	519	* Clear mapping to [[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]]
1.1	520
	521	3. Quality Gates Validate Approach
1.2	522
1.1	523	* 2 gates prove the concept
	524	* Remaining 5 gates added in POC2
	525	* Gates must demonstrably improve quality
	526
	527	4. Iterate Based on Results
1.2	528
1.1	529	* POC results determine next steps
	530	* Decision gate after POC1
	531	* Flexibility to pivot if needed
	532
1.2	533	=== 9.2 Success ===
1.1	534
1.2	535	Clear Path Forward ===
1.1	536
	537	POC succeeds if we can confidently answer:
	538
	539	✅ Technical Feasibility:
1.2	540
1.1	541	* Can AI extract claims reliably?
	542	* Can AI find balanced evidence?
	543	* Can AI compute reasonable verdicts?
	544
	545	✅ Quality Approach:
1.2	546
1.1	547	* Do quality gates improve output?
	548	* Can we measure and track quality?
	549	* Is the gate approach scalable?
	550
	551	✅ Production Path:
1.2	552
1.1	553	* Is the core architecture sound?
	554	* What needs improvement for production?
	555	* Is POC2 the right next step?
	556
	557	== 10. Related Pages ==
	558
1.7	559	* [[Main Requirements>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]] - Full system requirements (this POC implements a subset)
1.1	560	* [[POC1 Specification (Detailed)>>FactHarbor.Specification.POC.Specification]] - Detailed POC1 technical specs
	561	* [[POC Summary>>FactHarbor.Specification.POC.Summary]] - High-level POC overview
1.5	562	* [[Implementation Roadmap>>Archive.FactHarbor 2026\.01\.20.Roadmap.WebHome]] - POC1, POC2, Beta 0, V1.0 phases
1.6	563	* [[User Needs>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.User Needs.WebHome]] - What users need (drives requirements)
1.1	564
	565	Document Owner: Technical Team
	566	Review Frequency: After each POC iteration
	567	Version History:
1.2	568
1.1	569	* v1.0 - Initial POC requirements
	570	* v2.0 - Updated after specification cross-check
	571	* v3.0 - Aligned with Main Requirements (FR/NFR IDs added)