Skip to Content

Wiki source code of Requirements

Version 1.1 by Robert Schaub on 2025/12/24 11:54
Show last authors
1 = Requirements =
2
3
4 {{info}}
5 **Phase Assignments:** See [[Requirements Roadmap Matrix>>Test.FactHarbor.Roadmap.Requirements-Roadmap-Matrix.WebHome]] for which requirements are implemented in which phases.
6 {{/info}}
7
8 **This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.**
9
10 **Core Philosophy:** Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically.
11
12 == Navigation ==
13
14 * **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need from FactHarbor (drives these requirements)
15 * **This page** - How we fulfill those needs through system design
16
17 (% class="box infomessage" %)
18 (((
19 **How to read this page:**
20
21 1. **User Needs drive Requirements**: See [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] for what users need
22 2. **Requirements define implementation**: This page shows how we fulfill those needs
23 3. **Functional Requirements (FR)**: Specific features and capabilities
24 4. **Non-Functional Requirements (NFR)**: Quality attributes (performance, security, etc.)
25
26 Each requirement references which User Needs it fulfills.
27 )))
28
29 == 1. Roles ==
30
31 **Fulfills**: UN-12 (Submit claims), UN-13 (Cite verdicts), UN-14 (API access)
32
33 FactHarbor uses three simple roles plus a reputation system.
34
35 === 1.1 Reader ===
36
37 **Who**: Anyone (no login required)
38
39 **Can**:
40 * Browse and search claims
41 * View scenarios, evidence, verdicts, and confidence scores
42 * Flag issues or errors
43 * Use filters, search, and visualization tools
44 * Submit claims automatically (new claims added if not duplicates)
45
46 **Cannot**:
47 * Modify content
48 * Access edit history details
49
50 **User Needs served**: UN-1 (Trust assessment), UN-2 (Claim verification), UN-3 (Article summary with FactHarbor analysis summary), UN-4 (Social media fact-checking), UN-5 (Source tracing), UN-7 (Evidence transparency), UN-8 (Understanding disagreement), UN-12 (Submit claims), UN-17 (In-article highlighting)
51
52 === 1.2 Contributor ===
53
54 **Who**: Registered users (earns reputation through contributions)
55
56 **Can**:
57 * Everything a Reader can do
58 * Edit claims, evidence, and scenarios
59 * Add sources and citations
60 * Suggest improvements to AI-generated content
61 * Participate in discussions
62 * Earn reputation points for quality contributions
63
64 **Reputation System**:
65 * New contributors: Limited edit privileges
66 * Established contributors (established reputation): Full edit access
67 * Trusted contributors (substantial reputation): Can approve certain changes
68 * Reputation earned through: Accepted edits, helpful flags, quality contributions
69 * Reputation lost through: Reverted edits, invalid flags, abuse
70
71 **Cannot**:
72 * Delete or hide content (only moderators)
73 * Override moderation decisions
74
75 **User Needs served**: UN-13 (Cite and contribute)
76
77 === 1.3 Moderator ===
78
79 **Who**: Trusted community members with proven track record, appointed by governance board
80
81 **Can**:
82 * Review flagged content
83 * Hide harmful or abusive content
84 * Resolve disputes between contributors
85 * Issue warnings or temporary bans
86 * Make final decisions on content disputes
87 * Access full audit logs
88
89 **Cannot**:
90 * Change governance rules
91 * Permanently ban users without board approval
92 * Override technical quality gates
93
94 **Note**: Small team (3-5 initially), supported by automated moderation tools.
95
96 === 1.4 Domain Trusted Contributors (Optional, Task-Specific) ===
97
98 **Who**: Subject matter specialists invited for specific high-stakes disputes
99
100 **Not a permanent role**: Contacted externally when needed for contested claims in their domain
101
102 **When used**:
103 * Medical claims with life/safety implications
104 * Legal interpretations with significant impact
105 * Scientific claims with high controversy
106 * Technical claims requiring specialized knowledge
107
108 **Process**:
109 * Moderator identifies need for expert input
110 * Contact expert externally (don't require them to be users)
111 * Trusted Contributor provides written opinion with sources
112 * Opinion added to claim record
113 * Trusted Contributor acknowledged in claim
114
115 **User Needs served**: UN-16 (Expert validation status)
116
117 == 2. Content States ==
118
119 **Fulfills**: UN-1 (Trust indicators), UN-16 (Review status transparency)
120
121 FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping.
122
123 === 2.1 Published ===
124
125 **Status**: Visible to all users
126
127 **Includes**:
128 * AI-generated analyses (default state)
129 * User-contributed content
130 * Edited/improved content
131
132 **Quality Indicators** (displayed with content):
133 * **Confidence Score**: 0-100% (AI's confidence in analysis)
134 * **Source Quality Score**: 0-100% (based on source track record)
135 * **Controversy Flag**: If high dispute/edit activity
136 * **Completeness Score**: % of expected fields filled
137 * **Last Updated**: Date of most recent change
138 * **Edit Count**: Number of revisions
139 * **Review Status**: AI-generated / Human-reviewed / Expert-validated
140
141 **Automatic Warnings**:
142 * Confidence < 60%: "Low confidence - use caution"
143 * Source quality < 40%: "Sources may be unreliable"
144 * High controversy: "Disputed - multiple interpretations exist"
145 * Medical/Legal/Safety domain: "Seek professional advice"
146
147 **User Needs served**: UN-1 (Trust score), UN-9 (Methodology transparency), ~~UN-15 (Evolution timeline - Deferred)~~, UN-16 (Review status)
148
149 === 2.2 Hidden ===
150
151 **Status**: Not visible to regular users (only to moderators)
152
153 **Reasons**:
154 * Spam or advertising
155 * Personal attacks or harassment
156 * Illegal content
157 * Privacy violations
158 * Deliberate misinformation (verified)
159 * Abuse or harmful content
160
161 **Process**:
162 * Automated detection flags for moderator review
163 * Moderator confirms and hides
164 * Original author notified with reason
165 * Can appeal to board if disputes moderator decision
166
167 **Note**: Content is hidden, not deleted (for audit trail)
168
169 == 3. Contribution Rules ==
170
171 === 3.1 All Contributors Must ===
172
173 * Provide sources for factual claims
174 * Use clear, neutral language in FactHarbor's own summaries
175 * Respect others and maintain civil discourse
176 * Accept community feedback constructively
177 * Focus on improving quality, not protecting ego
178
179 === 3.2 AKEL (AI System) ===
180
181 **AKEL is the primary system**. Human contributions supplement and train AKEL.
182
183 **AKEL Must**:
184 * Mark all outputs as AI-generated
185 * Display confidence scores prominently
186 * Provide source citations
187 * Flag uncertainty clearly
188 * Identify contradictions in evidence
189 * Learn from human corrections
190
191 **When AKEL Makes Errors**:
192 1. Capture the error pattern (what, why, how common)
193 2. Improve the system (better prompt, model, validation)
194 3. Re-process affected claims automatically
195 4. Measure improvement (did quality increase?)
196
197 **Human Role**: Train AKEL through corrections, not replace AKEL
198
199 === 3.3 Contributors Should ===
200
201 * Improve clarity and structure
202 * Add missing sources
203 * Flag errors for system improvement
204 * Suggest better ways to present information
205 * Participate in quality discussions
206
207 === 3.4 Moderators Must ===
208
209 * Be impartial
210 * Document moderation decisions
211 * Respond to appeals promptly
212 * Use automated tools to scale efforts
213 * Focus on abuse/harm, not routine quality control
214
215 == 4. Quality Standards ==
216
217 **Fulfills**: UN-5 (Source reliability), UN-6 (Publisher track records), UN-7 (Evidence transparency), UN-9 (Methodology transparency)
218
219 === 4.1 Source Requirements ===
220
221 **Track Record Over Credentials**:
222 * Sources evaluated by historical accuracy
223 * Correction policy matters
224 * Independence from conflicts of interest
225 * Methodology transparency
226
227 **Source Quality Database**:
228 * Automated tracking of source accuracy
229 * Correction frequency
230 * Reliability score (updated continuously)
231 * Users can see source track record
232
233 **No automatic trust** for government, academia, or media - all evaluated by track record.
234
235 **User Needs served**: UN-5 (Source provenance), UN-6 (Publisher reliability)
236
237 === 4.2 Claim Requirements ===
238
239 * Clear subject and assertion
240 * Verifiable with available information
241 * Sourced (or explicitly marked as needing sources)
242 * Neutral language in FactHarbor summaries
243 * Appropriate context provided
244
245 **User Needs served**: UN-2 (Claim extraction and verification)
246
247 === 4.3 Evidence Requirements ===
248
249 * Publicly accessible (or explain why not)
250 * Properly cited with attribution
251 * Relevant to claim being evaluated
252 * Original source preferred over secondary
253
254 **User Needs served**: UN-7 (Evidence transparency)
255
256 === 4.4 Confidence Scoring ===
257
258 **Automated confidence calculation based on**:
259 * Source quality scores
260 * Evidence consistency
261 * Contradiction detection
262 * Completeness of analysis
263 * Historical accuracy of similar claims
264
265 **Thresholds**:
266 * < 40%: Too low to publish (needs improvement)
267 * 40-60%: Published with "Low confidence" warning
268 * 60-80%: Published as standard
269 * 80-100%: Published as "High confidence"
270
271 **User Needs served**: UN-1 (Trust assessment), UN-9 (Methodology transparency)
272
273 == 5. Automated Risk Scoring ==
274
275 **Fulfills**: UN-10 (Manipulation detection), UN-16 (Appropriate review level)
276
277 **Replace manual risk tiers with continuous automated scoring**.
278
279 === 5.1 Risk Score Calculation ===
280
281 **Factors** (weighted algorithm):
282 * **Domain sensitivity**: Medical, legal, safety auto-flagged higher
283 * **Potential impact**: Views, citations, spread
284 * **Controversy level**: Flags, disputes, edit wars
285 * **Uncertainty**: Low confidence, contradictory evidence
286 * **Source reliability**: Track record of sources used
287
288 **Score**: 0-100 (higher = more risk)
289
290 === 5.2 Automated Actions ===
291
292 * **Score > 80**: Flag for moderator review before publication
293 * **Score 60-80**: Publish with prominent warnings
294 * **Score 40-60**: Publish with standard warnings
295 * **Score < 40**: Publish normally
296
297 **Continuous monitoring**: Risk score recalculated as new information emerges
298
299 **User Needs served**: UN-10 (Detect manipulation tactics), UN-16 (Review status)
300
301 == 6. System Improvement Process ==
302
303 **Core principle**: Fix the system, not just the data.
304
305 === 6.1 Error Capture ===
306
307 **When users flag errors or make corrections**:
308 1. What was wrong? (categorize)
309 2. What should it have been?
310 3. Why did the system fail? (root cause)
311 4. How common is this pattern?
312 5. Store in ErrorPattern table (improvement queue)
313
314 === 6.2 Continuous Improvement Cycle ===
315
316 1. **Review**: Analyze top error patterns
317 2. **Develop**: Create fix (prompt, model, validation)
318 3. **Test**: Validate fix on sample claims
319 4. **Deploy**: Roll out if quality improves
320 5. **Re-process**: Automatically update affected claims
321 6. **Monitor**: Track quality metrics
322
323 === 6.3 Quality Metrics Dashboard ===
324
325 **Track continuously**:
326 * Error rate by category
327 * Source quality distribution
328 * Confidence score trends
329 * User flag rate (issues found)
330 * Correction acceptance rate
331 * Re-work rate
332 * Claims processed per hour
333
334 **Goal**: continuous improvement in error rate
335
336 == 7. Automated Quality Monitoring ==
337
338 **Replace manual audit sampling with automated monitoring**.
339
340 === 7.1 Continuous Metrics ===
341
342 * **Source quality**: Track record database
343 * **Consistency**: Contradiction detection
344 * **Clarity**: Readability scores
345 * **Completeness**: Field validation
346 * **Accuracy**: User corrections tracked
347
348 === 7.2 Anomaly Detection ===
349
350 **Automated alerts for**:
351 * Sudden quality drops
352 * Unusual patterns
353 * Contradiction clusters
354 * Source reliability changes
355 * User behavior anomalies
356
357 === 7.3 Targeted Review ===
358
359 * Review only flagged items
360 * Random sampling for calibration (not quotas)
361 * Learn from corrections to improve automation
362
363 == 8. Functional Requirements ==
364
365 This section defines specific features that fulfill user needs.
366
367 === 8.1 Claim Intake & Normalization ===
368
369 ==== FR1 — Claim Intake ====
370
371 **Fulfills**: UN-2 (Claim extraction), UN-4 (Quick fact-checking), UN-12 (Submit claims)
372
373 * Users submit claims via simple form or API
374 * Claims can be text, URL, or image
375 * Duplicate detection (semantic similarity)
376 * Auto-categorization by domain
377
378 ==== FR2 — Claim Normalization ====
379
380 **Fulfills**: UN-2 (Claim verification)
381
382 * Standardize to clear assertion format
383 * Extract key entities (who, what, when, where)
384 * Identify claim type (factual, predictive, evaluative)
385 * Link to existing similar claims
386
387 ==== FR3 — Claim Classification ====
388
389 **Fulfills**: UN-11 (Filtered research)
390
391 * Domain: Politics, Science, Health, etc.
392 * Type: Historical fact, current stat, prediction, etc.
393 * Risk score: Automated calculation
394 * Complexity: Simple, moderate, complex
395
396 === 8.2 Scenario System ===
397
398 ==== FR4 — Scenario Generation ====
399
400 **Fulfills**: UN-2 (Context-dependent verification), UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
401
402 **Automated scenario creation**:
403 * AKEL analyzes claim and generates likely scenarios (use-cases and contexts)
404 * Each scenario includes: assumptions, definitions, boundaries, evidence context
405 * Users can flag incorrect scenarios
406 * System learns from corrections
407
408 **Key Concept**: Scenarios represent different interpretations or contexts (e.g., "Clinical trials with healthy adults" vs. "Real-world data with diverse populations")
409
410 ==== FR5 — Evidence Linking ====
411
412 **Fulfills**: UN-5 (Source tracing), UN-7 (Evidence transparency)
413
414 * Automated evidence discovery from sources
415 * Relevance scoring
416 * Contradiction detection
417 * Source quality assessment
418
419 ==== FR6 — Scenario Comparison ====
420
421 **Fulfills**: UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
422
423 * Side-by-side comparison interface
424 * Highlight key differences between scenarios
425 * Show evidence supporting each scenario
426 * Display confidence scores per scenario
427
428 === 8.3 Verdicts & Analysis ===
429
430 ==== FR7 — Automated Verdicts ====
431
432 **Fulfills**: UN-1 (Trust score), UN-2 (Verification verdicts), UN-3 (Article summary with FactHarbor analysis summary), UN-13 (Cite verdicts)
433
434 * AKEL generates verdict based on evidence within each scenario
435 * **Likelihood range** displayed (e.g., "0.70-0.85 (likely true)") - NOT binary true/false
436 * **Uncertainty factors** explicitly listed (e.g., "Small sample sizes", "Long-term effects unknown")
437 * Confidence score displayed prominently
438 * Source quality indicators shown
439 * Contradictions noted
440 * Uncertainty acknowledged
441
442 **Key Innovation**: Detailed probabilistic verdicts with explicit uncertainty, not binary judgments
443
444 ==== FR8 — Time Evolution ====
445
446 {{warning}}
447 **Status:** Deferred (Not in V1.0)
448
449 This requirement has been **dropped from the current architecture and design**. Versioned entities have been replaced with simple edit history tracking only. Full evolution timeline functionality is deferred to future releases beyond V1.0.
450 {{/warning}}
451
452
453
454 **Fulfills**: UN-15 (Verdict evolution timeline)
455
456 * Claims and verdicts update as new evidence emerges
457 * Version history maintained for all verdicts
458 * Changes highlighted
459 * Confidence score trends visible
460 * Users can see "as of date X, what did we know?"
461
462 === 8.4 User Interface & Presentation ===
463
464 ==== FR12 — Two-Panel Summary View (Article Summary with FactHarbor Analysis Summary) ====
465
466 **Fulfills**: UN-3 (Article Summary with FactHarbor Analysis Summary)
467
468 **Purpose**: Provide side-by-side comparison of what a document claims vs. FactHarbor's complete analysis of its credibility
469
470 **Left Panel: Article Summary**:
471 * Document title, source, and claimed credibility
472 * "The Big Picture" - main thesis or position change
473 * "Key Findings" - structured summary of document's main claims
474 * "Reasoning" - document's explanation for positions
475 * "Conclusion" - document's bottom line
476
477 **Right Panel: FactHarbor Analysis Summary**:
478 * FactHarbor's independent source credibility assessment
479 * Claim-by-claim verdicts with confidence scores
480 * Methodology assessment (strengths, limitations)
481 * Overall verdict on document quality
482 * Analysis ID for reference
483
484 **Design Principles**:
485 * No scrolling required - both panels visible simultaneously
486 * Visual distinction between "what they say" and "FactHarbor's analysis"
487 * Color coding for verdicts (supported, uncertain, refuted)
488 * Confidence percentages clearly visible
489 * Mobile responsive (panels stack vertically on small screens)
490
491 **Implementation Notes**:
492 * Generated automatically by AKEL for every analyzed document
493 * Updates when verdict evolves (maintains version history)
494 * Exportable as standalone summary report
495 * Shareable via permanent URL
496
497 ==== FR13 — In-Article Claim Highlighting ====
498
499 **Fulfills**: UN-17 (In-article claim highlighting)
500
501 **Purpose**: Enable readers to quickly assess claim credibility while reading by visually highlighting factual claims with color-coded indicators
502
503 ==== Visual Example: Article with Highlighted Claims ====
504
505 (% class="box" %)
506 (((
507 **Article: "New Study Shows Benefits of Mediterranean Diet"**
508
509 A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.
510
511 (% class="box successmessage" style="margin:10px 0;" %)
512 (((
513 🟢 **Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups**
514
515 (% style="font-size:0.9em; color:#666;" %)
516 ↑ WELL SUPPORTED • 87% confidence
517 [[Click for evidence details →]]
518 (%%)
519 )))
520
521 The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
522
523 (% class="box warningmessage" style="margin:10px 0;" %)
524 (((
525 🟡 **Some experts believe this diet can completely prevent heart attacks**
526
527 (% style="font-size:0.9em; color:#666;" %)
528 ↑ UNCERTAIN • 45% confidence
529 Overstated - evidence shows risk reduction, not prevention
530 [[Click for details →]]
531 (%%)
532 )))
533
534 Dr. Maria Rodriguez, lead researcher, recommends incorporating more olive oil, fish, and vegetables into daily meals.
535
536 (% class="box errormessage" style="margin:10px 0;" %)
537 (((
538 🔴 **The study proves that saturated fats cause heart disease**
539
540 (% style="font-size:0.9em; color:#666;" %)
541 ↑ REFUTED • 15% confidence
542 Claim not supported by study design; correlation ≠ causation
543 [[Click for counter-evidence →]]
544 (%%)
545 )))
546
547 Participants also reported feeling more energetic and experiencing better sleep quality, though these were secondary measures.
548 )))
549
550 **Legend:**
551 * 🟢 = Well-supported claim (confidence ≥75%)
552 * 🟡 = Uncertain claim (confidence 40-74%)
553 * 🔴 = Refuted/unsupported claim (confidence <40%)
554 * Plain text = Non-factual content (context, opinions, recommendations)
555
556 ==== Tooltip on Hover/Click ====
557
558 (% class="box infomessage" %)
559 (((
560 **FactHarbor Analysis**
561
562 **Claim:**
563 "Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease"
564
565 **Verdict:** WELL SUPPORTED
566 **Confidence:** 87%
567
568 **Evidence Summary:**
569 * Meta-analysis of 12 RCTs confirms 23-28% risk reduction
570 * Consistent findings across multiple populations
571 * Published in peer-reviewed journal (high credibility)
572
573 **Uncertainty Factors:**
574 * Exact percentage varies by study (20-30% range)
575
576 [[View Full Analysis →]]
577 )))
578
579 **Color-Coding System**:
580 * **Green**: Well-supported claims (confidence ≥75%, strong evidence)
581 * **Yellow/Orange**: Uncertain claims (confidence 40-74%, conflicting or limited evidence)
582 * **Red**: Refuted or unsupported claims (confidence <40%, contradicted by evidence)
583 * **Gray/Neutral**: Non-factual content (opinions, questions, procedural text)
584
585 ==== Interactive Highlighting Example (Detailed View) ====
586
587 (% style="width:100%; border-collapse:collapse;" %)
588 |=**Article Text**|=**Status**|=**Analysis**
589 |(((A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Context - no highlighting
590 |(((//Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups//)))|(% style="background-color:#D4EDDA; text-align:center; padding:8px;" %)🟢 **WELL SUPPORTED**|(((
591 **87% confidence**
592
593 Meta-analysis of 12 RCTs confirms 23-28% risk reduction
594
595 [[View Full Analysis]]
596 )))
597 |(((The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Methodology - no highlighting
598 |(((//Some experts believe this diet can completely prevent heart attacks//)))|(% style="background-color:#FFF3CD; text-align:center; padding:8px;" %)🟡 **UNCERTAIN**|(((
599 **45% confidence**
600
601 Overstated - evidence shows risk reduction, not prevention
602
603 [[View Details]]
604 )))
605 |(((Dr. Rodriguez recommends incorporating more olive oil, fish, and vegetables into daily meals.)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Recommendation - no highlighting
606 |(((//The study proves that saturated fats cause heart disease//)))|(% style="background-color:#F8D7DA; text-align:center; padding:8px;" %)🔴 **REFUTED**|(((
607 **15% confidence**
608
609 Claim not supported by study; correlation ≠ causation
610
611 [[View Counter-Evidence]]
612 )))
613
614 **Design Notes:**
615 * Highlighted claims use italics to distinguish from plain text
616 * Color backgrounds match XWiki message box colors (success/warning/error)
617 * Status column shows verdict prominently
618 * Analysis column provides quick summary with link to details
619
620 **User Actions**:
621 * **Hover** over highlighted claim → Tooltip appears
622 * **Click** highlighted claim → Detailed analysis modal/panel
623 * **Toggle** button to turn highlighting on/off
624 * **Keyboard**: Tab through highlighted claims
625
626 **Interaction Design**:
627 * Hover/click on highlighted claim → Show tooltip with:
628 * Claim text
629 * Verdict (e.g., "WELL SUPPORTED")
630 * Confidence score (e.g., "85%")
631 * Brief evidence summary
632 * Link to detailed analysis
633 * Toggle highlighting on/off (user preference)
634 * Adjustable color intensity for accessibility
635
636 **Technical Requirements**:
637 * Real-time highlighting as page loads (non-blocking)
638 * Claim boundary detection (start/end of assertion)
639 * Handle nested or overlapping claims
640 * Preserve original article formatting
641 * Work with various content formats (HTML, plain text, PDFs)
642
643 **Performance Requirements**:
644 * Highlighting renders within 500ms of page load
645 * No perceptible delay in reading experience
646 * Efficient DOM manipulation (avoid reflows)
647
648 **Accessibility**:
649 * Color-blind friendly palette (use patterns/icons in addition to color)
650 * Screen reader compatible (ARIA labels for claim credibility)
651 * Keyboard navigation to highlighted claims
652
653 **Implementation Notes**:
654 * Claims extracted and analyzed by AKEL during initial processing
655 * Highlighting data stored as annotations with byte offsets
656 * Client-side rendering of highlights based on verdict data
657 * Mobile responsive (tap instead of hover)
658
659 === 8.5 Workflow & Moderation ===
660
661 ==== FR9 — Publication Workflow ====
662
663 **Fulfills**: UN-1 (Fast access to verified content), UN-16 (Clear review status)
664
665 **Simple flow**:
666 1. Claim submitted
667 2. AKEL processes (automated)
668 3. If confidence > threshold: Publish (labeled as AI-generated)
669 4. If confidence < threshold: Flag for improvement
670 5. If risk score > threshold: Flag for moderator
671
672 **No multi-stage approval process**
673
674 ==== FR10 — Moderation ====
675
676 **Focus on abuse, not routine quality**:
677 * Automated abuse detection
678 * Moderators handle flags
679 * Quick response to harmful content
680 * Minimal involvement in routine content
681
682 ==== FR11 — Audit Trail ====
683
684 **Fulfills**: UN-14 (API access to histories), UN-15 (Evolution tracking)
685
686 * All edits logged
687 * Version history public
688 * Moderation decisions documented
689 * System improvements tracked
690
691 == 9. Non-Functional Requirements ==
692
693 === 9.1 NFR1 — Performance ===
694
695 **Fulfills**: UN-4 (Fast fact-checking), UN-11 (Responsive filtering)
696
697 * Claim processing: < 30 seconds
698 * Search response: < 2 seconds
699 * Page load: < 3 seconds
700 * 99% uptime
701
702 === 9.2 NFR2 — Scalability ===
703
704 **Fulfills**: UN-14 (API access at scale)
705
706 * Handle 10,000 claims initially
707 * Scale to 1M+ claims
708 * Support 100K+ concurrent users
709 * Automated processing scales linearly
710
711 === 9.3 NFR3 — Transparency ===
712
713 **Fulfills**: UN-7 (Evidence transparency), UN-9 (Methodology transparency), UN-13 (Citable verdicts), UN-15 (Evolution visibility)
714
715 * All algorithms open source
716 * All data exportable
717 * All decisions documented
718 * Quality metrics public
719
720 === 9.4 NFR4 — Security & Privacy ===
721
722 * Follow [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
723 * Secure authentication
724 * Data encryption
725 * Regular security audits
726
727 === 9.5 NFR5 — Maintainability ===
728
729 * Modular architecture
730 * Automated testing
731 * Continuous integration
732 * Comprehensive documentation
733
734 === NFR11: AKEL Quality Assurance Framework ===
735
736 **Fulfills:** AI safety, IFCN methodology transparency
737
738 **Specification:**
739
740 Multi-layer AI quality gates to detect hallucinations, low-confidence results, and logical inconsistencies.
741
742 ==== Quality Gate 1: Claim Extraction Validation ====
743
744 **Purpose:** Ensure extracted claims are factual assertions (not opinions/predictions)
745
746 **Checks:**
747 1. **Factual Statement Test:** Is this verifiable? (Yes/No)
748 2. **Opinion Detection:** Contains hedging language? ("I think", "probably", "best")
749 3. **Future Prediction Test:** Makes claims about future events?
750 4. **Specificity Score:** Contains specific entities, numbers, dates?
751
752 **Thresholds:**
753 * Factual: Must be "Yes"
754 * Opinion markers: <2 hedging phrases
755 * Specificity: ≥3 specific elements
756
757 **Action if Failed:** Flag as "Non-verifiable", do NOT generate verdict
758
759 ==== Quality Gate 2: Evidence Relevance Validation ====
760
761 **Purpose:** Ensure AI-linked evidence actually relates to claim
762
763 **Checks:**
764 1. **Semantic Similarity Score:** Evidence vs. claim (embeddings)
765 2. **Entity Overlap:** Shared people/places/things?
766 3. **Topic Relevance:** Discusses claim subject?
767
768 **Thresholds:**
769 * Similarity: ≥0.6 (cosine similarity)
770 * Entity overlap: ≥1 shared entity
771 * Topic relevance: ≥0.5
772
773 **Action if Failed:** Discard irrelevant evidence
774
775 ==== Quality Gate 3: Scenario Coherence Check ====
776
777 **Purpose:** Validate scenario assumptions are logical and complete
778
779 **Checks:**
780 1. **Completeness:** All required fields populated
781 2. **Internal Consistency:** Assumptions don't contradict
782 3. **Distinguishability:** Scenarios meaningfully different
783
784 **Thresholds:**
785 * Required fields: 100%
786 * Contradiction score: <0.3
787 * Scenario similarity: <0.8
788
789 **Action if Failed:** Merge duplicates, reduce confidence -20%
790
791 ==== Quality Gate 4: Verdict Confidence Assessment ====
792
793 **Purpose:** Only publish high-confidence verdicts
794
795 **Checks:**
796 1. **Evidence Count:** Minimum 2 sources
797 2. **Source Quality:** Average reliability ≥0.6
798 3. **Evidence Agreement:** Supporting vs. contradicting ≥0.6
799 4. **Uncertainty Factors:** Hedging in reasoning
800
801 **Confidence Tiers:**
802 * **HIGH (80-100%):** ≥3 sources, ≥0.7 quality, ≥80% agreement
803 * **MEDIUM (50-79%):** ≥2 sources, ≥0.6 quality, ≥60% agreement
804 * **LOW (0-49%):** <2 sources OR low quality/agreement
805 * **INSUFFICIENT:** <2 sources → DO NOT PUBLISH
806
807 **Implementation Phases:**
808 * **POC1:** Gates 1 & 4 only (basic validation)
809 * **POC2:** All 4 gates (complete framework)
810 * **V1.0:** Hardened with <5% hallucination rate
811
812 **Acceptance Criteria:**
813 * ✅ All gates operational
814 * ✅ Hallucination rate <5%
815 * ✅ Quality metrics public
816
817 === NFR12: Security Controls ===
818
819 **Fulfills:** Data protection, system integrity, user privacy, production readiness
820
821
822 **Purpose:** Protect FactHarbor systems, user data, and operations from security threats, ensuring production-grade security posture.
823
824 **Specification:**
825
826 ==== API Security ====
827
828 **Rate Limiting:**
829 * **Analysis endpoints:** 100 requests/hour per IP
830 * **Read endpoints:** 1,000 requests/hour per IP
831 * **Search:** 500 requests/hour per IP
832 * **Authenticated users:** 5x higher limits
833 * **Burst protection:** Max 10 requests/second
834
835 **Authentication & Authorization:**
836 * **API Keys:** Required for programmatic access
837 * **JWT tokens:** For user sessions (1-hour expiry)
838 * **OAuth2:** For third-party integrations
839 * **Role-Based Access Control (RBAC):**
840 * Public: Read-only access to published claims
841 * Contributor: Submit claims, provide evidence
842 * Moderator: Review contributions, manage quality
843 * Admin: System configuration, user management
844
845 **CORS Policies:**
846 * Whitelist approved domains only
847 * No wildcard origins in production
848 * Credentials required for sensitive endpoints
849
850 **Input Sanitization:**
851 * Validate all user input against schemas
852 * Sanitize HTML/JavaScript in text submissions
853 * Prevent SQL injection (use parameterized queries)
854 * Prevent command injection (no shell execution of user input)
855 * Max request size: 10MB
856 * File upload restrictions: Whitelist file types, scan for malware
857
858 ---
859
860 ==== Data Security ====
861
862 **Encryption at Rest:**
863 * Database encryption using AES-256
864 * Encrypted backups
865 * Key management via cloud provider KMS (AWS KMS, Google Cloud KMS)
866 * Regular key rotation (90-day cycle)
867
868 **Encryption in Transit:**
869 * HTTPS/TLS 1.3 only (no TLS 1.0/1.1)
870 * Strong cipher suites only
871 * HSTS (HTTP Strict Transport Security) enabled
872 * Certificate pinning for mobile apps
873
874 **Secure Credential Storage:**
875 * Passwords hashed with bcrypt (cost factor 12+)
876 * API keys encrypted in database
877 * Secrets stored in environment variables (never in code)
878 * Use secrets manager (AWS Secrets Manager, HashiCorp Vault)
879
880 **Data Privacy:**
881 * Minimal data collection (privacy by design)
882 * User data deletion on request (GDPR compliance)
883 * PII encryption in database
884 * Anonymize logs (no PII in log files)
885
886 ---
887
888 ==== Application Security ====
889
890 **OWASP Top 10 Compliance:**
891
892 1. **Broken Access Control:** RBAC implementation, path traversal prevention
893 2. **Cryptographic Failures:** Strong encryption, secure key management
894 3. **Injection:** Parameterized queries, input validation
895 4. **Insecure Design:** Security review of all features
896 5. **Security Misconfiguration:** Hardened defaults, security headers
897 6. **Vulnerable Components:** Dependency scanning (see below)
898 7. **Authentication Failures:** Strong password policy, MFA support
899 8. **Data Integrity Failures:** Signature verification, checksums
900 9. **Security Logging Failures:** Comprehensive audit logs
901 10. **Server-Side Request Forgery:** URL validation, whitelist domains
902
903 **Security Headers:**
904 * `Content-Security-Policy`: Strict CSP to prevent XSS
905 * `X-Frame-Options`: DENY (prevent clickjacking)
906 * `X-Content-Type-Options`: nosniff
907 * `Referrer-Policy`: strict-origin-when-cross-origin
908 * `Permissions-Policy`: Restrict browser features
909
910 **Dependency Vulnerability Scanning:**
911 * **Tools:** Snyk, Dependabot, npm audit, pip-audit
912 * **Frequency:** Daily automated scans
913 * **Action:** Patch critical vulnerabilities within 24 hours
914 * **Policy:** No known high/critical CVEs in production
915
916 **Security Audits:**
917 * **Internal:** Quarterly security reviews
918 * **External:** Annual penetration testing by certified firm
919 * **Bug Bounty:** Public bug bounty program (V1.1+)
920 * **Compliance:** SOC 2 Type II certification target (V1.5)
921
922 ---
923
924 ==== Operational Security ====
925
926 **DDoS Protection:**
927 * CloudFlare or AWS Shield
928 * Rate limiting at CDN layer
929 * Automatic IP blocking for abuse patterns
930
931 **Monitoring & Alerting:**
932 * Real-time security event monitoring
933 * Alerts for:
934 * Failed login attempts (>5 in 10 minutes)
935 * API abuse patterns
936 * Unusual data access patterns
937 * Security scan detections
938 * Integration with SIEM (Security Information and Event Management)
939
940 **Incident Response:**
941 * Documented incident response plan
942 * Security incident classification (P1-P4)
943 * On-call rotation for security issues
944 * Post-mortem for all security incidents
945 * Public disclosure policy (coordinated disclosure)
946
947 **Backup & Recovery:**
948 * Daily encrypted backups
949 * 30-day retention period
950 * Tested recovery procedures (quarterly)
951 * Disaster recovery plan (RTO: 4 hours, RPO: 1 hour)
952
953 ---
954
955 ==== Compliance & Standards ====
956
957 **GDPR Compliance:**
958 * User consent management
959 * Right to access data
960 * Right to deletion
961 * Data portability
962 * Privacy policy published
963
964 **Accessibility:**
965 * WCAG 2.1 AA compliance
966 * Screen reader compatibility
967 * Keyboard navigation
968 * Alt text for images
969
970 **Browser Support:**
971 * Modern browsers only (Chrome/Edge/Firefox/Safari latest 2 versions)
972 * No IE11 support
973
974 **Acceptance Criteria:**
975
976 * ✅ Passes OWASP ZAP security scan (no high/critical findings)
977 * ✅ All dependencies with known vulnerabilities patched
978 * ✅ Penetration test completed with no critical findings
979 * ✅ Rate limiting blocks abuse attempts
980 * ✅ Encryption at rest and in transit verified
981 * ✅ Security headers scored A+ on securityheaders.com
982 * ✅ Incident response plan documented and tested
983 * ✅ 95% uptime over 30-day period
984
985
986 === NFR13: Quality Metrics Transparency ===
987
988 **Fulfills:** User trust, transparency, continuous improvement, IFCN methodology transparency
989
990
991 **Purpose:** Provide transparent, measurable quality metrics that demonstrate AKEL's performance and build user trust in automated fact-checking.
992
993 **Specification:**
994
995 ==== Component: Public Quality Dashboard ====
996
997 **Core Metrics to Display:**
998
999 **1. Verdict Quality Metrics**
1000
1001 **TIGERScore (Fact-Checking Quality):**
1002 * **Definition:** Measures how well generated verdicts match expert fact-checker judgments
1003 * **Scale:** 0-100 (higher is better)
1004 * **Calculation:** Using TIGERScore framework (Truth-conditional accuracy, Informativeness, Generality, Evaluativeness, Relevance)
1005 * **Target:** Average ≥80 for production release
1006 * **Display:**
1007 {{code}}
1008 Verdict Quality (TIGERScore):
1009 Overall: 84.2 ▲ (+2.1 from last month)
1010
1011 Distribution:
1012 Excellent (>80): 67%
1013 Good (60-80): 28%
1014 Needs Improvement (<60): 5%
1015
1016 Trend: [Graph showing improvement over time]
1017 {{/code}}
1018
1019 **2. Hallucination & Faithfulness Metrics**
1020
1021 **AlignScore (Faithfulness to Evidence):**
1022 * **Definition:** Measures how well verdicts align with actual evidence content
1023 * **Scale:** 0-1 (higher is better)
1024 * **Purpose:** Detect AI hallucinations (making claims not supported by evidence)
1025 * **Target:** Average ≥0.85, hallucination rate <5%
1026 * **Display:**
1027 {{code}}
1028 Evidence Faithfulness (AlignScore):
1029 Average: 0.87 ▼ (-0.02 from last month)
1030
1031 Hallucination Rate: 4.2%
1032 - Claims without evidence support: 3.1%
1033 - Misrepresented evidence: 1.1%
1034
1035 Action: Prompt engineering review scheduled
1036 {{/code}}
1037
1038 **3. Evidence Quality Metrics**
1039
1040 **Source Reliability:**
1041 * Average source quality score (0-1 scale)
1042 * Distribution of high/medium/low quality sources
1043 * Publisher track record trends
1044
1045 **Evidence Coverage:**
1046 * Average number of sources per claim
1047 * Percentage of claims with ≥2 sources (EFCSN minimum)
1048 * Geographic diversity of sources
1049
1050 **Display:**
1051 {{code}}
1052 Evidence Quality:
1053
1054 Average Sources per Claim: 4.2
1055 Claims with ≥2 sources: 94% (EFCSN compliant)
1056
1057 Source Quality Distribution:
1058 High quality (>0.8): 48%
1059 Medium quality (0.5-0.8): 43%
1060 Low quality (<0.5): 9%
1061
1062 Geographic Diversity: 23 countries represented
1063 {{/code}}
1064
1065 **4. Contributor Consensus Metrics** (when human reviewers involved)
1066
1067 **Inter-Rater Reliability (IRR):**
1068 * **Calculation:** Cohen's Kappa or Fleiss' Kappa for multiple raters
1069 * **Scale:** 0-1 (higher is better)
1070 * **Interpretation:**
1071 * >0.8: Almost perfect agreement
1072 * 0.6-0.8: Substantial agreement
1073 * 0.4-0.6: Moderate agreement
1074 * <0.4: Poor agreement
1075 * **Target:** Maintain ≥0.7 (substantial agreement)
1076
1077 **Display:**
1078 {{code}}
1079 Contributor Consensus:
1080
1081 Inter-Rater Reliability (IRR): 0.73 (Substantial agreement)
1082 - Verdict agreement: 78%
1083 - Evidence quality agreement: 71%
1084 - Scenario structure agreement: 69%
1085
1086 Cases requiring moderator review: 12
1087 Moderator override rate: 8%
1088 {{/code}}
1089
1090 ---
1091
1092 ==== Quality Dashboard Implementation ====
1093
1094 **Dashboard Location:** `/quality-metrics`
1095
1096 **Update Frequency:**
1097 * **POC2:** Weekly manual updates
1098 * **Beta 0:** Daily automated updates
1099 * **V1.0:** Real-time metrics (updated hourly)
1100
1101 **Dashboard Sections:**
1102
1103 1. **Overview:** Key metrics at a glance
1104 2. **Verdict Quality:** TIGERScore trends and distributions
1105 3. **Evidence Analysis:** Source quality and coverage
1106 4. **AI Performance:** Hallucination rates, AlignScore
1107 5. **Human Oversight:** Contributor consensus, review rates
1108 6. **System Health:** Processing times, error rates, uptime
1109
1110 **Example Dashboard Layout:**
1111
1112 {{code}}
1113 ┌─────────────────────────────────────────────────────────────┐
1114 │ FactHarbor Quality Metrics Last updated: │
1115 │ Public Dashboard 2 hours ago │
1116 └─────────────────────────────────────────────────────────────┘
1117
1118 📊 KEY METRICS
1119 ─────────────────────────────────────────────────────────────
1120 TIGERScore (Verdict Quality): 84.2 ▲ (+2.1)
1121 AlignScore (Faithfulness): 0.87 ▼ (-0.02)
1122 Hallucination Rate: 4.2% ✓ (Target: <5%)
1123 Average Sources per Claim: 4.2 ▲ (+0.3)
1124
1125 📈 TRENDS (30 days)
1126 ─────────────────────────────────────────────────────────────
1127 [Graph: TIGERScore trending upward]
1128 [Graph: Hallucination rate declining]
1129 [Graph: Evidence quality stable]
1130
1131 ⚠️ IMPROVEMENT TARGETS
1132 ─────────────────────────────────────────────────────────────
1133 1. Reduce hallucination rate to <3% (Current: 4.2%)
1134 2. Increase TIGERScore average to >85 (Current: 84.2)
1135 3. Maintain IRR >0.75 (Current: 0.73)
1136
1137 📄 DETAILED REPORTS
1138 ─────────────────────────────────────────────────────────────
1139 • Monthly Quality Report (PDF)
1140 • Methodology Documentation
1141 • AKEL Performance Analysis
1142 • Contributor Agreement Analysis
1143
1144 {{/code}}
1145
1146 ---
1147
1148 ==== Continuous Improvement Feedback Loop ====
1149
1150 **How Metrics Inform AKEL Improvements:**
1151
1152 1. **Identify Weak Areas:**
1153 * Low TIGERScore → Review prompt engineering
1154 * High hallucination → Strengthen evidence grounding
1155 * Low IRR → Clarify evaluation criteria
1156
1157 2. **A/B Testing Integration:**
1158 * Test prompt variations
1159 * Measure impact on quality metrics
1160 * Deploy winners automatically
1161
1162 3. **Alert Thresholds:**
1163 * TIGERScore drops below 75 → Alert team
1164 * Hallucination rate exceeds 7% → Pause auto-publishing
1165 * IRR below 0.6 → Moderator training needed
1166
1167 4. **Monthly Quality Reviews:**
1168 * Analyze trends
1169 * Identify systematic issues
1170 * Plan prompt improvements
1171 * Update AKEL models
1172
1173 ---
1174
1175 ==== Metric Calculation Details ====
1176
1177 **TIGERScore Implementation:**
1178 * Reference: https://github.com/TIGER-AI-Lab/TIGERScore
1179 * Input: Generated verdict + reference verdict (from expert)
1180 * Output: 0-100 score across 5 dimensions
1181 * Requires: Test set of expert-reviewed claims (minimum 100)
1182
1183 **AlignScore Implementation:**
1184 * Reference: https://github.com/yuh-zha/AlignScore
1185 * Input: Generated verdict + source evidence text
1186 * Output: 0-1 faithfulness score
1187 * Calculation: Semantic alignment between claim and evidence
1188
1189 **Source Quality Scoring:**
1190 * Use existing source reliability database (e.g., NewsGuard, MBFC)
1191 * Factor in: Publication history, corrections record, transparency
1192 * Scale: 0-1 (weighted average across sources)
1193
1194 ---
1195
1196 ==== Integration Points ====
1197
1198 * **NFR11: AKEL Quality Assurance** - Metrics validate quality gate effectiveness
1199 * **FR49: A/B Testing** - Metrics measure test success
1200 * **FR11: Audit Trail** - Source of quality data
1201 * **NFR3: Transparency** - Public metrics build trust
1202
1203 **Acceptance Criteria:**
1204
1205 * ✅ All core metrics implemented and calculating correctly
1206 * ✅ Dashboard updates daily (Beta 0) or hourly (V1.0)
1207 * ✅ Alerts trigger when metrics degrade beyond thresholds
1208 * ✅ Monthly quality report auto-generates
1209 * ✅ Dashboard is publicly accessible (no login required)
1210 * ✅ Mobile-responsive dashboard design
1211 * ✅ Metrics inform quarterly AKEL improvement planning
1212
1213
1214
1215
1216 == 13. Requirements Traceability ==
1217
1218 For full traceability matrix showing which requirements fulfill which user needs, see:
1219
1220 * [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] - Section 8 includes comprehensive mapping tables
1221
1222 == 14. Related Pages ==
1223
1224 **Non-Functional Requirements (see Section 9):**
1225 * [[NFR11 — AKEL Quality Assurance Framework>>#NFR11]]
1226 * [[NFR12 — Security Controls>>#NFR12]]
1227 * [[NFR13 — Quality Metrics Transparency>>#NFR13]]
1228
1229 **Other Requirements:**
1230 * [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]
1231 * [[V1.0 Requirements>>FactHarbor.Specification.Requirements.V10.]]
1232 * [[Gap Analysis>>FactHarbor.Specification.Requirements.GapAnalysis]]
1233
1234 * **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need (drives these requirements)
1235 * [[Architecture>>FactHarbor.Specification.Architecture.WebHome]] - How requirements are implemented
1236 * [[Data Model>>FactHarbor.Specification.Data Model.WebHome]] - Data structures supporting requirements
1237 * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] - User interaction workflows
1238 * [[AKEL>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements
1239 * [[Global Rules>>FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]]
1240 * [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
1241
1242 = V0.9.70 Additional Requirements =
1243
1244 == Functional Requirements (Additional) ==
1245
1246 === FR44: ClaimReview Schema Implementation ===
1247
1248 **Fulfills:** UN-13 (Cite FactHarbor Verdicts), UN-14 (API Access for Integration), UN-26 (Search Engine Visibility)
1249
1250
1251 **Purpose:** Generate valid ClaimReview structured data for every published analysis to enable Google/Bing search visibility and fact-check discovery.
1252
1253 **Specification:**
1254
1255 ==== Component: Schema.org Markup Generator ====
1256
1257 FactHarbor must generate valid ClaimReview structured data following Schema.org specifications for every published claim analysis.
1258
1259 **Required JSON-LD Schema:**
1260
1261 {{code language="json"}}
1262 {
1263 "@context": "https://schema.org",
1264 "@type": "ClaimReview",
1265 "datePublished": "YYYY-MM-DD",
1266 "url": "https://factharbor.org/claims/{claim_id}",
1267 "claimReviewed": "The exact claim text",
1268 "author": {
1269 "@type": "Organization",
1270 "name": "FactHarbor",
1271 "url": "https://factharbor.org"
1272 },
1273 "reviewRating": {
1274 "@type": "Rating",
1275 "ratingValue": "1-5",
1276 "bestRating": "5",
1277 "worstRating": "1",
1278 "alternateName": "FactHarbor likelihood score"
1279 },
1280 "itemReviewed": {
1281 "@type": "Claim",
1282 "author": {
1283 "@type": "Person",
1284 "name": "Claim author if known"
1285 },
1286 "datePublished": "YYYY-MM-DD if known",
1287 "appearance": {
1288 "@type": "CreativeWork",
1289 "url": "Original claim URL if from article"
1290 }
1291 }
1292 }
1293 {{/code}}
1294
1295 **FactHarbor-Specific Mapping:**
1296
1297 **Likelihood Score to Rating Scale:**
1298 * 80-100% likelihood → 5 (Highly Supported)
1299 * 60-79% likelihood → 4 (Supported)
1300 * 40-59% likelihood → 3 (Mixed/Uncertain)
1301 * 20-39% likelihood → 2 (Questionable)
1302 * 0-19% likelihood → 1 (Refuted)
1303
1304 **Multiple Scenarios Handling:**
1305 * If claim has multiple scenarios with different verdicts, generate **separate ClaimReview** for each scenario
1306 * Add `disambiguatingDescription` field explaining scenario context
1307 * Example: "Scenario: If interpreted as referring to 2023 data..."
1308
1309 ==== Implementation Requirements ====
1310
1311 1. **Auto-generate** on claim publication
1312 2. **Embed** in HTML `<head>` section as JSON-LD script
1313 3. **Validate** against Schema.org validator before publishing
1314 4. **Submit** to Google Search Console for indexing
1315 5. **Update** automatically when verdict changes (integrate with FR8: Time Evolution)
1316
1317 ==== Integration Points ====
1318
1319 * **FR7: Automated Verdicts** - Source of rating data and claim text
1320 * **FR8: Time Evolution** - Triggers schema updates when verdicts change
1321 * **FR11: Audit Trail** - Logs all schema generation and update events
1322
1323 ==== Resources ====
1324
1325 * ClaimReview Project: https://www.claimreviewproject.com
1326 * Schema.org ClaimReview: https://schema.org/ClaimReview
1327 * Google Fact Check Guidelines: https://developers.google.com/search/docs/appearance/fact-check
1328
1329 **Acceptance Criteria:**
1330
1331 * ✅ Passes Google Structured Data Testing Tool
1332 * ✅ Appears in Google Fact Check Explorer within 48 hours of publication
1333 * ✅ Valid JSON-LD syntax (no errors)
1334 * ✅ All required fields populated with correct data types
1335 * ✅ Handles multi-scenario claims correctly (separate ClaimReview per scenario)
1336
1337
1338 === FR45: User Corrections Notification System ===
1339
1340 **Fulfills:** IFCN Principle 5 (Open & Honest Corrections), EFCSN compliance
1341
1342
1343 **Purpose:** When any claim analysis is corrected, notify users who previously viewed the claim to maintain transparency and build trust.
1344
1345 **Specification:**
1346
1347 ==== Component: Corrections Visibility Framework ====
1348
1349 **Correction Types:**
1350
1351 1. **Major Correction:** Verdict changes category (e.g., "Supported" → "Refuted")
1352 2. **Significant Correction:** Likelihood score changes >20%
1353 3. **Minor Correction:** Evidence additions, source quality updates
1354 4. **Scenario Addition:** New scenario added to existing claim
1355
1356 ==== Notification Mechanisms ====
1357
1358 **1. In-Page Banner:**
1359
1360 Display prominent banner on claim page:
1361
1362 {{code}}
1363 [!] CORRECTION NOTICE
1364 This analysis was updated on [DATE]. [View what changed] [Dismiss]
1365
1366 Major changes:
1367 • Verdict changed from "Likely True (75%)" to "Uncertain (45%)"
1368 • New contradicting evidence added from [Source]
1369 • Scenario 2 updated with additional context
1370
1371 [See full correction log]
1372 {{/code}}
1373
1374 **2. Correction Log Page:**
1375
1376 * Public changelog at `/claims/{id}/corrections`
1377 * Displays for each correction:
1378 * Date/time of correction
1379 * What changed (before/after comparison)
1380 * Why changed (reason if provided)
1381 * Who made change (AKEL auto-update vs. contributor override)
1382
1383 **3. Email Notifications (opt-in):**
1384
1385 * Send to users who bookmarked or shared the claim
1386 * Subject: "FactHarbor Correction: [Claim title]"
1387 * Include summary of changes
1388 * Link to updated analysis
1389
1390 **4. RSS/API Feed:**
1391
1392 * Corrections feed at `/corrections.rss`
1393 * API endpoint: `GET /api/corrections?since={timestamp}`
1394 * Enables external monitoring by journalists and researchers
1395
1396 ==== Display Rules ====
1397
1398 * Show banner on **ALL pages** displaying the claim (search results, related claims, embeddings)
1399 * Banner persists for **30 days** after correction
1400 * **"Corrections" count badge** on claim card
1401 * **Timestamp** on every verdict: "Last updated: [datetime]"
1402
1403 ==== IFCN Compliance Requirements ====
1404
1405 * Corrections policy published at `/corrections-policy`
1406 * User can report suspected errors via `/report-error/{claim_id}`
1407 * Link to IFCN complaint process (if FactHarbor becomes signatory)
1408 * **Scrupulous transparency:** Never silently edit analyses
1409
1410 ==== Integration Points ====
1411
1412 * **FR8: Time Evolution** - Triggers corrections when verdicts change
1413 * **FR11: Audit Trail** - Source of correction data and change history
1414 * **NFR3: Transparency** - Public correction log demonstrates commitment
1415
1416 **Acceptance Criteria:**
1417
1418 * ✅ Banner appears within 60 seconds of correction
1419 * ✅ Correction log is permanent and publicly accessible
1420 * ✅ Email notifications deliver within 5 minutes
1421 * ✅ RSS feed updates in real-time
1422 * ✅ Mobile-responsive banner design
1423 * ✅ Accessible (screen reader compatible)
1424
1425
1426 === FR46: Image Verification System ===
1427
1428 **Fulfills:** UN-27 (Visual Claim Verification)
1429
1430
1431 **Purpose:** Verify authenticity and context of images shared with claims to detect manipulation, misattribution, and out-of-context usage.
1432
1433 **Specification:**
1434
1435 ==== Component: Multi-Method Image Verification ====
1436
1437 **Method 1: Reverse Image Search**
1438
1439 **Purpose:** Find earlier uses of the image to verify context
1440
1441 **Implementation:**
1442 * Integrate APIs:
1443 * **Google Vision AI** (reverse search)
1444 * **TinEye** (oldest known uses)
1445 * **Bing Visual Search** (broad coverage)
1446
1447 **Process:**
1448 1. Extract image from claim or user upload
1449 2. Query multiple reverse search services
1450 3. Analyze results for:
1451 * Earliest known publication
1452 * Original context (what was it really showing?)
1453 * Publication timeline
1454 * Geographic spread
1455
1456 **Output:**
1457 {{code}}
1458 Reverse Image Search Results:
1459
1460 Earliest known use: 2019-03-15 (5 years before claim)
1461 Original context: "Photo from 2019 flooding in Mumbai"
1462 This claim uses it for: "2024 hurricane damage in Florida"
1463
1464 ⚠️ Image is OUT OF CONTEXT
1465
1466 Found in 47 other articles:
1467 • 2019-03-15: Mumbai floods (original)
1468 • 2020-07-22: Bangladesh monsoon
1469 • 2024-10-15: Current claim (misattributed)
1470
1471 [View full timeline]
1472 {{/code}}
1473
1474 ---
1475
1476 **Method 2: AI Manipulation Detection**
1477
1478 **Purpose:** Detect deepfakes, face swaps, and digital alterations
1479
1480 **Implementation:**
1481 * Integrate detection services:
1482 * **Sensity AI** (deepfake detection)
1483 * **Reality Defender** (multimodal analysis)
1484 * **AWS Rekognition** (face detection inconsistencies)
1485
1486 **Detection Categories:**
1487 1. **Face Manipulation:**
1488 * Deepfake face swaps
1489 * Expression manipulation
1490 * Identity replacement
1491
1492 2. **Image Manipulation:**
1493 * Copy-paste artifacts
1494 * Clone stamp detection
1495 * Content-aware fill detection
1496 * JPEG compression inconsistencies
1497
1498 3. **AI Generation:**
1499 * Detect fully AI-generated images
1500 * Identify generation artifacts
1501 * Check for model signatures
1502
1503 **Confidence Scoring:**
1504 * **HIGH (80-100%):** Strong evidence of manipulation
1505 * **MEDIUM (50-79%):** Suspicious artifacts detected
1506 * **LOW (0-49%):** Minor inconsistencies or inconclusive
1507
1508 **Output:**
1509 {{code}}
1510 Manipulation Analysis:
1511
1512 Face Manipulation: LOW RISK (12%)
1513 Image Editing: MEDIUM RISK (64%)
1514 • Clone stamp artifacts detected in sky region
1515 • JPEG compression inconsistent between objects
1516
1517 AI Generation: LOW RISK (8%)
1518
1519 ⚠️ Possible manipulation detected. Manual review recommended.
1520 {{/code}}
1521
1522 ---
1523
1524 **Method 3: Metadata Analysis (EXIF)**
1525
1526 **Purpose:** Extract technical details that may reveal manipulation or misattribution
1527
1528 **Extracted Data:**
1529 * **Camera/Device:** Make, model, software
1530 * **Timestamps:** Original date, modification dates
1531 * **Location:** GPS coordinates (if present)
1532 * **Editing History:** Software used, edit count
1533 * **File Properties:** Resolution, compression, format conversions
1534
1535 **Red Flags:**
1536 * Metadata completely stripped (suspicious)
1537 * Timestamp conflicts with claimed date
1538 * GPS location conflicts with claimed location
1539 * Multiple edit rounds (hiding something?)
1540 * Creation date after modification date (impossible)
1541
1542 **Output:**
1543 {{code}}
1544 Image Metadata:
1545
1546 Camera: iPhone 14 Pro
1547 Original date: 2023-08-12 14:32:15
1548 Location: 40.7128°N, 74.0060°W (New York City)
1549 Modified: 2024-10-15 08:45:22
1550 Software: Adobe Photoshop 2024
1551
1552 ⚠️ Location conflicts with claim
1553 Claim says: "Taken in Los Angeles"
1554 EXIF says: New York City
1555
1556 ⚠️ Edited 14 months after capture
1557 {{/code}}
1558
1559 ---
1560
1561 ==== Verification Workflow ====
1562
1563 **Automatic Triggers:**
1564 1. User submits claim with image
1565 2. Article being analyzed contains images
1566 3. Social media post includes photos
1567
1568 **Process:**
1569 1. Extract images from content
1570 2. Run all 3 verification methods in parallel
1571 3. Aggregate results into confidence score
1572 4. Generate human-readable summary
1573 5. Display prominently in analysis
1574
1575 **Display Integration:**
1576
1577 Show image verification panel in claim analysis:
1578
1579 {{code}}
1580 📷 IMAGE VERIFICATION
1581
1582 [Image thumbnail]
1583
1584 ✅ Reverse Search: Original context verified
1585 ⚠️ Manipulation: Possible editing detected (64% confidence)
1586 ✅ Metadata: Consistent with claim details
1587
1588 Overall Assessment: CAUTION ADVISED
1589 This image may have been edited. Original context appears accurate.
1590
1591 [View detailed analysis]
1592 {{/code}}
1593
1594 ==== Integration Points ====
1595
1596 * **FR7: Automated Verdicts** - Image verification affects claim credibility
1597 * **FR4: Analysis Summary** - Image findings included in summary
1598 * **UN-27: Visual Claim Verification** - Direct fulfillment
1599
1600 ==== Cost Considerations ====
1601
1602 **API Costs (estimated per image):**
1603 * Google Vision AI: $0.001-0.003
1604 * TinEye: $0.02 (commercial API)
1605 * Sensity AI: $0.05-0.10
1606 * AWS Rekognition: $0.001-0.002
1607
1608 **Total per image:** ~$0.07-0.15
1609
1610 **Mitigation Strategies:**
1611 * Cache results for duplicate images
1612 * Use free tier quotas where available
1613 * Prioritize higher-value claims for deep analysis
1614 * Offer premium verification as paid tier
1615
1616 **Acceptance Criteria:**
1617
1618 * ✅ Reverse image search finds original sources
1619 * ✅ Manipulation detection accuracy >80% on test dataset
1620 * ✅ EXIF extraction works for major image formats (JPEG, PNG, HEIC)
1621 * ✅ Results display within 10 seconds
1622 * ✅ Mobile-friendly image comparison interface
1623 * ✅ False positive rate <15%
1624
1625
1626 === FR47: Archive.org Integration ===
1627
1628 **Importance:** CRITICAL
1629 **Fulfills:** Evidence persistence, FR5 (Evidence linking)
1630
1631 **Purpose:** Ensure evidence remains accessible even if original sources are deleted.
1632
1633 **Specification:**
1634
1635 **Automatic Archiving:**
1636
1637 When AKEL links evidence:
1638 1. Check if URL already archived (Wayback Machine API)
1639 2. If not, submit for archiving (Save Page Now API)
1640 3. Store both original URL and archive URL
1641 4. Display both to users
1642
1643 **Archive Display:**
1644
1645 {{code}}
1646 Evidence Source: [Original URL]
1647 Archived: [Archive.org URL] (Captured: [date])
1648
1649 [View Original] [View Archive]
1650 {{/code}}
1651
1652 **Fallback Logic:**
1653
1654 * If original URL unavailable → Auto-redirect to archive
1655 * If archive unavailable → Display warning
1656 * If both unavailable → Flag for manual review
1657
1658 **API Integration:**
1659
1660 * Use Wayback Machine Availability API
1661 * Use Save Page Now API (SPNv2)
1662 * Rate limiting: 15 requests/minute (Wayback limit)
1663
1664 **Acceptance Criteria:**
1665
1666 * ✅ All evidence URLs auto-archived
1667 * ✅ Archive links displayed to users
1668 * ✅ Fallback to archive if original unavailable
1669 * ✅ API rate limits respected
1670 * ✅ Archive status visible in evidence display
1671
1672
1673 == Category 4: Community Safety ===== FR48: Contributor Safety Framework ===
1674
1675 **Importance:** CRITICAL
1676 **Fulfills:** UN-28 (Safe contribution environment)
1677
1678 **Purpose:** Protect contributors from harassment, doxxing, and coordinated attacks.
1679
1680 **Specification:**
1681
1682 **1. Privacy Protection:**
1683
1684 * **Optional Pseudonymity:** Contributors can use pseudonyms
1685 * **Email Privacy:** Emails never displayed publicly
1686 * **Profile Privacy:** Contributors control what's public
1687 * **IP Logging:** Only for abuse prevention, not public
1688
1689 **2. Harassment Prevention:**
1690
1691 * **Automated Toxicity Detection:** Flag abusive comments
1692 * **Personal Information Detection:** Auto-block doxxing attempts
1693 * **Coordinated Attack Detection:** Identify brigading patterns
1694 * **Rapid Response:** Moderator alerts for harassment
1695
1696 **3. Safety Features:**
1697
1698 * **Block Users:** Contributors can block harassers
1699 * **Private Contributions:** Option to contribute anonymously
1700 * **Report Harassment:** One-click harassment reporting
1701 * **Safety Resources:** Links to support resources
1702
1703 **4. Moderator Tools:**
1704
1705 * **Quick Ban:** Immediately block abusers
1706 * **Pattern Detection:** Identify coordinated attacks
1707 * **Appeal Process:** Fair review of moderation actions
1708 * **Escalation:** Serious threats escalated to authorities
1709
1710 **5. Trusted Contributor Protection:**
1711
1712 * **Enhanced Privacy:** Additional protection for high-profile contributors
1713 * **Verification:** Optional identity verification (not public)
1714 * **Legal Support:** Resources for contributors facing legal threats
1715
1716 **Acceptance Criteria:**
1717
1718 * ✅ Pseudonyms supported
1719 * ✅ Toxicity detection active
1720 * ✅ Doxxing auto-blocked
1721 * ✅ Harassment reporting functional
1722 * ✅ Moderator tools implemented
1723 * ✅ Safety policy published
1724
1725
1726 == Category 5: Continuous Improvement ===== FR49: A/B Testing Framework ===
1727
1728 **Importance:** CRITICAL
1729 **Fulfills:** Continuous system improvement
1730
1731 **Purpose:** Test and measure improvements to AKEL prompts, algorithms, and workflows.
1732
1733 **Specification:**
1734
1735 **Test Capabilities:**
1736
1737 1. **Prompt Variations:**
1738 * Test different claim extraction prompts
1739 * Test different verdict generation prompts
1740 * Measure: Accuracy, clarity, completeness
1741
1742 2. **Algorithm Variations:**
1743 * Test different source scoring algorithms
1744 * Test different confidence calculations
1745 * Measure: Audit accuracy, user satisfaction
1746
1747 3. **Workflow Variations:**
1748 * Test different quality gate thresholds
1749 * Test different risk tier assignments
1750 * Measure: Publication rate, quality scores
1751
1752 **Implementation:**
1753
1754 * **Traffic Split:** 50/50 or 90/10 splits
1755 * **Randomization:** Consistent per claim (not per user)
1756 * **Metrics Collection:** Automatic for all variants
1757 * **Statistical Significance:** Minimum sample size calculation
1758 * **Rollout:** Winner promoted to 100% traffic
1759
1760 **A/B Test Workflow:**
1761
1762 {{code}}
1763 1. Hypothesis: "New prompt improves claim extraction"
1764 2. Design test: Control vs. Variant
1765 3. Define metrics: Extraction accuracy, completeness
1766 4. Run test: 7-14 days, minimum 100 claims each
1767 5. Analyze results: Statistical significance?
1768 6. Decision: Deploy winner or iterate
1769 {{/code}}
1770
1771 **Acceptance Criteria:**
1772
1773 * ✅ A/B testing framework implemented
1774 * ✅ Can test prompt variations
1775 * ✅ Can test algorithm variations
1776 * ✅ Metrics automatically collected
1777 * ✅ Statistical significance calculated
1778 * ✅ Results inform system improvements
1779
1780
1781 === FR54: Evidence Deduplication ===
1782
1783 **Importance:** CRITICAL (POC2/Beta)
1784 **Fulfills:** Accurate evidence counting, quality metrics
1785
1786 **Purpose:** Avoid counting the same source multiple times when it appears in different forms.
1787
1788 **Specification:**
1789
1790 **Deduplication Logic:**
1791
1792 1. **URL Normalization:**
1793 * Remove tracking parameters (?utm_source=...)
1794 * Normalize http/https
1795 * Normalize www/non-www
1796 * Handle redirects
1797
1798 2. **Content Similarity:**
1799 * If two sources have >90% text similarity → Same source
1800 * If one is subset of other → Same source
1801 * Use fuzzy matching for minor differences
1802
1803 3. **Cross-Domain Syndication:**
1804 * Detect wire service content (AP, Reuters)
1805 * Mark as single source if syndicated
1806 * Count original publication only
1807
1808 **Display:**
1809
1810 {{code}}
1811 Evidence Sources (3 unique, 5 total):
1812
1813 1. Original Article (NYTimes)
1814 - Also appeared in: WashPost, Guardian (syndicated)
1815
1816 2. Research Paper (Nature)
1817
1818 3. Official Statement (WHO)
1819 {{/code}}
1820
1821 **Acceptance Criteria:**
1822
1823 * ✅ URL normalization works
1824 * ✅ Content similarity detected
1825 * ✅ Syndicated content identified
1826 * ✅ Unique vs. total counts accurate
1827 * ✅ Improves evidence quality metrics
1828
1829
1830 == Additional Requirements (Lower Importance) ===== FR50: OSINT Toolkit Integration ===
1831
1832
1833
1834 **Fulfills:** Advanced media verification
1835
1836 **Purpose:** Integrate open-source intelligence tools for advanced verification.
1837
1838 **Tools to Integrate:**
1839 * InVID/WeVerify (video verification)
1840 * Bellingcat toolkit
1841 * Additional TBD based on V1.0 learnings
1842
1843 === FR51: Video Verification System ===
1844
1845
1846
1847 **Fulfills:** UN-27 (Visual claims), advanced media verification
1848
1849 **Purpose:** Verify video-based claims.
1850
1851 **Specification:**
1852 * Keyframe extraction
1853 * Reverse video search
1854 * Deepfake detection (AI-powered)
1855 * Metadata analysis
1856 * Acoustic signature analysis
1857
1858 === FR52: Interactive Detection Training ===
1859
1860
1861
1862 **Fulfills:** Media literacy education
1863
1864 **Purpose:** Teach users to identify misinformation.
1865
1866 **Specification:**
1867 * Interactive tutorials
1868 * Practice exercises
1869 * Detection quizzes
1870 * Gamification elements
1871
1872 === FR53: Cross-Organizational Sharing ===
1873
1874
1875
1876 **Fulfills:** Collaboration with other fact-checkers
1877
1878 **Purpose:** Share findings with IFCN/EFCSN members.
1879
1880 **Specification:**
1881 * API for fact-checking organizations
1882 * Structured data exchange
1883 * Privacy controls
1884 * Attribution requirements
1885
1886
1887 == Summary ==
1888
1889 **V1.0 Critical Requirements (Must Have):**
1890
1891 * FR44: ClaimReview Schema ✅
1892 * FR45: Corrections Notification ✅
1893 * FR46: Image Verification ✅
1894 * FR47: Archive.org Integration ✅
1895 * FR48: Contributor Safety ✅
1896 * FR49: A/B Testing ✅
1897 * FR54: Evidence Deduplication ✅
1898 * NFR11: Quality Assurance Framework ✅
1899 * NFR12: Security Controls ✅
1900 * NFR13: Quality Metrics Dashboard ✅
1901
1902 **V1.1+ (Future):**
1903
1904 * FR50: OSINT Integration
1905 * FR51: Video Verification
1906 * FR52: Detection Training
1907 * FR53: Cross-Org Sharing
1908
1909
1910 **Total:** 11 critical requirements for V1.0
1911
1912 === FR54: Evidence Deduplication ===
1913
1914
1915
1916 **Fulfills:** Accurate evidence counting, quality metrics
1917
1918 **Purpose:** Avoid counting the same source multiple times when it appears in different forms.
1919
1920 **Specification:**
1921
1922 **Deduplication Logic:**
1923
1924 1. **URL Normalization:**
1925 * Remove tracking parameters (?utm_source=...)
1926 * Normalize http/https
1927 * Normalize www/non-www
1928 * Handle redirects
1929
1930 2. **Content Similarity:**
1931 * If two sources have >90% text similarity → Same source
1932 * If one is subset of other → Same source
1933 * Use fuzzy matching for minor differences
1934
1935 3. **Cross-Domain Syndication:**
1936 * Detect wire service content (AP, Reuters)
1937 * Mark as single source if syndicated
1938 * Count original publication only
1939
1940 **Display:**
1941
1942 {{code}}
1943 Evidence Sources (3 unique, 5 total):
1944
1945 1. Original Article (NYTimes)
1946 - Also appeared in: WashPost, Guardian (syndicated)
1947
1948 2. Research Paper (Nature)
1949
1950 3. Official Statement (WHO)
1951 {{/code}}
1952
1953 **Acceptance Criteria:**
1954
1955 * ✅ URL normalization works
1956 * ✅ Content similarity detected
1957 * ✅ Syndicated content identified
1958 * ✅ Unique vs. total counts accurate
1959 * ✅ Improves evidence quality metrics
1960
1961
1962 == Additional Requirements (Lower Importance) ===== FR7: Automated Verdicts (Enhanced with Quality Gates) ===
1963
1964 **POC1+ Enhancement:**
1965
1966 After AKEL generates verdict, it passes through quality gates:
1967
1968 {{code}}
1969 Workflow:
1970 1. Extract claims
1971 ↓
1972 2. [GATE 1] Validate fact-checkable
1973 ↓
1974 3. Generate scenarios
1975 ↓
1976 4. Generate verdicts
1977 ↓
1978 5. [GATE 4] Validate confidence
1979 ↓
1980 6. Display to user
1981 {{/code}}
1982
1983 **Updated Verdict States:**
1984 * PUBLISHED
1985 * INSUFFICIENT_EVIDENCE
1986 * NON_FACTUAL_CLAIM
1987 * PROCESSING
1988 * ERROR
1989
1990 === FR4: Analysis Summary (Enhanced with Quality Metadata) ===
1991
1992 **POC1+ Enhancement:**
1993
1994 Display quality indicators:
1995
1996 {{code}}
1997 Analysis Summary:
1998 Verifiable Claims: 3/5
1999 High Confidence Verdicts: 1
2000 Medium Confidence: 2
2001 Evidence Sources: 12
2002 Avg Source Quality: 0.73
2003 Quality Score: 8.5/10
2004 {{/code}}