Skip to Content

Wiki source code of Requirements

Last modified by Robert Schaub on 2025/12/23 16:13
Show last authors
1 = Requirements =
2
3
4 {{info}}
5 **Phase Assignments:** See [[Requirements Roadmap Matrix>>Test.FactHarbor V0\.9\.83.Roadmap.Requirements-Roadmap-Matrix.WebHome]] for which requirements are implemented in which phases.
6 {{/info}}
7
8 **This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.**
9
10 **Core Philosophy:** Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically.
11
12 == Navigation ==
13
14 * **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need from FactHarbor (drives these requirements)
15 * **This page** - How we fulfill those needs through system design
16
17 (% class="box infomessage" %)
18 (((
19 **How to read this page:**
20
21 1. **User Needs drive Requirements**: See [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] for what users need
22 2. **Requirements define implementation**: This page shows how we fulfill those needs
23 3. **Functional Requirements (FR)**: Specific features and capabilities
24 4. **Non-Functional Requirements (NFR)**: Quality attributes (performance, security, etc.)
25
26 Each requirement references which User Needs it fulfills.
27 )))
28
29 == 1. Roles ==
30
31 **Fulfills**: UN-12 (Submit claims), UN-13 (Cite verdicts), UN-14 (API access)
32
33 FactHarbor uses three simple roles plus a reputation system.
34
35 === 1.1 Reader ===
36
37 **Who**: Anyone (no login required)
38
39 **Can**:
40
41 * Browse and search claims
42 * View scenarios, evidence, verdicts, and confidence scores
43 * Flag issues or errors
44 * Use filters, search, and visualization tools
45 * Submit claims automatically (new claims added if not duplicates)
46
47 **Cannot**:
48
49 * Modify content
50 * Access edit history details
51
52 **User Needs served**: UN-1 (Trust assessment), UN-2 (Claim verification), UN-3 (Article summary with FactHarbor analysis summary), UN-4 (Social media fact-checking), UN-5 (Source tracing), UN-7 (Evidence transparency), UN-8 (Understanding disagreement), UN-12 (Submit claims), UN-17 (In-article highlighting)
53
54 === 1.2 Contributor ===
55
56 **Who**: Registered users (earns reputation through contributions)
57
58 **Can**:
59
60 * Everything a Reader can do
61 * Edit claims, evidence, and scenarios
62 * Add sources and citations
63 * Suggest improvements to AI-generated content
64 * Participate in discussions
65 * Earn reputation points for quality contributions
66
67 **Reputation System**:
68
69 * New contributors: Limited edit privileges
70 * Established contributors (established reputation): Full edit access
71 * Trusted contributors (substantial reputation): Can approve certain changes
72 * Reputation earned through: Accepted edits, helpful flags, quality contributions
73 * Reputation lost through: Reverted edits, invalid flags, abuse
74
75 **Cannot**:
76
77 * Delete or hide content (only moderators)
78 * Override moderation decisions
79
80 **User Needs served**: UN-13 (Cite and contribute)
81
82 === 1.3 Moderator ===
83
84 **Who**: Trusted community members with proven track record, appointed by governance board
85
86 **Can**:
87
88 * Review flagged content
89 * Hide harmful or abusive content
90 * Resolve disputes between contributors
91 * Issue warnings or temporary bans
92 * Make final decisions on content disputes
93 * Access full audit logs
94
95 **Cannot**:
96
97 * Change governance rules
98 * Permanently ban users without board approval
99 * Override technical quality gates
100
101 **Note**: Small team (3-5 initially), supported by automated moderation tools.
102
103 === 1.4 Domain Trusted Contributors (Optional, Task-Specific) ===
104
105 **Who**: Subject matter specialists invited for specific high-stakes disputes
106
107 **Not a permanent role**: Contacted externally when needed for contested claims in their domain
108
109 **When used**:
110
111 * Medical claims with life/safety implications
112 * Legal interpretations with significant impact
113 * Scientific claims with high controversy
114 * Technical claims requiring specialized knowledge
115
116 **Process**:
117
118 * Moderator identifies need for expert input
119 * Contact expert externally (don't require them to be users)
120 * Trusted Contributor provides written opinion with sources
121 * Opinion added to claim record
122 * Trusted Contributor acknowledged in claim
123
124 **User Needs served**: UN-16 (Expert validation status)
125
126 == 2. Content States ==
127
128 **Fulfills**: UN-1 (Trust indicators), UN-16 (Review status transparency)
129
130 FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping.
131
132 === 2.1 Published ===
133
134 **Status**: Visible to all users
135
136 **Includes**:
137
138 * AI-generated analyses (default state)
139 * User-contributed content
140 * Edited/improved content
141
142 **Quality Indicators** (displayed with content):
143
144 * **Confidence Score**: 0-100% (AI's confidence in analysis)
145 * **Source Quality Score**: 0-100% (based on source track record)
146 * **Controversy Flag**: If high dispute/edit activity
147 * **Completeness Score**: % of expected fields filled
148 * **Last Updated**: Date of most recent change
149 * **Edit Count**: Number of revisions
150 * **Review Status**: AI-generated / Human-reviewed / Expert-validated
151
152 **Automatic Warnings**:
153
154 * Confidence < 60%: "Low confidence - use caution"
155 * Source quality < 40%: "Sources may be unreliable"
156 * High controversy: "Disputed - multiple interpretations exist"
157 * Medical/Legal/Safety domain: "Seek professional advice"
158
159 **User Needs served**: UN-1 (Trust score), UN-9 (Methodology transparency), UN-15 (Evolution timeline), UN-16 (Review status)
160
161 === 2.2 Hidden ===
162
163 **Status**: Not visible to regular users (only to moderators)
164
165 **Reasons**:
166
167 * Spam or advertising
168 * Personal attacks or harassment
169 * Illegal content
170 * Privacy violations
171 * Deliberate misinformation (verified)
172 * Abuse or harmful content
173
174 **Process**:
175
176 * Automated detection flags for moderator review
177 * Moderator confirms and hides
178 * Original author notified with reason
179 * Can appeal to board if disputes moderator decision
180
181 **Note**: Content is hidden, not deleted (for audit trail)
182
183 == 3. Contribution Rules ==
184
185 === 3.1 All Contributors Must ===
186
187 * Provide sources for factual claims
188 * Use clear, neutral language in FactHarbor's own summaries
189 * Respect others and maintain civil discourse
190 * Accept community feedback constructively
191 * Focus on improving quality, not protecting ego
192
193 === 3.2 AKEL (AI System) ===
194
195 **AKEL is the primary system**. Human contributions supplement and train AKEL.
196
197 **AKEL Must**:
198
199 * Mark all outputs as AI-generated
200 * Display confidence scores prominently
201 * Provide source citations
202 * Flag uncertainty clearly
203 * Identify contradictions in evidence
204 * Learn from human corrections
205
206 **When AKEL Makes Errors**:
207
208 1. Capture the error pattern (what, why, how common)
209 2. Improve the system (better prompt, model, validation)
210 3. Re-process affected claims automatically
211 4. Measure improvement (did quality increase?)
212
213 **Human Role**: Train AKEL through corrections, not replace AKEL
214
215 === 3.3 Contributors Should ===
216
217 * Improve clarity and structure
218 * Add missing sources
219 * Flag errors for system improvement
220 * Suggest better ways to present information
221 * Participate in quality discussions
222
223 === 3.4 Moderators Must ===
224
225 * Be impartial
226 * Document moderation decisions
227 * Respond to appeals promptly
228 * Use automated tools to scale efforts
229 * Focus on abuse/harm, not routine quality control
230
231 == 4. Quality Standards ==
232
233 **Fulfills**: UN-5 (Source reliability), UN-6 (Publisher track records), UN-7 (Evidence transparency), UN-9 (Methodology transparency)
234
235 === 4.1 Source Requirements ===
236
237 **Track Record Over Credentials**:
238
239 * Sources evaluated by historical accuracy
240 * Correction policy matters
241 * Independence from conflicts of interest
242 * Methodology transparency
243
244 **Source Quality Database**:
245
246 * Automated tracking of source accuracy
247 * Correction frequency
248 * Reliability score (updated continuously)
249 * Users can see source track record
250
251 **No automatic trust** for government, academia, or media - all evaluated by track record.
252
253 **User Needs served**: UN-5 (Source provenance), UN-6 (Publisher reliability)
254
255 === 4.2 Claim Requirements ===
256
257 * Clear subject and assertion
258 * Verifiable with available information
259 * Sourced (or explicitly marked as needing sources)
260 * Neutral language in FactHarbor summaries
261 * Appropriate context provided
262
263 **User Needs served**: UN-2 (Claim extraction and verification)
264
265 === 4.3 Evidence Requirements ===
266
267 * Publicly accessible (or explain why not)
268 * Properly cited with attribution
269 * Relevant to claim being evaluated
270 * Original source preferred over secondary
271
272 **User Needs served**: UN-7 (Evidence transparency)
273
274 === 4.4 Confidence Scoring ===
275
276 **Automated confidence calculation based on**:
277
278 * Source quality scores
279 * Evidence consistency
280 * Contradiction detection
281 * Completeness of analysis
282 * Historical accuracy of similar claims
283
284 **Thresholds**:
285
286 * < 40%: Too low to publish (needs improvement)
287 * 40-60%: Published with "Low confidence" warning
288 * 60-80%: Published as standard
289 * 80-100%: Published as "High confidence"
290
291 **User Needs served**: UN-1 (Trust assessment), UN-9 (Methodology transparency)
292
293 == 5. Automated Risk Scoring ==
294
295 **Fulfills**: UN-10 (Manipulation detection), UN-16 (Appropriate review level)
296
297 **Replace manual risk tiers with continuous automated scoring**.
298
299 === 5.1 Risk Score Calculation ===
300
301 **Factors** (weighted algorithm):
302
303 * **Domain sensitivity**: Medical, legal, safety auto-flagged higher
304 * **Potential impact**: Views, citations, spread
305 * **Controversy level**: Flags, disputes, edit wars
306 * **Uncertainty**: Low confidence, contradictory evidence
307 * **Source reliability**: Track record of sources used
308
309 **Score**: 0-100 (higher = more risk)
310
311 === 5.2 Automated Actions ===
312
313 * **Score > 80**: Flag for moderator review before publication
314 * **Score 60-80**: Publish with prominent warnings
315 * **Score 40-60**: Publish with standard warnings
316 * **Score < 40**: Publish normally
317
318 **Continuous monitoring**: Risk score recalculated as new information emerges
319
320 **User Needs served**: UN-10 (Detect manipulation tactics), UN-16 (Review status)
321
322 == 6. System Improvement Process ==
323
324 **Core principle**: Fix the system, not just the data.
325
326 === 6.1 Error Capture ===
327
328 **When users flag errors or make corrections**:
329
330 1. What was wrong? (categorize)
331 2. What should it have been?
332 3. Why did the system fail? (root cause)
333 4. How common is this pattern?
334 5. Store in ErrorPattern table (improvement queue)
335
336 === 6.2 Continuous Improvement Cycle ===
337
338 1. **Review**: Analyze top error patterns
339 2. **Develop**: Create fix (prompt, model, validation)
340 3. **Test**: Validate fix on sample claims
341 4. **Deploy**: Roll out if quality improves
342 5. **Re-process**: Automatically update affected claims
343 6. **Monitor**: Track quality metrics
344
345 === 6.3 Quality Metrics Dashboard ===
346
347 **Track continuously**:
348
349 * Error rate by category
350 * Source quality distribution
351 * Confidence score trends
352 * User flag rate (issues found)
353 * Correction acceptance rate
354 * Re-work rate
355 * Claims processed per hour
356
357 **Goal**: continuous improvement in error rate
358
359 == 7. Automated Quality Monitoring ==
360
361 **Replace manual audit sampling with automated monitoring**.
362
363 === 7.1 Continuous Metrics ===
364
365 * **Source quality**: Track record database
366 * **Consistency**: Contradiction detection
367 * **Clarity**: Readability scores
368 * **Completeness**: Field validation
369 * **Accuracy**: User corrections tracked
370
371 === 7.2 Anomaly Detection ===
372
373 **Automated alerts for**:
374
375 * Sudden quality drops
376 * Unusual patterns
377 * Contradiction clusters
378 * Source reliability changes
379 * User behavior anomalies
380
381 === 7.3 Targeted Review ===
382
383 * Review only flagged items
384 * Random sampling for calibration (not quotas)
385 * Learn from corrections to improve automation
386
387 == 8. Functional Requirements ==
388
389 This section defines specific features that fulfill user needs.
390
391 === 8.1 Claim Intake & Normalization ===
392
393 ==== FR1 — Claim Intake ====
394
395 **Fulfills**: UN-2 (Claim extraction), UN-4 (Quick fact-checking), UN-12 (Submit claims)
396
397 * Users submit claims via simple form or API
398 * Claims can be text, URL, or image
399 * Duplicate detection (semantic similarity)
400 * Auto-categorization by domain
401
402 ==== FR2 — Claim Normalization ====
403
404 **Fulfills**: UN-2 (Claim verification)
405
406 * Standardize to clear assertion format
407 * Extract key entities (who, what, when, where)
408 * Identify claim type (factual, predictive, evaluative)
409 * Link to existing similar claims
410
411 ==== FR3 — Claim Classification ====
412
413 **Fulfills**: UN-11 (Filtered research)
414
415 * Domain: Politics, Science, Health, etc.
416 * Type: Historical fact, current stat, prediction, etc.
417 * Risk score: Automated calculation
418 * Complexity: Simple, moderate, complex
419
420 === 8.2 Scenario System ===
421
422 ==== FR4 — Scenario Generation ====
423
424 **Fulfills**: UN-2 (Context-dependent verification), UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
425
426 **Automated scenario creation**:
427
428 * AKEL analyzes claim and generates likely scenarios (use-cases and contexts)
429 * Each scenario includes: assumptions, definitions, boundaries, evidence context
430 * Users can flag incorrect scenarios
431 * System learns from corrections
432
433 **Key Concept**: Scenarios represent different interpretations or contexts (e.g., "Clinical trials with healthy adults" vs. "Real-world data with diverse populations")
434
435 ==== FR5 — Evidence Linking ====
436
437 **Fulfills**: UN-5 (Source tracing), UN-7 (Evidence transparency)
438
439 * Automated evidence discovery from sources
440 * Relevance scoring
441 * Contradiction detection
442 * Source quality assessment
443
444 ==== FR6 — Scenario Comparison ====
445
446 **Fulfills**: UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
447
448 * Side-by-side comparison interface
449 * Highlight key differences between scenarios
450 * Show evidence supporting each scenario
451 * Display confidence scores per scenario
452
453 === 8.3 Verdicts & Analysis ===
454
455 ==== FR7 — Automated Verdicts ====
456
457 **Fulfills**: UN-1 (Trust score), UN-2 (Verification verdicts), UN-3 (Article summary with FactHarbor analysis summary), UN-13 (Cite verdicts)
458
459 * AKEL generates verdict based on evidence within each scenario
460 * **Likelihood range** displayed (e.g., "0.70-0.85 (likely true)") - NOT binary true/false
461 * **Uncertainty factors** explicitly listed (e.g., "Small sample sizes", "Long-term effects unknown")
462 * Confidence score displayed prominently
463 * Source quality indicators shown
464 * Contradictions noted
465 * Uncertainty acknowledged
466
467 **Key Innovation**: Detailed probabilistic verdicts with explicit uncertainty, not binary judgments
468
469 ==== FR8 — Time Evolution ====
470
471 **Fulfills**: UN-15 (Verdict evolution timeline)
472
473 * Claims and verdicts update as new evidence emerges
474 * Version history maintained for all verdicts
475 * Changes highlighted
476 * Confidence score trends visible
477 * Users can see "as of date X, what did we know?"
478
479 === 8.4 User Interface & Presentation ===
480
481 ==== FR12 — Two-Panel Summary View (Article Summary with FactHarbor Analysis Summary) ====
482
483 **Fulfills**: UN-3 (Article Summary with FactHarbor Analysis Summary)
484
485 **Purpose**: Provide side-by-side comparison of what a document claims vs. FactHarbor's complete analysis of its credibility
486
487 **Left Panel: Article Summary**:
488
489 * Document title, source, and claimed credibility
490 * "The Big Picture" - main thesis or position change
491 * "Key Findings" - structured summary of document's main claims
492 * "Reasoning" - document's explanation for positions
493 * "Conclusion" - document's bottom line
494
495 **Right Panel: FactHarbor Analysis Summary**:
496
497 * FactHarbor's independent source credibility assessment
498 * Claim-by-claim verdicts with confidence scores
499 * Methodology assessment (strengths, limitations)
500 * Overall verdict on document quality
501 * Analysis ID for reference
502
503 **Design Principles**:
504
505 * No scrolling required - both panels visible simultaneously
506 * Visual distinction between "what they say" and "FactHarbor's analysis"
507 * Color coding for verdicts (supported, uncertain, refuted)
508 * Confidence percentages clearly visible
509 * Mobile responsive (panels stack vertically on small screens)
510
511 **Implementation Notes**:
512
513 * Generated automatically by AKEL for every analyzed document
514 * Updates when verdict evolves (maintains version history)
515 * Exportable as standalone summary report
516 * Shareable via permanent URL
517
518 ==== FR13 — In-Article Claim Highlighting ====
519
520 **Fulfills**: UN-17 (In-article claim highlighting)
521
522 **Purpose**: Enable readers to quickly assess claim credibility while reading by visually highlighting factual claims with color-coded indicators
523
524 ==== Visual Example: Article with Highlighted Claims ====
525
526 (% class="box" %)
527 (((
528 **Article: "New Study Shows Benefits of Mediterranean Diet"**
529
530 A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.
531
532 (% class="box successmessage" style="margin:10px 0;" %)
533 (((
534 🟢 **Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups**
535
536 (% style="font-size:0.9em; color:#666;" %)
537 ↑ WELL SUPPORTED • 87% confidence
538 [[Click for evidence details →]]
539
540
541 )))
542
543 The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
544
545 (% class="box warningmessage" style="margin:10px 0;" %)
546 (((
547 🟡 **Some experts believe this diet can completely prevent heart attacks**
548
549 (% style="font-size:0.9em; color:#666;" %)
550 ↑ UNCERTAIN • 45% confidence
551 Overstated - evidence shows risk reduction, not prevention
552 [[Click for details →]]
553
554
555 )))
556
557 Dr. Maria Rodriguez, lead researcher, recommends incorporating more olive oil, fish, and vegetables into daily meals.
558
559 (% class="box errormessage" style="margin:10px 0;" %)
560 (((
561 🔴 **The study proves that saturated fats cause heart disease**
562
563 (% style="font-size:0.9em; color:#666;" %)
564 ↑ REFUTED • 15% confidence
565 Claim not supported by study design; correlation ≠ causation
566 [[Click for counter-evidence →]]
567
568
569 )))
570
571 Participants also reported feeling more energetic and experiencing better sleep quality, though these were secondary measures.
572 )))
573
574 **Legend:**
575
576 * 🟢 = Well-supported claim (confidence ≥75%)
577 * 🟡 = Uncertain claim (confidence 40-74%)
578 * 🔴 = Refuted/unsupported claim (confidence <40%)
579 * Plain text = Non-factual content (context, opinions, recommendations)
580
581 ==== Tooltip on Hover/Click ====
582
583 (% class="box infomessage" %)
584 (((
585 **FactHarbor Analysis**
586
587 **Claim:**
588 "Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease"
589
590 **Verdict:** WELL SUPPORTED
591 **Confidence:** 87%
592
593 **Evidence Summary:**
594
595 * Meta-analysis of 12 RCTs confirms 23-28% risk reduction
596 * Consistent findings across multiple populations
597 * Published in peer-reviewed journal (high credibility)
598
599 **Uncertainty Factors:**
600
601 * Exact percentage varies by study (20-30% range)
602
603 [[View Full Analysis →]]
604 )))
605
606 **Color-Coding System**:
607
608 * **Green**: Well-supported claims (confidence ≥75%, strong evidence)
609 * **Yellow/Orange**: Uncertain claims (confidence 40-74%, conflicting or limited evidence)
610 * **Red**: Refuted or unsupported claims (confidence <40%, contradicted by evidence)
611 * **Gray/Neutral**: Non-factual content (opinions, questions, procedural text)
612
613 ==== Interactive Highlighting Example (Detailed View) ====
614
615 (% style="width:100%; border-collapse:collapse;" %)
616 |=**Article Text**|=**Status**|=**Analysis**
617 |(((
618 A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.
619 )))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Context - no highlighting
620 |(((
621 //Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups//
622 )))|(% style="background-color:#D4EDDA; text-align:center; padding:8px;" %)🟢 **WELL SUPPORTED**|(((
623 **87% confidence**
624
625 Meta-analysis of 12 RCTs confirms 23-28% risk reduction
626
627 [[View Full Analysis]]
628 )))
629 |(((
630 The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
631 )))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Methodology - no highlighting
632 |(((
633 //Some experts believe this diet can completely prevent heart attacks//
634 )))|(% style="background-color:#FFF3CD; text-align:center; padding:8px;" %)🟡 **UNCERTAIN**|(((
635 **45% confidence**
636
637 Overstated - evidence shows risk reduction, not prevention
638
639 [[View Details]]
640 )))
641 |(((
642 Dr. Rodriguez recommends incorporating more olive oil, fish, and vegetables into daily meals.
643 )))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Recommendation - no highlighting
644 |(((
645 //The study proves that saturated fats cause heart disease//
646 )))|(% style="background-color:#F8D7DA; text-align:center; padding:8px;" %)🔴 **REFUTED**|(((
647 **15% confidence**
648
649 Claim not supported by study; correlation ≠ causation
650
651 [[View Counter-Evidence]]
652 )))
653
654 **Design Notes:**
655
656 * Highlighted claims use italics to distinguish from plain text
657 * Color backgrounds match XWiki message box colors (success/warning/error)
658 * Status column shows verdict prominently
659 * Analysis column provides quick summary with link to details
660
661 **User Actions**:
662
663 * **Hover** over highlighted claim → Tooltip appears
664 * **Click** highlighted claim → Detailed analysis modal/panel
665 * **Toggle** button to turn highlighting on/off
666 * **Keyboard**: Tab through highlighted claims
667
668 **Interaction Design**:
669
670 * Hover/click on highlighted claim → Show tooltip with:
671 * Claim text
672 * Verdict (e.g., "WELL SUPPORTED")
673 * Confidence score (e.g., "85%")
674 * Brief evidence summary
675 * Link to detailed analysis
676 * Toggle highlighting on/off (user preference)
677 * Adjustable color intensity for accessibility
678
679 **Technical Requirements**:
680
681 * Real-time highlighting as page loads (non-blocking)
682 * Claim boundary detection (start/end of assertion)
683 * Handle nested or overlapping claims
684 * Preserve original article formatting
685 * Work with various content formats (HTML, plain text, PDFs)
686
687 **Performance Requirements**:
688
689 * Highlighting renders within 500ms of page load
690 * No perceptible delay in reading experience
691 * Efficient DOM manipulation (avoid reflows)
692
693 **Accessibility**:
694
695 * Color-blind friendly palette (use patterns/icons in addition to color)
696 * Screen reader compatible (ARIA labels for claim credibility)
697 * Keyboard navigation to highlighted claims
698
699 **Implementation Notes**:
700
701 * Claims extracted and analyzed by AKEL during initial processing
702 * Highlighting data stored as annotations with byte offsets
703 * Client-side rendering of highlights based on verdict data
704 * Mobile responsive (tap instead of hover)
705
706 === 8.5 Workflow & Moderation ===
707
708 ==== FR9 — Publication Workflow ====
709
710 **Fulfills**: UN-1 (Fast access to verified content), UN-16 (Clear review status)
711
712 **Simple flow**:
713
714 1. Claim submitted
715 2. AKEL processes (automated)
716 3. If confidence > threshold: Publish (labeled as AI-generated)
717 4. If confidence < threshold: Flag for improvement
718 5. If risk score > threshold: Flag for moderator
719
720 **No multi-stage approval process**
721
722 ==== FR10 — Moderation ====
723
724 **Focus on abuse, not routine quality**:
725
726 * Automated abuse detection
727 * Moderators handle flags
728 * Quick response to harmful content
729 * Minimal involvement in routine content
730
731 ==== FR11 — Audit Trail ====
732
733 **Fulfills**: UN-14 (API access to histories), UN-15 (Evolution tracking)
734
735 * All edits logged
736 * Version history public
737 * Moderation decisions documented
738 * System improvements tracked
739
740 == 9. Non-Functional Requirements ==
741
742 === 9.1 NFR1 — Performance ===
743
744 **Fulfills**: UN-4 (Fast fact-checking), UN-11 (Responsive filtering)
745
746 * Claim processing: < 30 seconds
747 * Search response: < 2 seconds
748 * Page load: < 3 seconds
749 * 99% uptime
750
751 === 9.2 NFR2 — Scalability ===
752
753 **Fulfills**: UN-14 (API access at scale)
754
755 * Handle 10,000 claims initially
756 * Scale to 1M+ claims
757 * Support 100K+ concurrent users
758 * Automated processing scales linearly
759
760 === 9.3 NFR3 — Transparency ===
761
762 **Fulfills**: UN-7 (Evidence transparency), UN-9 (Methodology transparency), UN-13 (Citable verdicts), UN-15 (Evolution visibility)
763
764 * All algorithms open source
765 * All data exportable
766 * All decisions documented
767 * Quality metrics public
768
769 === 9.4 NFR4 — Security & Privacy ===
770
771 * Follow [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
772 * Secure authentication
773 * Data encryption
774 * Regular security audits
775
776 === 9.5 NFR5 — Maintainability ===
777
778 * Modular architecture
779 * Automated testing
780 * Continuous integration
781 * Comprehensive documentation
782
783 === NFR11: AKEL Quality Assurance Framework ===
784
785 **Fulfills:** AI safety, IFCN methodology transparency
786
787 **Specification:**
788
789 Multi-layer AI quality gates to detect hallucinations, low-confidence results, and logical inconsistencies.
790
791 ==== Quality Gate 1: Claim Extraction Validation ====
792
793 **Purpose:** Ensure extracted claims are factual assertions (not opinions/predictions)
794
795 **Checks:**
796
797 1. **Factual Statement Test:** Is this verifiable? (Yes/No)
798 2. **Opinion Detection:** Contains hedging language? ("I think", "probably", "best")
799 3. **Future Prediction Test:** Makes claims about future events?
800 4. **Specificity Score:** Contains specific entities, numbers, dates?
801
802 **Thresholds:**
803
804 * Factual: Must be "Yes"
805 * Opinion markers: <2 hedging phrases
806 * Specificity: ≥3 specific elements
807
808 **Action if Failed:** Flag as "Non-verifiable", do NOT generate verdict
809
810 ==== Quality Gate 2: Evidence Relevance Validation ====
811
812 **Purpose:** Ensure AI-linked evidence actually relates to claim
813
814 **Checks:**
815
816 1. **Semantic Similarity Score:** Evidence vs. claim (embeddings)
817 2. **Entity Overlap:** Shared people/places/things?
818 3. **Topic Relevance:** Discusses claim subject?
819
820 **Thresholds:**
821
822 * Similarity: ≥0.6 (cosine similarity)
823 * Entity overlap: ≥1 shared entity
824 * Topic relevance: ≥0.5
825
826 **Action if Failed:** Discard irrelevant evidence
827
828 ==== Quality Gate 3: Scenario Coherence Check ====
829
830 **Purpose:** Validate scenario assumptions are logical and complete
831
832 **Checks:**
833
834 1. **Completeness:** All required fields populated
835 2. **Internal Consistency:** Assumptions don't contradict
836 3. **Distinguishability:** Scenarios meaningfully different
837
838 **Thresholds:**
839
840 * Required fields: 100%
841 * Contradiction score: <0.3
842 * Scenario similarity: <0.8
843
844 **Action if Failed:** Merge duplicates, reduce confidence -20%
845
846 ==== Quality Gate 4: Verdict Confidence Assessment ====
847
848 **Purpose:** Only publish high-confidence verdicts
849
850 **Checks:**
851
852 1. **Evidence Count:** Minimum 2 sources
853 2. **Source Quality:** Average reliability ≥0.6
854 3. **Evidence Agreement:** Supporting vs. contradicting ≥0.6
855 4. **Uncertainty Factors:** Hedging in reasoning
856
857 **Confidence Tiers:**
858
859 * **HIGH (80-100%):** ≥3 sources, ≥0.7 quality, ≥80% agreement
860 * **MEDIUM (50-79%):** ≥2 sources, ≥0.6 quality, ≥60% agreement
861 * **LOW (0-49%):** <2 sources OR low quality/agreement
862 * **INSUFFICIENT:** <2 sources → DO NOT PUBLISH
863
864 **Implementation Phases:**
865
866 * **POC1:** Gates 1 & 4 only (basic validation)
867 * **POC2:** All 4 gates (complete framework)
868 * **V1.0:** Hardened with <5% hallucination rate
869
870 **Acceptance Criteria:**
871
872 * ✅ All gates operational
873 * ✅ Hallucination rate <5%
874 * ✅ Quality metrics public
875
876 === NFR12: Security Controls ===
877
878 **Fulfills:** Data protection, system integrity, user privacy, production readiness
879
880
881 **Purpose:** Protect FactHarbor systems, user data, and operations from security threats, ensuring production-grade security posture.
882
883 **Specification:**
884
885 ==== API Security ====
886
887 **Rate Limiting:**
888
889 * **Analysis endpoints:** 100 requests/hour per IP
890 * **Read endpoints:** 1,000 requests/hour per IP
891 * **Search:** 500 requests/hour per IP
892 * **Authenticated users:** 5x higher limits
893 * **Burst protection:** Max 10 requests/second
894
895 **Authentication & Authorization:**
896
897 * **API Keys:** Required for programmatic access
898 * **JWT tokens:** For user sessions (1-hour expiry)
899 * **OAuth2:** For third-party integrations
900 * **Role-Based Access Control (RBAC):**
901 * Public: Read-only access to published claims
902 * Contributor: Submit claims, provide evidence
903 * Moderator: Review contributions, manage quality
904 * Admin: System configuration, user management
905
906 **CORS Policies:**
907
908 * Whitelist approved domains only
909 * No wildcard origins in production
910 * Credentials required for sensitive endpoints
911
912 **Input Sanitization:**
913
914 * Validate all user input against schemas
915 * Sanitize HTML/JavaScript in text submissions
916 * Prevent SQL injection (use parameterized queries)
917 * Prevent command injection (no shell execution of user input)
918 * Max request size: 10MB
919 * File upload restrictions: Whitelist file types, scan for malware
920
921 -----
922
923 ==== Data Security ====
924
925 **Encryption at Rest:**
926
927 * Database encryption using AES-256
928 * Encrypted backups
929 * Key management via cloud provider KMS (AWS KMS, Google Cloud KMS)
930 * Regular key rotation (90-day cycle)
931
932 **Encryption in Transit:**
933
934 * HTTPS/TLS 1.3 only (no TLS 1.0/1.1)
935 * Strong cipher suites only
936 * HSTS (HTTP Strict Transport Security) enabled
937 * Certificate pinning for mobile apps
938
939 **Secure Credential Storage:**
940
941 * Passwords hashed with bcrypt (cost factor 12+)
942 * API keys encrypted in database
943 * Secrets stored in environment variables (never in code)
944 * Use secrets manager (AWS Secrets Manager, HashiCorp Vault)
945
946 **Data Privacy:**
947
948 * Minimal data collection (privacy by design)
949 * User data deletion on request (GDPR compliance)
950 * PII encryption in database
951 * Anonymize logs (no PII in log files)
952
953 -----
954
955 ==== Application Security ====
956
957 **OWASP Top 10 Compliance:**
958
959 1. **Broken Access Control:** RBAC implementation, path traversal prevention
960 2. **Cryptographic Failures:** Strong encryption, secure key management
961 3. **Injection:** Parameterized queries, input validation
962 4. **Insecure Design:** Security review of all features
963 5. **Security Misconfiguration:** Hardened defaults, security headers
964 6. **Vulnerable Components:** Dependency scanning (see below)
965 7. **Authentication Failures:** Strong password policy, MFA support
966 8. **Data Integrity Failures:** Signature verification, checksums
967 9. **Security Logging Failures:** Comprehensive audit logs
968 10. **Server-Side Request Forgery:** URL validation, whitelist domains
969
970 **Security Headers:**
971
972 * `Content-Security-Policy`: Strict CSP to prevent XSS
973 * `X-Frame-Options`: DENY (prevent clickjacking)
974 * `X-Content-Type-Options`: nosniff
975 * `Referrer-Policy`: strict-origin-when-cross-origin
976 * `Permissions-Policy`: Restrict browser features
977
978 **Dependency Vulnerability Scanning:**
979
980 * **Tools:** Snyk, Dependabot, npm audit, pip-audit
981 * **Frequency:** Daily automated scans
982 * **Action:** Patch critical vulnerabilities within 24 hours
983 * **Policy:** No known high/critical CVEs in production
984
985 **Security Audits:**
986
987 * **Internal:** Quarterly security reviews
988 * **External:** Annual penetration testing by certified firm
989 * **Bug Bounty:** Public bug bounty program (V1.1+)
990 * **Compliance:** SOC 2 Type II certification target (V1.5)
991
992 -----
993
994 ==== Operational Security ====
995
996 **DDoS Protection:**
997
998 * CloudFlare or AWS Shield
999 * Rate limiting at CDN layer
1000 * Automatic IP blocking for abuse patterns
1001
1002 **Monitoring & Alerting:**
1003
1004 * Real-time security event monitoring
1005 * Alerts for:
1006 * Failed login attempts (>5 in 10 minutes)
1007 * API abuse patterns
1008 * Unusual data access patterns
1009 * Security scan detections
1010 * Integration with SIEM (Security Information and Event Management)
1011
1012 **Incident Response:**
1013
1014 * Documented incident response plan
1015 * Security incident classification (P1-P4)
1016 * On-call rotation for security issues
1017 * Post-mortem for all security incidents
1018 * Public disclosure policy (coordinated disclosure)
1019
1020 **Backup & Recovery:**
1021
1022 * Daily encrypted backups
1023 * 30-day retention period
1024 * Tested recovery procedures (quarterly)
1025 * Disaster recovery plan (RTO: 4 hours, RPO: 1 hour)
1026
1027 -----
1028
1029 ==== Compliance & Standards ====
1030
1031 **GDPR Compliance:**
1032
1033 * User consent management
1034 * Right to access data
1035 * Right to deletion
1036 * Data portability
1037 * Privacy policy published
1038
1039 **Accessibility:**
1040
1041 * WCAG 2.1 AA compliance
1042 * Screen reader compatibility
1043 * Keyboard navigation
1044 * Alt text for images
1045
1046 **Browser Support:**
1047
1048 * Modern browsers only (Chrome/Edge/Firefox/Safari latest 2 versions)
1049 * No IE11 support
1050
1051 **Acceptance Criteria:**
1052
1053 * ✅ Passes OWASP ZAP security scan (no high/critical findings)
1054 * ✅ All dependencies with known vulnerabilities patched
1055 * ✅ Penetration test completed with no critical findings
1056 * ✅ Rate limiting blocks abuse attempts
1057 * ✅ Encryption at rest and in transit verified
1058 * ✅ Security headers scored A+ on securityheaders.com
1059 * ✅ Incident response plan documented and tested
1060 * ✅ 95% uptime over 30-day period
1061
1062 === NFR13: Quality Metrics Transparency ===
1063
1064 **Fulfills:** User trust, transparency, continuous improvement, IFCN methodology transparency
1065
1066
1067 **Purpose:** Provide transparent, measurable quality metrics that demonstrate AKEL's performance and build user trust in automated fact-checking.
1068
1069 **Specification:**
1070
1071 ==== Component: Public Quality Dashboard ====
1072
1073 **Core Metrics to Display:**
1074
1075 *
1076 **
1077 **1. Verdict Quality Metrics
1078
1079 **TIGERScore (Fact-Checking Quality):**
1080
1081 * **Definition:** Measures how well generated verdicts match expert fact-checker judgments
1082 * **Scale:** 0-100 (higher is better)
1083 * **Calculation:** Using TIGERScore framework (Truth-conditional accuracy, Informativeness, Generality, Evaluativeness, Relevance)
1084 * **Target:** Average ≥80 for production release
1085 * **Display:**
1086 {{code}}Verdict Quality (TIGERScore):
1087 Overall: 84.2 ▲ (+2.1 from last month)
1088
1089 Distribution:
1090 Excellent (>80): 67%
1091 Good (60-80): 28%
1092 Needs Improvement (<60): 5%
1093
1094 Trend: [Graph showing improvement over time]{{/code}}
1095
1096 **2. Hallucination & Faithfulness Metrics**
1097
1098 **AlignScore (Faithfulness to Evidence):**
1099
1100 * **Definition:** Measures how well verdicts align with actual evidence content
1101 * **Scale:** 0-1 (higher is better)
1102 * **Purpose:** Detect AI hallucinations (making claims not supported by evidence)
1103 * **Target:** Average ≥0.85, hallucination rate <5%
1104 * **Display:**
1105 {{code}}Evidence Faithfulness (AlignScore):
1106 Average: 0.87 ▼ (-0.02 from last month)
1107
1108 Hallucination Rate: 4.2%
1109 - Claims without evidence support: 3.1%
1110 - Misrepresented evidence: 1.1%
1111
1112 Action: Prompt engineering review scheduled{{/code}}
1113
1114 **3. Evidence Quality Metrics**
1115
1116 **Source Reliability:**
1117
1118 * Average source quality score (0-1 scale)
1119 * Distribution of high/medium/low quality sources
1120 * Publisher track record trends
1121
1122 **Evidence Coverage:**
1123
1124 * Average number of sources per claim
1125 * Percentage of claims with ≥2 sources (EFCSN minimum)
1126 * Geographic diversity of sources
1127
1128 **Display:**
1129 {{code}}Evidence Quality:
1130
1131 Average Sources per Claim: 4.2
1132 Claims with ≥2 sources: 94% (EFCSN compliant)
1133
1134 Source Quality Distribution:
1135 High quality (>0.8): 48%
1136 Medium quality (0.5-0.8): 43%
1137 Low quality (<0.5): 9%
1138
1139 Geographic Diversity: 23 countries represented{{/code}}
1140
1141 **4. Contributor Consensus Metrics** (when human reviewers involved)
1142
1143 **Inter-Rater Reliability (IRR):**
1144
1145 * **Calculation:** Cohen's Kappa or Fleiss' Kappa for multiple raters
1146 * **Scale:** 0-1 (higher is better)
1147 * **Interpretation:**
1148 * >0.8: Almost perfect agreement
1149 * 0.6-0.8: Substantial agreement
1150 * 0.4-0.6: Moderate agreement
1151 * <0.4: Poor agreement
1152 * **Target:** Maintain ≥0.7 (substantial agreement)
1153
1154 **Display:**
1155 {{code}}Contributor Consensus:
1156
1157 Inter-Rater Reliability (IRR): 0.73 (Substantial agreement)
1158 - Verdict agreement: 78%
1159 - Evidence quality agreement: 71%
1160 - Scenario structure agreement: 69%
1161
1162 Cases requiring moderator review: 12
1163 Moderator override rate: 8%{{/code}}
1164
1165 -----
1166
1167 ==== Quality Dashboard Implementation ====
1168
1169 **Dashboard Location:** `/quality-metrics`
1170
1171 **Update Frequency:**
1172
1173 * **POC2:** Weekly manual updates
1174 * **Beta 0:** Daily automated updates
1175 * **V1.0:** Real-time metrics (updated hourly)
1176
1177 **Dashboard Sections:**
1178
1179 1. **Overview:** Key metrics at a glance
1180 2. **Verdict Quality:** TIGERScore trends and distributions
1181 3. **Evidence Analysis:** Source quality and coverage
1182 4. **AI Performance:** Hallucination rates, AlignScore
1183 5. **Human Oversight:** Contributor consensus, review rates
1184 6. **System Health:** Processing times, error rates, uptime
1185
1186 **Example Dashboard Layout:**
1187
1188 {{code}}
1189 ┌─────────────────────────────────────────────────────────────┐
1190 │ FactHarbor Quality Metrics Last updated: │
1191 │ Public Dashboard 2 hours ago │
1192 └─────────────────────────────────────────────────────────────┘
1193
1194 📊 KEY METRICS
1195 ─────────────────────────────────────────────────────────────
1196 TIGERScore (Verdict Quality): 84.2 ▲ (+2.1)
1197 AlignScore (Faithfulness): 0.87 ▼ (-0.02)
1198 Hallucination Rate: 4.2% ✓ (Target: <5%)
1199 Average Sources per Claim: 4.2 ▲ (+0.3)
1200
1201 📈 TRENDS (30 days)
1202 ─────────────────────────────────────────────────────────────
1203 [Graph: TIGERScore trending upward]
1204 [Graph: Hallucination rate declining]
1205 [Graph: Evidence quality stable]
1206
1207 ⚠️ IMPROVEMENT TARGETS
1208 ─────────────────────────────────────────────────────────────
1209 1. Reduce hallucination rate to <3% (Current: 4.2%)
1210 2. Increase TIGERScore average to >85 (Current: 84.2)
1211 3. Maintain IRR >0.75 (Current: 0.73)
1212
1213 📄 DETAILED REPORTS
1214 ─────────────────────────────────────────────────────────────
1215 • Monthly Quality Report (PDF)
1216 • Methodology Documentation
1217 • AKEL Performance Analysis
1218 • Contributor Agreement Analysis
1219
1220 {{/code}}
1221
1222 -----
1223
1224 ==== Continuous Improvement Feedback Loop ====
1225
1226 **How Metrics Inform AKEL Improvements:**
1227
1228 1. **Identify Weak Areas:**
1229
1230 * Low TIGERScore → Review prompt engineering
1231 * High hallucination → Strengthen evidence grounding
1232 * Low IRR → Clarify evaluation criteria
1233
1234 2. **A/B Testing Integration:**
1235
1236 * Test prompt variations
1237 * Measure impact on quality metrics
1238 * Deploy winners automatically
1239
1240 3. **Alert Thresholds:**
1241
1242 * TIGERScore drops below 75 → Alert team
1243 * Hallucination rate exceeds 7% → Pause auto-publishing
1244 * IRR below 0.6 → Moderator training needed
1245
1246 4. **Monthly Quality Reviews:**
1247
1248 * Analyze trends
1249 * Identify systematic issues
1250 * Plan prompt improvements
1251 * Update AKEL models
1252
1253 -----
1254
1255 ==== Metric Calculation Details ====
1256
1257 **TIGERScore Implementation:**
1258
1259 * Reference: https://github.com/TIGER-AI-Lab/TIGERScore
1260 * Input: Generated verdict + reference verdict (from expert)
1261 * Output: 0-100 score across 5 dimensions
1262 * Requires: Test set of expert-reviewed claims (minimum 100)
1263
1264 **AlignScore Implementation:**
1265
1266 * Reference: https://github.com/yuh-zha/AlignScore
1267 * Input: Generated verdict + source evidence text
1268 * Output: 0-1 faithfulness score
1269 * Calculation: Semantic alignment between claim and evidence
1270
1271 **Source Quality Scoring:**
1272
1273 * Use existing source reliability database (e.g., NewsGuard, MBFC)
1274 * Factor in: Publication history, corrections record, transparency
1275 * Scale: 0-1 (weighted average across sources)
1276
1277 -----
1278
1279 ==== Integration Points ====
1280
1281 * **NFR11: AKEL Quality Assurance** - Metrics validate quality gate effectiveness
1282 * **FR49: A/B Testing** - Metrics measure test success
1283 * **FR11: Audit Trail** - Source of quality data
1284 * **NFR3: Transparency** - Public metrics build trust
1285
1286 **Acceptance Criteria:**
1287
1288 * ✅ All core metrics implemented and calculating correctly
1289 * ✅ Dashboard updates daily (Beta 0) or hourly (V1.0)
1290 * ✅ Alerts trigger when metrics degrade beyond thresholds
1291 * ✅ Monthly quality report auto-generates
1292 * ✅ Dashboard is publicly accessible (no login required)
1293 * ✅ Mobile-responsive dashboard design
1294 * ✅ Metrics inform quarterly AKEL improvement planning
1295
1296
1297
1298
1299 == 13. Requirements Traceability ==
1300
1301 For full traceability matrix showing which requirements fulfill which user needs, see:
1302
1303 * [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] - Section 8 includes comprehensive mapping tables
1304
1305 == 14. Related Pages ==
1306
1307 **Non-Functional Requirements (see Section 9):**
1308
1309 * [[NFR11 — AKEL Quality Assurance Framework>>#NFR11]]
1310 * [[NFR12 — Security Controls>>#NFR12]]
1311 * [[NFR13 — Quality Metrics Transparency>>#NFR13]]
1312
1313 **Other Requirements:**
1314
1315 * [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]
1316 * [[V1.0 Requirements>>FactHarbor.Specification.Requirements.V10.]]
1317 * [[Gap Analysis>>FactHarbor.Specification.Requirements.GapAnalysis]]
1318
1319 * **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need (drives these requirements)
1320 * [[Architecture>>FactHarbor.Specification.Architecture.WebHome]] - How requirements are implemented
1321 * [[Data Model>>FactHarbor.Specification.Data Model.WebHome]] - Data structures supporting requirements
1322 * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] - User interaction workflows
1323 * [[AKEL>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements
1324 * [[Global Rules>>FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]]
1325 * [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
1326
1327 = V0.9.70 Additional Requirements =
1328
1329 == Functional Requirements (Additional) ==
1330
1331 === FR44: ClaimReview Schema Implementation ===
1332
1333 **Fulfills:** UN-13 (Cite FactHarbor Verdicts), UN-14 (API Access for Integration), UN-26 (Search Engine Visibility)
1334
1335
1336 **Purpose:** Generate valid ClaimReview structured data for every published analysis to enable Google/Bing search visibility and fact-check discovery.
1337
1338 **Specification:**
1339
1340 ==== Component: Schema.org Markup Generator ====
1341
1342 FactHarbor must generate valid ClaimReview structured data following Schema.org specifications for every published claim analysis.
1343
1344 **Required JSON-LD Schema:**
1345
1346 {{code language="json"}}
1347 {
1348 "@context": "https://schema.org",
1349 "@type": "ClaimReview",
1350 "datePublished": "YYYY-MM-DD",
1351 "url": "https://factharbor.org/claims/{claim_id}",
1352 "claimReviewed": "The exact claim text",
1353 "author": {
1354 "@type": "Organization",
1355 "name": "FactHarbor",
1356 "url": "https://factharbor.org"
1357 },
1358 "reviewRating": {
1359 "@type": "Rating",
1360 "ratingValue": "1-5",
1361 "bestRating": "5",
1362 "worstRating": "1",
1363 "alternateName": "FactHarbor likelihood score"
1364 },
1365 "itemReviewed": {
1366 "@type": "Claim",
1367 "author": {
1368 "@type": "Person",
1369 "name": "Claim author if known"
1370 },
1371 "datePublished": "YYYY-MM-DD if known",
1372 "appearance": {
1373 "@type": "CreativeWork",
1374 "url": "Original claim URL if from article"
1375 }
1376 }
1377 }
1378 {{/code}}
1379
1380 **FactHarbor-Specific Mapping:**
1381
1382 **Likelihood Score to Rating Scale:**
1383
1384 * 80-100% likelihood → 5 (Highly Supported)
1385 * 60-79% likelihood → 4 (Supported)
1386 * 40-59% likelihood → 3 (Mixed/Uncertain)
1387 * 20-39% likelihood → 2 (Questionable)
1388 * 0-19% likelihood → 1 (Refuted)
1389
1390 **Multiple Scenarios Handling:**
1391
1392 * If claim has multiple scenarios with different verdicts, generate **separate ClaimReview** for each scenario
1393 * Add `disambiguatingDescription` field explaining scenario context
1394 * Example: "Scenario: If interpreted as referring to 2023 data..."
1395
1396 ==== Implementation Requirements ====
1397
1398 1. **Auto-generate** on claim publication
1399 2. **Embed** in HTML `<head>` section as JSON-LD script
1400 3. **Validate** against Schema.org validator before publishing
1401 4. **Submit** to Google Search Console for indexing
1402 5. **Update** automatically when verdict changes (integrate with FR8: Time Evolution)
1403
1404 ==== Integration Points ====
1405
1406 * **FR7: Automated Verdicts** - Source of rating data and claim text
1407 * **FR8: Time Evolution** - Triggers schema updates when verdicts change
1408 * **FR11: Audit Trail** - Logs all schema generation and update events
1409
1410 ==== Resources ====
1411
1412 * ClaimReview Project: https://www.claimreviewproject.com
1413 * Schema.org ClaimReview: https://schema.org/ClaimReview
1414 * Google Fact Check Guidelines: https://developers.google.com/search/docs/appearance/fact-check
1415
1416 **Acceptance Criteria:**
1417
1418 * ✅ Passes Google Structured Data Testing Tool
1419 * ✅ Appears in Google Fact Check Explorer within 48 hours of publication
1420 * ✅ Valid JSON-LD syntax (no errors)
1421 * ✅ All required fields populated with correct data types
1422 * ✅ Handles multi-scenario claims correctly (separate ClaimReview per scenario)
1423
1424 === FR45: User Corrections Notification System ===
1425
1426 **Fulfills:** IFCN Principle 5 (Open & Honest Corrections), EFCSN compliance
1427
1428
1429 **Purpose:** When any claim analysis is corrected, notify users who previously viewed the claim to maintain transparency and build trust.
1430
1431 **Specification:**
1432
1433 ==== Component: Corrections Visibility Framework ====
1434
1435 **Correction Types:**
1436
1437 1. **Major Correction:** Verdict changes category (e.g., "Supported" → "Refuted")
1438 2. **Significant Correction:** Likelihood score changes >20%
1439 3. **Minor Correction:** Evidence additions, source quality updates
1440 4. **Scenario Addition:** New scenario added to existing claim
1441
1442 ==== Notification Mechanisms ====
1443
1444 *
1445 **
1446 **1. In-Page Banner:
1447
1448 Display prominent banner on claim page:
1449
1450 {{code}}
1451 [!] CORRECTION NOTICE
1452 This analysis was updated on [DATE]. [View what changed] [Dismiss]
1453
1454 Major changes:
1455 • Verdict changed from "Likely True (75%)" to "Uncertain (45%)"
1456 • New contradicting evidence added from [Source]
1457 • Scenario 2 updated with additional context
1458
1459 [See full correction log]
1460 {{/code}}
1461
1462 **2. Correction Log Page:**
1463
1464 * Public changelog at `/claims/{id}/corrections`
1465 * Displays for each correction:
1466 * Date/time of correction
1467 * What changed (before/after comparison)
1468 * Why changed (reason if provided)
1469 * Who made change (AKEL auto-update vs. contributor override)
1470
1471 **3. Email Notifications (opt-in):**
1472
1473 * Send to users who bookmarked or shared the claim
1474 * Subject: "FactHarbor Correction: [Claim title]"
1475 * Include summary of changes
1476 * Link to updated analysis
1477
1478 **4. RSS/API Feed:**
1479
1480 * Corrections feed at `/corrections.rss`
1481 * API endpoint: `GET /api/corrections?since={timestamp}`
1482 * Enables external monitoring by journalists and researchers
1483
1484 ==== Display Rules ====
1485
1486 * Show banner on **ALL pages** displaying the claim (search results, related claims, embeddings)
1487 * Banner persists for **30 days** after correction
1488 * **"Corrections" count badge** on claim card
1489 * **Timestamp** on every verdict: "Last updated: [datetime]"
1490
1491 ==== IFCN Compliance Requirements ====
1492
1493 * Corrections policy published at `/corrections-policy`
1494 * User can report suspected errors via `/report-error/{claim_id}`
1495 * Link to IFCN complaint process (if FactHarbor becomes signatory)
1496 * **Scrupulous transparency:** Never silently edit analyses
1497
1498 ==== Integration Points ====
1499
1500 * **FR8: Time Evolution** - Triggers corrections when verdicts change
1501 * **FR11: Audit Trail** - Source of correction data and change history
1502 * **NFR3: Transparency** - Public correction log demonstrates commitment
1503
1504 **Acceptance Criteria:**
1505
1506 * ✅ Banner appears within 60 seconds of correction
1507 * ✅ Correction log is permanent and publicly accessible
1508 * ✅ Email notifications deliver within 5 minutes
1509 * ✅ RSS feed updates in real-time
1510 * ✅ Mobile-responsive banner design
1511 * ✅ Accessible (screen reader compatible)
1512
1513 === FR46: Image Verification System ===
1514
1515 **Fulfills:** UN-27 (Visual Claim Verification)
1516
1517
1518 **Purpose:** Verify authenticity and context of images shared with claims to detect manipulation, misattribution, and out-of-context usage.
1519
1520 **Specification:**
1521
1522 ==== Component: Multi-Method Image Verification ====
1523
1524 **Method 1: Reverse Image Search**
1525
1526 **Purpose:** Find earlier uses of the image to verify context
1527
1528 **Implementation:**
1529
1530 * Integrate APIs:
1531 * **Google Vision AI** (reverse search)
1532 * **TinEye** (oldest known uses)
1533 * **Bing Visual Search** (broad coverage)
1534
1535 **Process:**
1536
1537 1. Extract image from claim or user upload
1538 2. Query multiple reverse search services
1539 3. Analyze results for:
1540
1541 * Earliest known publication
1542 * Original context (what was it really showing?)
1543 * Publication timeline
1544 * Geographic spread
1545
1546 **Output:**
1547 {{code}}Reverse Image Search Results:
1548
1549 Earliest known use: 2019-03-15 (5 years before claim)
1550 Original context: "Photo from 2019 flooding in Mumbai"
1551 This claim uses it for: "2024 hurricane damage in Florida"
1552
1553 ⚠️ Image is OUT OF CONTEXT
1554
1555 Found in 47 other articles:
1556 • 2019-03-15: Mumbai floods (original)
1557 • 2020-07-22: Bangladesh monsoon
1558 • 2024-10-15: Current claim (misattributed)
1559
1560 [View full timeline]{{/code}}
1561
1562 -----
1563
1564 **Method 2: AI Manipulation Detection**
1565
1566 **Purpose:** Detect deepfakes, face swaps, and digital alterations
1567
1568 **Implementation:**
1569
1570 * Integrate detection services:
1571 * **Sensity AI** (deepfake detection)
1572 * **Reality Defender** (multimodal analysis)
1573 * **AWS Rekognition** (face detection inconsistencies)
1574
1575 **Detection Categories:**
1576
1577 1. **Face Manipulation:**
1578
1579 * Deepfake face swaps
1580 * Expression manipulation
1581 * Identity replacement
1582
1583 2. **Image Manipulation:**
1584
1585 * Copy-paste artifacts
1586 * Clone stamp detection
1587 * Content-aware fill detection
1588 * JPEG compression inconsistencies
1589
1590 3. **AI Generation:**
1591
1592 * Detect fully AI-generated images
1593 * Identify generation artifacts
1594 * Check for model signatures
1595
1596 **Confidence Scoring:**
1597
1598 * **HIGH (80-100%):** Strong evidence of manipulation
1599 * **MEDIUM (50-79%):** Suspicious artifacts detected
1600 * **LOW (0-49%):** Minor inconsistencies or inconclusive
1601
1602 **Output:**
1603 {{code}}Manipulation Analysis:
1604
1605 Face Manipulation: LOW RISK (12%)
1606 Image Editing: MEDIUM RISK (64%)
1607 • Clone stamp artifacts detected in sky region
1608 • JPEG compression inconsistent between objects
1609
1610 AI Generation: LOW RISK (8%)
1611
1612 ⚠️ Possible manipulation detected. Manual review recommended.{{/code}}
1613
1614 -----
1615
1616 **Method 3: Metadata Analysis (EXIF)**
1617
1618 **Purpose:** Extract technical details that may reveal manipulation or misattribution
1619
1620 **Extracted Data:**
1621
1622 * **Camera/Device:** Make, model, software
1623 * **Timestamps:** Original date, modification dates
1624 * **Location:** GPS coordinates (if present)
1625 * **Editing History:** Software used, edit count
1626 * **File Properties:** Resolution, compression, format conversions
1627
1628 **Red Flags:**
1629
1630 * Metadata completely stripped (suspicious)
1631 * Timestamp conflicts with claimed date
1632 * GPS location conflicts with claimed location
1633 * Multiple edit rounds (hiding something?)
1634 * Creation date after modification date (impossible)
1635
1636 **Output:**
1637 {{code}}Image Metadata:
1638
1639 Camera: iPhone 14 Pro
1640 Original date: 2023-08-12 14:32:15
1641 Location: 40.7128°N, 74.0060°W (New York City)
1642 Modified: 2024-10-15 08:45:22
1643 Software: Adobe Photoshop 2024
1644
1645 ⚠️ Location conflicts with claim
1646 Claim says: "Taken in Los Angeles"
1647 EXIF says: New York City
1648
1649 ⚠️ Edited 14 months after capture{{/code}}
1650
1651 -----
1652
1653 ==== Verification Workflow ====
1654
1655 **Automatic Triggers:**
1656
1657 1. User submits claim with image
1658 2. Article being analyzed contains images
1659 3. Social media post includes photos
1660
1661 **Process:**
1662
1663 1. Extract images from content
1664 2. Run all 3 verification methods in parallel
1665 3. Aggregate results into confidence score
1666 4. Generate human-readable summary
1667 5. Display prominently in analysis
1668
1669 **Display Integration:**
1670
1671 Show image verification panel in claim analysis:
1672
1673 {{code}}
1674 📷 IMAGE VERIFICATION
1675
1676 [Image thumbnail]
1677
1678 ✅ Reverse Search: Original context verified
1679 ⚠️ Manipulation: Possible editing detected (64% confidence)
1680 ✅ Metadata: Consistent with claim details
1681
1682 Overall Assessment: CAUTION ADVISED
1683 This image may have been edited. Original context appears accurate.
1684
1685 [View detailed analysis]
1686 {{/code}}
1687
1688 ==== Integration Points ====
1689
1690 * **FR7: Automated Verdicts** - Image verification affects claim credibility
1691 * **FR4: Analysis Summary** - Image findings included in summary
1692 * **UN-27: Visual Claim Verification** - Direct fulfillment
1693
1694 ==== Cost Considerations ====
1695
1696 **API Costs (estimated per image):**
1697
1698 * Google Vision AI: $0.001-0.003
1699 * TinEye: $0.02 (commercial API)
1700 * Sensity AI: $0.05-0.10
1701 * AWS Rekognition: $0.001-0.002
1702
1703 **Total per image:** $0.07-0.15**
1704
1705 **Mitigation Strategies:**
1706
1707 * Cache results for duplicate images
1708 * Use free tier quotas where available
1709 * Prioritize higher-value claims for deep analysis
1710 * Offer premium verification as paid tier
1711
1712 **Acceptance Criteria:**
1713
1714 * ✅ Reverse image search finds original sources
1715 * ✅ Manipulation detection accuracy >80% on test dataset
1716 * ✅ EXIF extraction works for major image formats (JPEG, PNG, HEIC)
1717 * ✅ Results display within 10 seconds
1718 * ✅ Mobile-friendly image comparison interface
1719 * ✅ False positive rate <15%
1720
1721 === FR47: Archive.org Integration ===
1722
1723 **Importance:** CRITICAL
1724 **Fulfills:** Evidence persistence, FR5 (Evidence linking)
1725
1726 **Purpose:** Ensure evidence remains accessible even if original sources are deleted.
1727
1728 **Specification:**
1729
1730 **Automatic Archiving:**
1731
1732 When AKEL links evidence:
1733
1734 1. Check if URL already archived (Wayback Machine API)
1735 2. If not, submit for archiving (Save Page Now API)
1736 3. Store both original URL and archive URL
1737 4. Display both to users
1738
1739 **Archive Display:**
1740
1741 {{code}}
1742 Evidence Source: [Original URL]
1743 Archived: [Archive.org URL] (Captured: [date])
1744
1745 [View Original] [View Archive]
1746 {{/code}}
1747
1748 **Fallback Logic:**
1749
1750 * If original URL unavailable → Auto-redirect to archive
1751 * If archive unavailable → Display warning
1752 * If both unavailable → Flag for manual review
1753
1754 **API Integration:**
1755
1756 * Use Wayback Machine Availability API
1757 * Use Save Page Now API (SPNv2)
1758 * Rate limiting: 15 requests/minute (Wayback limit)
1759
1760 **Acceptance Criteria:**
1761
1762 * ✅ All evidence URLs auto-archived
1763 * ✅ Archive links displayed to users
1764 * ✅ Fallback to archive if original unavailable
1765 * ✅ API rate limits respected
1766 * ✅ Archive status visible in evidence display
1767
1768 == Category 4: Community Safety ==
1769
1770 FR48: Contributor Safety Framework ===
1771
1772 **Importance:** CRITICAL
1773 **Fulfills:** UN-28 (Safe contribution environment)
1774
1775 **Purpose:** Protect contributors from harassment, doxxing, and coordinated attacks.
1776
1777 **Specification:**
1778
1779 *
1780 **
1781 **1. Privacy Protection:
1782
1783 * **Optional Pseudonymity:** Contributors can use pseudonyms
1784 * **Email Privacy:** Emails never displayed publicly
1785 * **Profile Privacy:** Contributors control what's public
1786 * **IP Logging:** Only for abuse prevention, not public
1787
1788 **2. Harassment Prevention:**
1789
1790 * **Automated Toxicity Detection:** Flag abusive comments
1791 * **Personal Information Detection:** Auto-block doxxing attempts
1792 * **Coordinated Attack Detection:** Identify brigading patterns
1793 * **Rapid Response:** Moderator alerts for harassment
1794
1795 **3. Safety Features:**
1796
1797 * **Block Users:** Contributors can block harassers
1798 * **Private Contributions:** Option to contribute anonymously
1799 * **Report Harassment:** One-click harassment reporting
1800 * **Safety Resources:** Links to support resources
1801
1802 **4. Moderator Tools:**
1803
1804 * **Quick Ban:** Immediately block abusers
1805 * **Pattern Detection:** Identify coordinated attacks
1806 * **Appeal Process:** Fair review of moderation actions
1807 * **Escalation:** Serious threats escalated to authorities
1808
1809 **5. Trusted Contributor Protection:**
1810
1811 * **Enhanced Privacy:** Additional protection for high-profile contributors
1812 * **Verification:** Optional identity verification (not public)
1813 * **Legal Support:** Resources for contributors facing legal threats
1814
1815 **Acceptance Criteria:**
1816
1817 * ✅ Pseudonyms supported
1818 * ✅ Toxicity detection active
1819 * ✅ Doxxing auto-blocked
1820 * ✅ Harassment reporting functional
1821 * ✅ Moderator tools implemented
1822 * ✅ Safety policy published
1823
1824 == Category 5: Continuous Improvement ==
1825
1826 FR49: A/B Testing Framework ===
1827
1828 **Importance:** CRITICAL
1829 **Fulfills:** Continuous system improvement
1830
1831 **Purpose:** Test and measure improvements to AKEL prompts, algorithms, and workflows.
1832
1833 **Specification:**
1834
1835 **Test Capabilities:**
1836
1837 1. **Prompt Variations:**
1838
1839 * Test different claim extraction prompts
1840 * Test different verdict generation prompts
1841 * Measure: Accuracy, clarity, completeness
1842
1843 2. **Algorithm Variations:**
1844
1845 * Test different source scoring algorithms
1846 * Test different confidence calculations
1847 * Measure: Audit accuracy, user satisfaction
1848
1849 3. **Workflow Variations:**
1850
1851 * Test different quality gate thresholds
1852 * Test different risk tier assignments
1853 * Measure: Publication rate, quality scores
1854
1855 **Implementation:**
1856
1857 * **Traffic Split:** 50/50 or 90/10 splits
1858 * **Randomization:** Consistent per claim (not per user)
1859 * **Metrics Collection:** Automatic for all variants
1860 * **Statistical Significance:** Minimum sample size calculation
1861 * **Rollout:** Winner promoted to 100% traffic
1862
1863 **A/B Test Workflow:**
1864
1865 {{code}}
1866 1. Hypothesis: "New prompt improves claim extraction"
1867 2. Design test: Control vs. Variant
1868 3. Define metrics: Extraction accuracy, completeness
1869 4. Run test: 7-14 days, minimum 100 claims each
1870 5. Analyze results: Statistical significance?
1871 6. Decision: Deploy winner or iterate
1872 {{/code}}
1873
1874 **Acceptance Criteria:**
1875
1876 * ✅ A/B testing framework implemented
1877 * ✅ Can test prompt variations
1878 * ✅ Can test algorithm variations
1879 * ✅ Metrics automatically collected
1880 * ✅ Statistical significance calculated
1881 * ✅ Results inform system improvements
1882
1883 === FR54: Evidence Deduplication ===
1884
1885 **Importance:** CRITICAL (POC2/Beta)
1886 **Fulfills:** Accurate evidence counting, quality metrics
1887
1888 **Purpose:** Avoid counting the same source multiple times when it appears in different forms.
1889
1890 **Specification:**
1891
1892 **Deduplication Logic:**
1893
1894 1. **URL Normalization:**
1895
1896 * Remove tracking parameters (?utm_source=...)
1897 * Normalize http/https
1898 * Normalize www/non-www
1899 * Handle redirects
1900
1901 2. **Content Similarity:**
1902
1903 * If two sources have >90% text similarity → Same source
1904 * If one is subset of other → Same source
1905 * Use fuzzy matching for minor differences
1906
1907 3. **Cross-Domain Syndication:**
1908
1909 * Detect wire service content (AP, Reuters)
1910 * Mark as single source if syndicated
1911 * Count original publication only
1912
1913 **Display:**
1914
1915 {{code}}
1916 Evidence Sources (3 unique, 5 total):
1917
1918 1. Original Article (NYTimes)
1919 - Also appeared in: WashPost, Guardian (syndicated)
1920
1921 2. Research Paper (Nature)
1922
1923 3. Official Statement (WHO)
1924 {{/code}}
1925
1926 **Acceptance Criteria:**
1927
1928 * ✅ URL normalization works
1929 * ✅ Content similarity detected
1930 * ✅ Syndicated content identified
1931 * ✅ Unique vs. total counts accurate
1932 * ✅ Improves evidence quality metrics
1933
1934 == Additional Requirements (Lower Importance) ==
1935
1936 FR50: OSINT Toolkit Integration ===
1937
1938
1939
1940 **Fulfills:** Advanced media verification
1941
1942 **Purpose:** Integrate open-source intelligence tools for advanced verification.
1943
1944 **Tools to Integrate:**
1945
1946 * InVID/WeVerify (video verification)
1947 * Bellingcat toolkit
1948 * Additional TBD based on V1.0 learnings
1949
1950 === FR51: Video Verification System ===
1951
1952
1953
1954 **Fulfills:** UN-27 (Visual claims), advanced media verification
1955
1956 **Purpose:** Verify video-based claims.
1957
1958 **Specification:**
1959
1960 * Keyframe extraction
1961 * Reverse video search
1962 * Deepfake detection (AI-powered)
1963 * Metadata analysis
1964 * Acoustic signature analysis
1965
1966 === FR52: Interactive Detection Training ===
1967
1968
1969
1970 **Fulfills:** Media literacy education
1971
1972 **Purpose:** Teach users to identify misinformation.
1973
1974 **Specification:**
1975
1976 * Interactive tutorials
1977 * Practice exercises
1978 * Detection quizzes
1979 * Gamification elements
1980
1981 === FR53: Cross-Organizational Sharing ===
1982
1983
1984
1985 **Fulfills:** Collaboration with other fact-checkers
1986
1987 **Purpose:** Share findings with IFCN/EFCSN members.
1988
1989 **Specification:**
1990
1991 * API for fact-checking organizations
1992 * Structured data exchange
1993 * Privacy controls
1994 * Attribution requirements
1995
1996 == Summary ==
1997
1998 **V1.0 Critical Requirements (Must Have):**
1999
2000 * FR44: ClaimReview Schema ✅
2001 * FR45: Corrections Notification ✅
2002 * FR46: Image Verification ✅
2003 * FR47: Archive.org Integration ✅
2004 * FR48: Contributor Safety ✅
2005 * FR49: A/B Testing ✅
2006 * FR54: Evidence Deduplication ✅
2007 * NFR11: Quality Assurance Framework ✅
2008 * NFR12: Security Controls ✅
2009 * NFR13: Quality Metrics Dashboard ✅
2010
2011 **V1.1+ (Future):**
2012
2013 * FR50: OSINT Integration
2014 * FR51: Video Verification
2015 * FR52: Detection Training
2016 * FR53: Cross-Org Sharing
2017
2018 **Total:** 11 critical requirements for V1.0
2019
2020 === FR54: Evidence Deduplication ===
2021
2022
2023
2024 **Fulfills:** Accurate evidence counting, quality metrics
2025
2026 **Purpose:** Avoid counting the same source multiple times when it appears in different forms.
2027
2028 **Specification:**
2029
2030 **Deduplication Logic:**
2031
2032 1. **URL Normalization:**
2033
2034 * Remove tracking parameters (?utm_source=...)
2035 * Normalize http/https
2036 * Normalize www/non-www
2037 * Handle redirects
2038
2039 2. **Content Similarity:**
2040
2041 * If two sources have >90% text similarity → Same source
2042 * If one is subset of other → Same source
2043 * Use fuzzy matching for minor differences
2044
2045 3. **Cross-Domain Syndication:**
2046
2047 * Detect wire service content (AP, Reuters)
2048 * Mark as single source if syndicated
2049 * Count original publication only
2050
2051 **Display:**
2052
2053 {{code}}
2054 Evidence Sources (3 unique, 5 total):
2055
2056 1. Original Article (NYTimes)
2057 - Also appeared in: WashPost, Guardian (syndicated)
2058
2059 2. Research Paper (Nature)
2060
2061 3. Official Statement (WHO)
2062 {{/code}}
2063
2064 **Acceptance Criteria:**
2065
2066 * ✅ URL normalization works
2067 * ✅ Content similarity detected
2068 * ✅ Syndicated content identified
2069 * ✅ Unique vs. total counts accurate
2070 * ✅ Improves evidence quality metrics
2071
2072 == Additional Requirements (Lower Importance) ==
2073
2074 FR7: Automated Verdicts (Enhanced with Quality Gates) ===
2075
2076 **POC1+ Enhancement:**
2077
2078 After AKEL generates verdict, it passes through quality gates:
2079
2080 {{code}}
2081 Workflow:
2082 1. Extract claims
2083 ↓
2084 2. [GATE 1] Validate fact-checkable
2085 ↓
2086 3. Generate scenarios
2087 ↓
2088 4. Generate verdicts
2089 ↓
2090 5. [GATE 4] Validate confidence
2091 ↓
2092 6. Display to user
2093 {{/code}}
2094
2095 **Updated Verdict States:**
2096
2097 * PUBLISHED
2098 * INSUFFICIENT_EVIDENCE
2099 * NON_FACTUAL_CLAIM
2100 * PROCESSING
2101 * ERROR
2102
2103 === FR4: Analysis Summary (Enhanced with Quality Metadata) ===
2104
2105 **POC1+ Enhancement:**
2106
2107 Display quality indicators:
2108
2109 {{code}}
2110 Analysis Summary:
2111 Verifiable Claims: 3/5
2112 High Confidence Verdicts: 1
2113 Medium Confidence: 2
2114 Evidence Sources: 12
2115 Avg Source Quality: 0.73
2116 Quality Score: 8.5/10
2117 {{/code}}