Skip to Content

Wiki source code of Requirements

Last modified by Robert Schaub on 2025/12/23 16:50
Show last authors
1 = Requirements =
2
3
4 {{info}}
5 **Phase Assignments:** See [[Requirements Roadmap Matrix>>Test.FactHarbor V0\.9\.84.Roadmap.Requirements-Roadmap-Matrix.WebHome]] for which requirements are implemented in which phases.
6 {{/info}}
7
8 **This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.**
9
10 **Core Philosophy:** Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically.
11
12 == Navigation ==
13
14 * **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need from FactHarbor (drives these requirements)
15 * **This page** - How we fulfill those needs through system design
16
17 (% class="box infomessage" %)
18 (((
19 **How to read this page:**
20
21 1. **User Needs drive Requirements**: See [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] for what users need
22 2. **Requirements define implementation**: This page shows how we fulfill those needs
23 3. **Functional Requirements (FR)**: Specific features and capabilities
24 4. **Non-Functional Requirements (NFR)**: Quality attributes (performance, security, etc.)
25
26 Each requirement references which User Needs it fulfills.
27 )))
28
29 == 1. Roles ==
30
31 **Fulfills**: UN-12 (Submit claims), UN-13 (Cite verdicts), UN-14 (API access)
32
33 FactHarbor uses three simple roles plus a reputation system.
34
35 === 1.1 Reader ===
36
37 **Who**: Anyone (no login required)
38
39 **Can**:
40
41 * Browse and search claims
42 * View scenarios, evidence, verdicts, and confidence scores
43 * Flag issues or errors
44 * Use filters, search, and visualization tools
45 * Submit claims automatically (new claims added if not duplicates)
46
47 **Cannot**:
48
49 * Modify content
50 * Access edit history details
51
52 **User Needs served**: UN-1 (Trust assessment), UN-2 (Claim verification), UN-3 (Article summary with FactHarbor analysis summary), UN-4 (Social media fact-checking), UN-5 (Source tracing), UN-7 (Evidence transparency), UN-8 (Understanding disagreement), UN-12 (Submit claims), UN-17 (In-article highlighting)
53
54 === 1.2 Contributor ===
55
56 **Who**: Registered users (earns reputation through contributions)
57
58 **Can**:
59
60 * Everything a Reader can do
61 * Edit claims, evidence, and scenarios
62 * Add sources and citations
63 * Suggest improvements to AI-generated content
64 * Participate in discussions
65 * Earn reputation points for quality contributions
66
67 **Reputation System**:
68
69 * New contributors: Limited edit privileges
70 * Established contributors (established reputation): Full edit access
71 * Trusted contributors (substantial reputation): Can approve certain changes
72 * Reputation earned through: Accepted edits, helpful flags, quality contributions
73 * Reputation lost through: Reverted edits, invalid flags, abuse
74
75 **Cannot**:
76
77 * Delete or hide content (only moderators)
78 * Override moderation decisions
79
80 **User Needs served**: UN-13 (Cite and contribute)
81
82 === 1.3 Moderator ===
83
84 **Who**: Trusted community members with proven track record, appointed by governance board
85
86 **Can**:
87
88 * Review flagged content
89 * Hide harmful or abusive content
90 * Resolve disputes between contributors
91 * Issue warnings or temporary bans
92 * Make final decisions on content disputes
93 * Access full audit logs
94
95 **Cannot**:
96
97 * Change governance rules
98 * Permanently ban users without board approval
99 * Override technical quality gates
100
101 **Note**: Small team (3-5 initially), supported by automated moderation tools.
102
103 === 1.4 Domain Trusted Contributors (Optional, Task-Specific) ===
104
105 **Who**: Subject matter specialists invited for specific high-stakes disputes
106
107 **Not a permanent role**: Contacted externally when needed for contested claims in their domain
108
109 **When used**:
110
111 * Medical claims with life/safety implications
112 * Legal interpretations with significant impact
113 * Scientific claims with high controversy
114 * Technical claims requiring specialized knowledge
115
116 **Process**:
117
118 * Moderator identifies need for expert input
119 * Contact expert externally (don't require them to be users)
120 * Trusted Contributor provides written opinion with sources
121 * Opinion added to claim record
122 * Trusted Contributor acknowledged in claim
123
124 **User Needs served**: UN-16 (Expert validation status)
125
126 == 2. Content States ==
127
128 **Fulfills**: UN-1 (Trust indicators), UN-16 (Review status transparency)
129
130 FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping.
131
132 === 2.1 Published ===
133
134 **Status**: Visible to all users
135
136 **Includes**:
137
138 * AI-generated analyses (default state)
139 * User-contributed content
140 * Edited/improved content
141
142 **Quality Indicators** (displayed with content):
143
144 * **Confidence Score**: 0-100% (AI's confidence in analysis)
145 * **Source Quality Score**: 0-100% (based on source track record)
146 * **Controversy Flag**: If high dispute/edit activity
147 * **Completeness Score**: % of expected fields filled
148 * **Last Updated**: Date of most recent change
149 * **Edit Count**: Number of revisions
150 * **Review Status**: AI-generated / Human-reviewed / Expert-validated
151
152 **Automatic Warnings**:
153
154 * Confidence < 60%: "Low confidence - use caution"
155 * Source quality < 40%: "Sources may be unreliable"
156 * High controversy: "Disputed - multiple interpretations exist"
157 * Medical/Legal/Safety domain: "Seek professional advice"
158
159 **User Needs served**: UN-1 (Trust score), UN-9 (Methodology transparency), ~~UN-15 (Evolution timeline - Deferred)~~, UN-16 (Review status)
160
161 === 2.2 Hidden ===
162
163 **Status**: Not visible to regular users (only to moderators)
164
165 **Reasons**:
166
167 * Spam or advertising
168 * Personal attacks or harassment
169 * Illegal content
170 * Privacy violations
171 * Deliberate misinformation (verified)
172 * Abuse or harmful content
173
174 **Process**:
175
176 * Automated detection flags for moderator review
177 * Moderator confirms and hides
178 * Original author notified with reason
179 * Can appeal to board if disputes moderator decision
180
181 **Note**: Content is hidden, not deleted (for audit trail)
182
183 == 3. Contribution Rules ==
184
185 === 3.1 All Contributors Must ===
186
187 * Provide sources for factual claims
188 * Use clear, neutral language in FactHarbor's own summaries
189 * Respect others and maintain civil discourse
190 * Accept community feedback constructively
191 * Focus on improving quality, not protecting ego
192
193 === 3.2 AKEL (AI System) ===
194
195 **AKEL is the primary system**. Human contributions supplement and train AKEL.
196
197 **AKEL Must**:
198
199 * Mark all outputs as AI-generated
200 * Display confidence scores prominently
201 * Provide source citations
202 * Flag uncertainty clearly
203 * Identify contradictions in evidence
204 * Learn from human corrections
205
206 **When AKEL Makes Errors**:
207
208 1. Capture the error pattern (what, why, how common)
209 2. Improve the system (better prompt, model, validation)
210 3. Re-process affected claims automatically
211 4. Measure improvement (did quality increase?)
212
213 **Human Role**: Train AKEL through corrections, not replace AKEL
214
215 === 3.3 Contributors Should ===
216
217 * Improve clarity and structure
218 * Add missing sources
219 * Flag errors for system improvement
220 * Suggest better ways to present information
221 * Participate in quality discussions
222
223 === 3.4 Moderators Must ===
224
225 * Be impartial
226 * Document moderation decisions
227 * Respond to appeals promptly
228 * Use automated tools to scale efforts
229 * Focus on abuse/harm, not routine quality control
230
231 == 4. Quality Standards ==
232
233 **Fulfills**: UN-5 (Source reliability), UN-6 (Publisher track records), UN-7 (Evidence transparency), UN-9 (Methodology transparency)
234
235 === 4.1 Source Requirements ===
236
237 **Track Record Over Credentials**:
238
239 * Sources evaluated by historical accuracy
240 * Correction policy matters
241 * Independence from conflicts of interest
242 * Methodology transparency
243
244 **Source Quality Database**:
245
246 * Automated tracking of source accuracy
247 * Correction frequency
248 * Reliability score (updated continuously)
249 * Users can see source track record
250
251 **No automatic trust** for government, academia, or media - all evaluated by track record.
252
253 **User Needs served**: UN-5 (Source provenance), UN-6 (Publisher reliability)
254
255 === 4.2 Claim Requirements ===
256
257 * Clear subject and assertion
258 * Verifiable with available information
259 * Sourced (or explicitly marked as needing sources)
260 * Neutral language in FactHarbor summaries
261 * Appropriate context provided
262
263 **User Needs served**: UN-2 (Claim extraction and verification)
264
265 === 4.3 Evidence Requirements ===
266
267 * Publicly accessible (or explain why not)
268 * Properly cited with attribution
269 * Relevant to claim being evaluated
270 * Original source preferred over secondary
271
272 **User Needs served**: UN-7 (Evidence transparency)
273
274 === 4.4 Confidence Scoring ===
275
276 **Automated confidence calculation based on**:
277
278 * Source quality scores
279 * Evidence consistency
280 * Contradiction detection
281 * Completeness of analysis
282 * Historical accuracy of similar claims
283
284 **Thresholds**:
285
286 * < 40%: Too low to publish (needs improvement)
287 * 40-60%: Published with "Low confidence" warning
288 * 60-80%: Published as standard
289 * 80-100%: Published as "High confidence"
290
291 **User Needs served**: UN-1 (Trust assessment), UN-9 (Methodology transparency)
292
293 == 5. Automated Risk Scoring ==
294
295 **Fulfills**: UN-10 (Manipulation detection), UN-16 (Appropriate review level)
296
297 **Replace manual risk tiers with continuous automated scoring**.
298
299 === 5.1 Risk Score Calculation ===
300
301 **Factors** (weighted algorithm):
302
303 * **Domain sensitivity**: Medical, legal, safety auto-flagged higher
304 * **Potential impact**: Views, citations, spread
305 * **Controversy level**: Flags, disputes, edit wars
306 * **Uncertainty**: Low confidence, contradictory evidence
307 * **Source reliability**: Track record of sources used
308
309 **Score**: 0-100 (higher = more risk)
310
311 === 5.2 Automated Actions ===
312
313 * **Score > 80**: Flag for moderator review before publication
314 * **Score 60-80**: Publish with prominent warnings
315 * **Score 40-60**: Publish with standard warnings
316 * **Score < 40**: Publish normally
317
318 **Continuous monitoring**: Risk score recalculated as new information emerges
319
320 **User Needs served**: UN-10 (Detect manipulation tactics), UN-16 (Review status)
321
322 == 6. System Improvement Process ==
323
324 **Core principle**: Fix the system, not just the data.
325
326 === 6.1 Error Capture ===
327
328 **When users flag errors or make corrections**:
329
330 1. What was wrong? (categorize)
331 2. What should it have been?
332 3. Why did the system fail? (root cause)
333 4. How common is this pattern?
334 5. Store in ErrorPattern table (improvement queue)
335
336 === 6.2 Continuous Improvement Cycle ===
337
338 1. **Review**: Analyze top error patterns
339 2. **Develop**: Create fix (prompt, model, validation)
340 3. **Test**: Validate fix on sample claims
341 4. **Deploy**: Roll out if quality improves
342 5. **Re-process**: Automatically update affected claims
343 6. **Monitor**: Track quality metrics
344
345 === 6.3 Quality Metrics Dashboard ===
346
347 **Track continuously**:
348
349 * Error rate by category
350 * Source quality distribution
351 * Confidence score trends
352 * User flag rate (issues found)
353 * Correction acceptance rate
354 * Re-work rate
355 * Claims processed per hour
356
357 **Goal**: continuous improvement in error rate
358
359 == 7. Automated Quality Monitoring ==
360
361 **Replace manual audit sampling with automated monitoring**.
362
363 === 7.1 Continuous Metrics ===
364
365 * **Source quality**: Track record database
366 * **Consistency**: Contradiction detection
367 * **Clarity**: Readability scores
368 * **Completeness**: Field validation
369 * **Accuracy**: User corrections tracked
370
371 === 7.2 Anomaly Detection ===
372
373 **Automated alerts for**:
374
375 * Sudden quality drops
376 * Unusual patterns
377 * Contradiction clusters
378 * Source reliability changes
379 * User behavior anomalies
380
381 === 7.3 Targeted Review ===
382
383 * Review only flagged items
384 * Random sampling for calibration (not quotas)
385 * Learn from corrections to improve automation
386
387 == 8. Functional Requirements ==
388
389 This section defines specific features that fulfill user needs.
390
391 === 8.1 Claim Intake & Normalization ===
392
393 ==== FR1 — Claim Intake ====
394
395 **Fulfills**: UN-2 (Claim extraction), UN-4 (Quick fact-checking), UN-12 (Submit claims)
396
397 * Users submit claims via simple form or API
398 * Claims can be text, URL, or image
399 * Duplicate detection (semantic similarity)
400 * Auto-categorization by domain
401
402 ==== FR2 — Claim Normalization ====
403
404 **Fulfills**: UN-2 (Claim verification)
405
406 * Standardize to clear assertion format
407 * Extract key entities (who, what, when, where)
408 * Identify claim type (factual, predictive, evaluative)
409 * Link to existing similar claims
410
411 ==== FR3 — Claim Classification ====
412
413 **Fulfills**: UN-11 (Filtered research)
414
415 * Domain: Politics, Science, Health, etc.
416 * Type: Historical fact, current stat, prediction, etc.
417 * Risk score: Automated calculation
418 * Complexity: Simple, moderate, complex
419
420 === 8.2 Scenario System ===
421
422 ==== FR4 — Scenario Generation ====
423
424 **Fulfills**: UN-2 (Context-dependent verification), UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
425
426 **Automated scenario creation**:
427
428 * AKEL analyzes claim and generates likely scenarios (use-cases and contexts)
429 * Each scenario includes: assumptions, definitions, boundaries, evidence context
430 * Users can flag incorrect scenarios
431 * System learns from corrections
432
433 **Key Concept**: Scenarios represent different interpretations or contexts (e.g., "Clinical trials with healthy adults" vs. "Real-world data with diverse populations")
434
435 ==== FR5 — Evidence Linking ====
436
437 **Fulfills**: UN-5 (Source tracing), UN-7 (Evidence transparency)
438
439 * Automated evidence discovery from sources
440 * Relevance scoring
441 * Contradiction detection
442 * Source quality assessment
443
444 ==== FR6 — Scenario Comparison ====
445
446 **Fulfills**: UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
447
448 * Side-by-side comparison interface
449 * Highlight key differences between scenarios
450 * Show evidence supporting each scenario
451 * Display confidence scores per scenario
452
453 === 8.3 Verdicts & Analysis ===
454
455 ==== FR7 — Automated Verdicts ====
456
457 **Fulfills**: UN-1 (Trust score), UN-2 (Verification verdicts), UN-3 (Article summary with FactHarbor analysis summary), UN-13 (Cite verdicts)
458
459 * AKEL generates verdict based on evidence within each scenario
460 * **Likelihood range** displayed (e.g., "0.70-0.85 (likely true)") - NOT binary true/false
461 * **Uncertainty factors** explicitly listed (e.g., "Small sample sizes", "Long-term effects unknown")
462 * Confidence score displayed prominently
463 * Source quality indicators shown
464 * Contradictions noted
465 * Uncertainty acknowledged
466
467 **Key Innovation**: Detailed probabilistic verdicts with explicit uncertainty, not binary judgments
468
469 ==== FR8 — Time Evolution ====
470
471 {{warning}}
472 **Status:** Deferred (Not in V1.0)
473
474 This requirement has been **dropped from the current architecture and design**. Versioned entities have been replaced with simple edit history tracking only. Full evolution timeline functionality is deferred to future releases beyond V1.0.
475 {{/warning}}
476
477
478
479 **Fulfills**: UN-15 (Verdict evolution timeline)
480
481 * Claims and verdicts update as new evidence emerges
482 * Version history maintained for all verdicts
483 * Changes highlighted
484 * Confidence score trends visible
485 * Users can see "as of date X, what did we know?"
486
487 === 8.4 User Interface & Presentation ===
488
489 ==== FR12 — Two-Panel Summary View (Article Summary with FactHarbor Analysis Summary) ====
490
491 **Fulfills**: UN-3 (Article Summary with FactHarbor Analysis Summary)
492
493 **Purpose**: Provide side-by-side comparison of what a document claims vs. FactHarbor's complete analysis of its credibility
494
495 **Left Panel: Article Summary**:
496
497 * Document title, source, and claimed credibility
498 * "The Big Picture" - main thesis or position change
499 * "Key Findings" - structured summary of document's main claims
500 * "Reasoning" - document's explanation for positions
501 * "Conclusion" - document's bottom line
502
503 **Right Panel: FactHarbor Analysis Summary**:
504
505 * FactHarbor's independent source credibility assessment
506 * Claim-by-claim verdicts with confidence scores
507 * Methodology assessment (strengths, limitations)
508 * Overall verdict on document quality
509 * Analysis ID for reference
510
511 **Design Principles**:
512
513 * No scrolling required - both panels visible simultaneously
514 * Visual distinction between "what they say" and "FactHarbor's analysis"
515 * Color coding for verdicts (supported, uncertain, refuted)
516 * Confidence percentages clearly visible
517 * Mobile responsive (panels stack vertically on small screens)
518
519 **Implementation Notes**:
520
521 * Generated automatically by AKEL for every analyzed document
522 * Updates when verdict evolves (maintains version history)
523 * Exportable as standalone summary report
524 * Shareable via permanent URL
525
526 ==== FR13 — In-Article Claim Highlighting ====
527
528 **Fulfills**: UN-17 (In-article claim highlighting)
529
530 **Purpose**: Enable readers to quickly assess claim credibility while reading by visually highlighting factual claims with color-coded indicators
531
532 ==== Visual Example: Article with Highlighted Claims ====
533
534 (% class="box" %)
535 (((
536 **Article: "New Study Shows Benefits of Mediterranean Diet"**
537
538 A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.
539
540 (% class="box successmessage" style="margin:10px 0;" %)
541 (((
542 🟢 **Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups**
543
544 (% style="font-size:0.9em; color:#666;" %)
545 ↑ WELL SUPPORTED • 87% confidence
546 [[Click for evidence details →]]
547
548
549 )))
550
551 The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
552
553 (% class="box warningmessage" style="margin:10px 0;" %)
554 (((
555 🟡 **Some experts believe this diet can completely prevent heart attacks**
556
557 (% style="font-size:0.9em; color:#666;" %)
558 ↑ UNCERTAIN • 45% confidence
559 Overstated - evidence shows risk reduction, not prevention
560 [[Click for details →]]
561
562
563 )))
564
565 Dr. Maria Rodriguez, lead researcher, recommends incorporating more olive oil, fish, and vegetables into daily meals.
566
567 (% class="box errormessage" style="margin:10px 0;" %)
568 (((
569 🔴 **The study proves that saturated fats cause heart disease**
570
571 (% style="font-size:0.9em; color:#666;" %)
572 ↑ REFUTED • 15% confidence
573 Claim not supported by study design; correlation ≠ causation
574 [[Click for counter-evidence →]]
575
576
577 )))
578
579 Participants also reported feeling more energetic and experiencing better sleep quality, though these were secondary measures.
580 )))
581
582 **Legend:**
583
584 * 🟢 = Well-supported claim (confidence ≥75%)
585 * 🟡 = Uncertain claim (confidence 40-74%)
586 * 🔴 = Refuted/unsupported claim (confidence <40%)
587 * Plain text = Non-factual content (context, opinions, recommendations)
588
589 ==== Tooltip on Hover/Click ====
590
591 (% class="box infomessage" %)
592 (((
593 **FactHarbor Analysis**
594
595 **Claim:**
596 "Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease"
597
598 **Verdict:** WELL SUPPORTED
599 **Confidence:** 87%
600
601 **Evidence Summary:**
602
603 * Meta-analysis of 12 RCTs confirms 23-28% risk reduction
604 * Consistent findings across multiple populations
605 * Published in peer-reviewed journal (high credibility)
606
607 **Uncertainty Factors:**
608
609 * Exact percentage varies by study (20-30% range)
610
611 [[View Full Analysis →]]
612 )))
613
614 **Color-Coding System**:
615
616 * **Green**: Well-supported claims (confidence ≥75%, strong evidence)
617 * **Yellow/Orange**: Uncertain claims (confidence 40-74%, conflicting or limited evidence)
618 * **Red**: Refuted or unsupported claims (confidence <40%, contradicted by evidence)
619 * **Gray/Neutral**: Non-factual content (opinions, questions, procedural text)
620
621 ==== Interactive Highlighting Example (Detailed View) ====
622
623 (% style="width:100%; border-collapse:collapse;" %)
624 |=**Article Text**|=**Status**|=**Analysis**
625 |(((
626 A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.
627 )))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Context - no highlighting
628 |(((
629 //Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups//
630 )))|(% style="background-color:#D4EDDA; text-align:center; padding:8px;" %)🟢 **WELL SUPPORTED**|(((
631 **87% confidence**
632
633 Meta-analysis of 12 RCTs confirms 23-28% risk reduction
634
635 [[View Full Analysis]]
636 )))
637 |(((
638 The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
639 )))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Methodology - no highlighting
640 |(((
641 //Some experts believe this diet can completely prevent heart attacks//
642 )))|(% style="background-color:#FFF3CD; text-align:center; padding:8px;" %)🟡 **UNCERTAIN**|(((
643 **45% confidence**
644
645 Overstated - evidence shows risk reduction, not prevention
646
647 [[View Details]]
648 )))
649 |(((
650 Dr. Rodriguez recommends incorporating more olive oil, fish, and vegetables into daily meals.
651 )))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Recommendation - no highlighting
652 |(((
653 //The study proves that saturated fats cause heart disease//
654 )))|(% style="background-color:#F8D7DA; text-align:center; padding:8px;" %)🔴 **REFUTED**|(((
655 **15% confidence**
656
657 Claim not supported by study; correlation ≠ causation
658
659 [[View Counter-Evidence]]
660 )))
661
662 **Design Notes:**
663
664 * Highlighted claims use italics to distinguish from plain text
665 * Color backgrounds match XWiki message box colors (success/warning/error)
666 * Status column shows verdict prominently
667 * Analysis column provides quick summary with link to details
668
669 **User Actions**:
670
671 * **Hover** over highlighted claim → Tooltip appears
672 * **Click** highlighted claim → Detailed analysis modal/panel
673 * **Toggle** button to turn highlighting on/off
674 * **Keyboard**: Tab through highlighted claims
675
676 **Interaction Design**:
677
678 * Hover/click on highlighted claim → Show tooltip with:
679 * Claim text
680 * Verdict (e.g., "WELL SUPPORTED")
681 * Confidence score (e.g., "85%")
682 * Brief evidence summary
683 * Link to detailed analysis
684 * Toggle highlighting on/off (user preference)
685 * Adjustable color intensity for accessibility
686
687 **Technical Requirements**:
688
689 * Real-time highlighting as page loads (non-blocking)
690 * Claim boundary detection (start/end of assertion)
691 * Handle nested or overlapping claims
692 * Preserve original article formatting
693 * Work with various content formats (HTML, plain text, PDFs)
694
695 **Performance Requirements**:
696
697 * Highlighting renders within 500ms of page load
698 * No perceptible delay in reading experience
699 * Efficient DOM manipulation (avoid reflows)
700
701 **Accessibility**:
702
703 * Color-blind friendly palette (use patterns/icons in addition to color)
704 * Screen reader compatible (ARIA labels for claim credibility)
705 * Keyboard navigation to highlighted claims
706
707 **Implementation Notes**:
708
709 * Claims extracted and analyzed by AKEL during initial processing
710 * Highlighting data stored as annotations with byte offsets
711 * Client-side rendering of highlights based on verdict data
712 * Mobile responsive (tap instead of hover)
713
714 === 8.5 Workflow & Moderation ===
715
716 ==== FR9 — Publication Workflow ====
717
718 **Fulfills**: UN-1 (Fast access to verified content), UN-16 (Clear review status)
719
720 **Simple flow**:
721
722 1. Claim submitted
723 2. AKEL processes (automated)
724 3. If confidence > threshold: Publish (labeled as AI-generated)
725 4. If confidence < threshold: Flag for improvement
726 5. If risk score > threshold: Flag for moderator
727
728 **No multi-stage approval process**
729
730 ==== FR10 — Moderation ====
731
732 **Focus on abuse, not routine quality**:
733
734 * Automated abuse detection
735 * Moderators handle flags
736 * Quick response to harmful content
737 * Minimal involvement in routine content
738
739 ==== FR11 — Audit Trail ====
740
741 **Fulfills**: UN-14 (API access to histories), UN-15 (Evolution tracking)
742
743 * All edits logged
744 * Version history public
745 * Moderation decisions documented
746 * System improvements tracked
747
748 == 9. Non-Functional Requirements ==
749
750 === 9.1 NFR1 — Performance ===
751
752 **Fulfills**: UN-4 (Fast fact-checking), UN-11 (Responsive filtering)
753
754 * Claim processing: < 30 seconds
755 * Search response: < 2 seconds
756 * Page load: < 3 seconds
757 * 99% uptime
758
759 === 9.2 NFR2 — Scalability ===
760
761 **Fulfills**: UN-14 (API access at scale)
762
763 * Handle 10,000 claims initially
764 * Scale to 1M+ claims
765 * Support 100K+ concurrent users
766 * Automated processing scales linearly
767
768 === 9.3 NFR3 — Transparency ===
769
770 **Fulfills**: UN-7 (Evidence transparency), UN-9 (Methodology transparency), UN-13 (Citable verdicts), UN-15 (Evolution visibility)
771
772 * All algorithms open source
773 * All data exportable
774 * All decisions documented
775 * Quality metrics public
776
777 === 9.4 NFR4 — Security & Privacy ===
778
779 * Follow [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
780 * Secure authentication
781 * Data encryption
782 * Regular security audits
783
784 === 9.5 NFR5 — Maintainability ===
785
786 * Modular architecture
787 * Automated testing
788 * Continuous integration
789 * Comprehensive documentation
790
791 === NFR11: AKEL Quality Assurance Framework ===
792
793 **Fulfills:** AI safety, IFCN methodology transparency
794
795 **Specification:**
796
797 Multi-layer AI quality gates to detect hallucinations, low-confidence results, and logical inconsistencies.
798
799 ==== Quality Gate 1: Claim Extraction Validation ====
800
801 **Purpose:** Ensure extracted claims are factual assertions (not opinions/predictions)
802
803 **Checks:**
804
805 1. **Factual Statement Test:** Is this verifiable? (Yes/No)
806 2. **Opinion Detection:** Contains hedging language? ("I think", "probably", "best")
807 3. **Future Prediction Test:** Makes claims about future events?
808 4. **Specificity Score:** Contains specific entities, numbers, dates?
809
810 **Thresholds:**
811
812 * Factual: Must be "Yes"
813 * Opinion markers: <2 hedging phrases
814 * Specificity: ≥3 specific elements
815
816 **Action if Failed:** Flag as "Non-verifiable", do NOT generate verdict
817
818 ==== Quality Gate 2: Evidence Relevance Validation ====
819
820 **Purpose:** Ensure AI-linked evidence actually relates to claim
821
822 **Checks:**
823
824 1. **Semantic Similarity Score:** Evidence vs. claim (embeddings)
825 2. **Entity Overlap:** Shared people/places/things?
826 3. **Topic Relevance:** Discusses claim subject?
827
828 **Thresholds:**
829
830 * Similarity: ≥0.6 (cosine similarity)
831 * Entity overlap: ≥1 shared entity
832 * Topic relevance: ≥0.5
833
834 **Action if Failed:** Discard irrelevant evidence
835
836 ==== Quality Gate 3: Scenario Coherence Check ====
837
838 **Purpose:** Validate scenario assumptions are logical and complete
839
840 **Checks:**
841
842 1. **Completeness:** All required fields populated
843 2. **Internal Consistency:** Assumptions don't contradict
844 3. **Distinguishability:** Scenarios meaningfully different
845
846 **Thresholds:**
847
848 * Required fields: 100%
849 * Contradiction score: <0.3
850 * Scenario similarity: <0.8
851
852 **Action if Failed:** Merge duplicates, reduce confidence -20%
853
854 ==== Quality Gate 4: Verdict Confidence Assessment ====
855
856 **Purpose:** Only publish high-confidence verdicts
857
858 **Checks:**
859
860 1. **Evidence Count:** Minimum 2 sources
861 2. **Source Quality:** Average reliability ≥0.6
862 3. **Evidence Agreement:** Supporting vs. contradicting ≥0.6
863 4. **Uncertainty Factors:** Hedging in reasoning
864
865 **Confidence Tiers:**
866
867 * **HIGH (80-100%):** ≥3 sources, ≥0.7 quality, ≥80% agreement
868 * **MEDIUM (50-79%):** ≥2 sources, ≥0.6 quality, ≥60% agreement
869 * **LOW (0-49%):** <2 sources OR low quality/agreement
870 * **INSUFFICIENT:** <2 sources → DO NOT PUBLISH
871
872 **Implementation Phases:**
873
874 * **POC1:** Gates 1 & 4 only (basic validation)
875 * **POC2:** All 4 gates (complete framework)
876 * **V1.0:** Hardened with <5% hallucination rate
877
878 **Acceptance Criteria:**
879
880 * ✅ All gates operational
881 * ✅ Hallucination rate <5%
882 * ✅ Quality metrics public
883
884 === NFR12: Security Controls ===
885
886 **Fulfills:** Data protection, system integrity, user privacy, production readiness
887
888
889 **Purpose:** Protect FactHarbor systems, user data, and operations from security threats, ensuring production-grade security posture.
890
891 **Specification:**
892
893 ==== API Security ====
894
895 **Rate Limiting:**
896
897 * **Analysis endpoints:** 100 requests/hour per IP
898 * **Read endpoints:** 1,000 requests/hour per IP
899 * **Search:** 500 requests/hour per IP
900 * **Authenticated users:** 5x higher limits
901 * **Burst protection:** Max 10 requests/second
902
903 **Authentication & Authorization:**
904
905 * **API Keys:** Required for programmatic access
906 * **JWT tokens:** For user sessions (1-hour expiry)
907 * **OAuth2:** For third-party integrations
908 * **Role-Based Access Control (RBAC):**
909 * Public: Read-only access to published claims
910 * Contributor: Submit claims, provide evidence
911 * Moderator: Review contributions, manage quality
912 * Admin: System configuration, user management
913
914 **CORS Policies:**
915
916 * Whitelist approved domains only
917 * No wildcard origins in production
918 * Credentials required for sensitive endpoints
919
920 **Input Sanitization:**
921
922 * Validate all user input against schemas
923 * Sanitize HTML/JavaScript in text submissions
924 * Prevent SQL injection (use parameterized queries)
925 * Prevent command injection (no shell execution of user input)
926 * Max request size: 10MB
927 * File upload restrictions: Whitelist file types, scan for malware
928
929 -----
930
931 ==== Data Security ====
932
933 **Encryption at Rest:**
934
935 * Database encryption using AES-256
936 * Encrypted backups
937 * Key management via cloud provider KMS (AWS KMS, Google Cloud KMS)
938 * Regular key rotation (90-day cycle)
939
940 **Encryption in Transit:**
941
942 * HTTPS/TLS 1.3 only (no TLS 1.0/1.1)
943 * Strong cipher suites only
944 * HSTS (HTTP Strict Transport Security) enabled
945 * Certificate pinning for mobile apps
946
947 **Secure Credential Storage:**
948
949 * Passwords hashed with bcrypt (cost factor 12+)
950 * API keys encrypted in database
951 * Secrets stored in environment variables (never in code)
952 * Use secrets manager (AWS Secrets Manager, HashiCorp Vault)
953
954 **Data Privacy:**
955
956 * Minimal data collection (privacy by design)
957 * User data deletion on request (GDPR compliance)
958 * PII encryption in database
959 * Anonymize logs (no PII in log files)
960
961 -----
962
963 ==== Application Security ====
964
965 **OWASP Top 10 Compliance:**
966
967 1. **Broken Access Control:** RBAC implementation, path traversal prevention
968 2. **Cryptographic Failures:** Strong encryption, secure key management
969 3. **Injection:** Parameterized queries, input validation
970 4. **Insecure Design:** Security review of all features
971 5. **Security Misconfiguration:** Hardened defaults, security headers
972 6. **Vulnerable Components:** Dependency scanning (see below)
973 7. **Authentication Failures:** Strong password policy, MFA support
974 8. **Data Integrity Failures:** Signature verification, checksums
975 9. **Security Logging Failures:** Comprehensive audit logs
976 10. **Server-Side Request Forgery:** URL validation, whitelist domains
977
978 **Security Headers:**
979
980 * `Content-Security-Policy`: Strict CSP to prevent XSS
981 * `X-Frame-Options`: DENY (prevent clickjacking)
982 * `X-Content-Type-Options`: nosniff
983 * `Referrer-Policy`: strict-origin-when-cross-origin
984 * `Permissions-Policy`: Restrict browser features
985
986 **Dependency Vulnerability Scanning:**
987
988 * **Tools:** Snyk, Dependabot, npm audit, pip-audit
989 * **Frequency:** Daily automated scans
990 * **Action:** Patch critical vulnerabilities within 24 hours
991 * **Policy:** No known high/critical CVEs in production
992
993 **Security Audits:**
994
995 * **Internal:** Quarterly security reviews
996 * **External:** Annual penetration testing by certified firm
997 * **Bug Bounty:** Public bug bounty program (V1.1+)
998 * **Compliance:** SOC 2 Type II certification target (V1.5)
999
1000 -----
1001
1002 ==== Operational Security ====
1003
1004 **DDoS Protection:**
1005
1006 * CloudFlare or AWS Shield
1007 * Rate limiting at CDN layer
1008 * Automatic IP blocking for abuse patterns
1009
1010 **Monitoring & Alerting:**
1011
1012 * Real-time security event monitoring
1013 * Alerts for:
1014 * Failed login attempts (>5 in 10 minutes)
1015 * API abuse patterns
1016 * Unusual data access patterns
1017 * Security scan detections
1018 * Integration with SIEM (Security Information and Event Management)
1019
1020 **Incident Response:**
1021
1022 * Documented incident response plan
1023 * Security incident classification (P1-P4)
1024 * On-call rotation for security issues
1025 * Post-mortem for all security incidents
1026 * Public disclosure policy (coordinated disclosure)
1027
1028 **Backup & Recovery:**
1029
1030 * Daily encrypted backups
1031 * 30-day retention period
1032 * Tested recovery procedures (quarterly)
1033 * Disaster recovery plan (RTO: 4 hours, RPO: 1 hour)
1034
1035 -----
1036
1037 ==== Compliance & Standards ====
1038
1039 **GDPR Compliance:**
1040
1041 * User consent management
1042 * Right to access data
1043 * Right to deletion
1044 * Data portability
1045 * Privacy policy published
1046
1047 **Accessibility:**
1048
1049 * WCAG 2.1 AA compliance
1050 * Screen reader compatibility
1051 * Keyboard navigation
1052 * Alt text for images
1053
1054 **Browser Support:**
1055
1056 * Modern browsers only (Chrome/Edge/Firefox/Safari latest 2 versions)
1057 * No IE11 support
1058
1059 **Acceptance Criteria:**
1060
1061 * ✅ Passes OWASP ZAP security scan (no high/critical findings)
1062 * ✅ All dependencies with known vulnerabilities patched
1063 * ✅ Penetration test completed with no critical findings
1064 * ✅ Rate limiting blocks abuse attempts
1065 * ✅ Encryption at rest and in transit verified
1066 * ✅ Security headers scored A+ on securityheaders.com
1067 * ✅ Incident response plan documented and tested
1068 * ✅ 95% uptime over 30-day period
1069
1070 === NFR13: Quality Metrics Transparency ===
1071
1072 **Fulfills:** User trust, transparency, continuous improvement, IFCN methodology transparency
1073
1074
1075 **Purpose:** Provide transparent, measurable quality metrics that demonstrate AKEL's performance and build user trust in automated fact-checking.
1076
1077 **Specification:**
1078
1079 ==== Component: Public Quality Dashboard ====
1080
1081 **Core Metrics to Display:**
1082
1083 *
1084 **
1085 **1. Verdict Quality Metrics
1086
1087 **TIGERScore (Fact-Checking Quality):**
1088
1089 * **Definition:** Measures how well generated verdicts match expert fact-checker judgments
1090 * **Scale:** 0-100 (higher is better)
1091 * **Calculation:** Using TIGERScore framework (Truth-conditional accuracy, Informativeness, Generality, Evaluativeness, Relevance)
1092 * **Target:** Average ≥80 for production release
1093 * **Display:**
1094 {{code}}Verdict Quality (TIGERScore):
1095 Overall: 84.2 ▲ (+2.1 from last month)
1096
1097 Distribution:
1098 Excellent (>80): 67%
1099 Good (60-80): 28%
1100 Needs Improvement (<60): 5%
1101
1102 Trend: [Graph showing improvement over time]{{/code}}
1103
1104 **2. Hallucination & Faithfulness Metrics**
1105
1106 **AlignScore (Faithfulness to Evidence):**
1107
1108 * **Definition:** Measures how well verdicts align with actual evidence content
1109 * **Scale:** 0-1 (higher is better)
1110 * **Purpose:** Detect AI hallucinations (making claims not supported by evidence)
1111 * **Target:** Average ≥0.85, hallucination rate <5%
1112 * **Display:**
1113 {{code}}Evidence Faithfulness (AlignScore):
1114 Average: 0.87 ▼ (-0.02 from last month)
1115
1116 Hallucination Rate: 4.2%
1117 - Claims without evidence support: 3.1%
1118 - Misrepresented evidence: 1.1%
1119
1120 Action: Prompt engineering review scheduled{{/code}}
1121
1122 **3. Evidence Quality Metrics**
1123
1124 **Source Reliability:**
1125
1126 * Average source quality score (0-1 scale)
1127 * Distribution of high/medium/low quality sources
1128 * Publisher track record trends
1129
1130 **Evidence Coverage:**
1131
1132 * Average number of sources per claim
1133 * Percentage of claims with ≥2 sources (EFCSN minimum)
1134 * Geographic diversity of sources
1135
1136 **Display:**
1137 {{code}}Evidence Quality:
1138
1139 Average Sources per Claim: 4.2
1140 Claims with ≥2 sources: 94% (EFCSN compliant)
1141
1142 Source Quality Distribution:
1143 High quality (>0.8): 48%
1144 Medium quality (0.5-0.8): 43%
1145 Low quality (<0.5): 9%
1146
1147 Geographic Diversity: 23 countries represented{{/code}}
1148
1149 **4. Contributor Consensus Metrics** (when human reviewers involved)
1150
1151 **Inter-Rater Reliability (IRR):**
1152
1153 * **Calculation:** Cohen's Kappa or Fleiss' Kappa for multiple raters
1154 * **Scale:** 0-1 (higher is better)
1155 * **Interpretation:**
1156 * >0.8: Almost perfect agreement
1157 * 0.6-0.8: Substantial agreement
1158 * 0.4-0.6: Moderate agreement
1159 * <0.4: Poor agreement
1160 * **Target:** Maintain ≥0.7 (substantial agreement)
1161
1162 **Display:**
1163 {{code}}Contributor Consensus:
1164
1165 Inter-Rater Reliability (IRR): 0.73 (Substantial agreement)
1166 - Verdict agreement: 78%
1167 - Evidence quality agreement: 71%
1168 - Scenario structure agreement: 69%
1169
1170 Cases requiring moderator review: 12
1171 Moderator override rate: 8%{{/code}}
1172
1173 -----
1174
1175 ==== Quality Dashboard Implementation ====
1176
1177 **Dashboard Location:** `/quality-metrics`
1178
1179 **Update Frequency:**
1180
1181 * **POC2:** Weekly manual updates
1182 * **Beta 0:** Daily automated updates
1183 * **V1.0:** Real-time metrics (updated hourly)
1184
1185 **Dashboard Sections:**
1186
1187 1. **Overview:** Key metrics at a glance
1188 2. **Verdict Quality:** TIGERScore trends and distributions
1189 3. **Evidence Analysis:** Source quality and coverage
1190 4. **AI Performance:** Hallucination rates, AlignScore
1191 5. **Human Oversight:** Contributor consensus, review rates
1192 6. **System Health:** Processing times, error rates, uptime
1193
1194 **Example Dashboard Layout:**
1195
1196 {{code}}
1197 ┌─────────────────────────────────────────────────────────────┐
1198 │ FactHarbor Quality Metrics Last updated: │
1199 │ Public Dashboard 2 hours ago │
1200 └─────────────────────────────────────────────────────────────┘
1201
1202 📊 KEY METRICS
1203 ─────────────────────────────────────────────────────────────
1204 TIGERScore (Verdict Quality): 84.2 ▲ (+2.1)
1205 AlignScore (Faithfulness): 0.87 ▼ (-0.02)
1206 Hallucination Rate: 4.2% ✓ (Target: <5%)
1207 Average Sources per Claim: 4.2 ▲ (+0.3)
1208
1209 📈 TRENDS (30 days)
1210 ─────────────────────────────────────────────────────────────
1211 [Graph: TIGERScore trending upward]
1212 [Graph: Hallucination rate declining]
1213 [Graph: Evidence quality stable]
1214
1215 ⚠️ IMPROVEMENT TARGETS
1216 ─────────────────────────────────────────────────────────────
1217 1. Reduce hallucination rate to <3% (Current: 4.2%)
1218 2. Increase TIGERScore average to >85 (Current: 84.2)
1219 3. Maintain IRR >0.75 (Current: 0.73)
1220
1221 📄 DETAILED REPORTS
1222 ─────────────────────────────────────────────────────────────
1223 • Monthly Quality Report (PDF)
1224 • Methodology Documentation
1225 • AKEL Performance Analysis
1226 • Contributor Agreement Analysis
1227
1228 {{/code}}
1229
1230 -----
1231
1232 ==== Continuous Improvement Feedback Loop ====
1233
1234 **How Metrics Inform AKEL Improvements:**
1235
1236 1. **Identify Weak Areas:**
1237
1238 * Low TIGERScore → Review prompt engineering
1239 * High hallucination → Strengthen evidence grounding
1240 * Low IRR → Clarify evaluation criteria
1241
1242 2. **A/B Testing Integration:**
1243
1244 * Test prompt variations
1245 * Measure impact on quality metrics
1246 * Deploy winners automatically
1247
1248 3. **Alert Thresholds:**
1249
1250 * TIGERScore drops below 75 → Alert team
1251 * Hallucination rate exceeds 7% → Pause auto-publishing
1252 * IRR below 0.6 → Moderator training needed
1253
1254 4. **Monthly Quality Reviews:**
1255
1256 * Analyze trends
1257 * Identify systematic issues
1258 * Plan prompt improvements
1259 * Update AKEL models
1260
1261 -----
1262
1263 ==== Metric Calculation Details ====
1264
1265 **TIGERScore Implementation:**
1266
1267 * Reference: https://github.com/TIGER-AI-Lab/TIGERScore
1268 * Input: Generated verdict + reference verdict (from expert)
1269 * Output: 0-100 score across 5 dimensions
1270 * Requires: Test set of expert-reviewed claims (minimum 100)
1271
1272 **AlignScore Implementation:**
1273
1274 * Reference: https://github.com/yuh-zha/AlignScore
1275 * Input: Generated verdict + source evidence text
1276 * Output: 0-1 faithfulness score
1277 * Calculation: Semantic alignment between claim and evidence
1278
1279 **Source Quality Scoring:**
1280
1281 * Use existing source reliability database (e.g., NewsGuard, MBFC)
1282 * Factor in: Publication history, corrections record, transparency
1283 * Scale: 0-1 (weighted average across sources)
1284
1285 -----
1286
1287 ==== Integration Points ====
1288
1289 * **NFR11: AKEL Quality Assurance** - Metrics validate quality gate effectiveness
1290 * **FR49: A/B Testing** - Metrics measure test success
1291 * **FR11: Audit Trail** - Source of quality data
1292 * **NFR3: Transparency** - Public metrics build trust
1293
1294 **Acceptance Criteria:**
1295
1296 * ✅ All core metrics implemented and calculating correctly
1297 * ✅ Dashboard updates daily (Beta 0) or hourly (V1.0)
1298 * ✅ Alerts trigger when metrics degrade beyond thresholds
1299 * ✅ Monthly quality report auto-generates
1300 * ✅ Dashboard is publicly accessible (no login required)
1301 * ✅ Mobile-responsive dashboard design
1302 * ✅ Metrics inform quarterly AKEL improvement planning
1303
1304
1305
1306
1307 == 13. Requirements Traceability ==
1308
1309 For full traceability matrix showing which requirements fulfill which user needs, see:
1310
1311 * [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] - Section 8 includes comprehensive mapping tables
1312
1313 == 14. Related Pages ==
1314
1315 **Non-Functional Requirements (see Section 9):**
1316
1317 * [[NFR11 — AKEL Quality Assurance Framework>>#NFR11]]
1318 * [[NFR12 — Security Controls>>#NFR12]]
1319 * [[NFR13 — Quality Metrics Transparency>>#NFR13]]
1320
1321 **Other Requirements:**
1322
1323 * [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]
1324 * [[V1.0 Requirements>>FactHarbor.Specification.Requirements.V10.]]
1325 * [[Gap Analysis>>FactHarbor.Specification.Requirements.GapAnalysis]]
1326
1327 * **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need (drives these requirements)
1328 * [[Architecture>>FactHarbor.Specification.Architecture.WebHome]] - How requirements are implemented
1329 * [[Data Model>>FactHarbor.Specification.Data Model.WebHome]] - Data structures supporting requirements
1330 * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] - User interaction workflows
1331 * [[AKEL>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements
1332 * [[Global Rules>>FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]]
1333 * [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
1334
1335 = V0.9.70 Additional Requirements =
1336
1337 == Functional Requirements (Additional) ==
1338
1339 === FR44: ClaimReview Schema Implementation ===
1340
1341 **Fulfills:** UN-13 (Cite FactHarbor Verdicts), UN-14 (API Access for Integration), UN-26 (Search Engine Visibility)
1342
1343
1344 **Purpose:** Generate valid ClaimReview structured data for every published analysis to enable Google/Bing search visibility and fact-check discovery.
1345
1346 **Specification:**
1347
1348 ==== Component: Schema.org Markup Generator ====
1349
1350 FactHarbor must generate valid ClaimReview structured data following Schema.org specifications for every published claim analysis.
1351
1352 **Required JSON-LD Schema:**
1353
1354 {{code language="json"}}
1355 {
1356 "@context": "https://schema.org",
1357 "@type": "ClaimReview",
1358 "datePublished": "YYYY-MM-DD",
1359 "url": "https://factharbor.org/claims/{claim_id}",
1360 "claimReviewed": "The exact claim text",
1361 "author": {
1362 "@type": "Organization",
1363 "name": "FactHarbor",
1364 "url": "https://factharbor.org"
1365 },
1366 "reviewRating": {
1367 "@type": "Rating",
1368 "ratingValue": "1-5",
1369 "bestRating": "5",
1370 "worstRating": "1",
1371 "alternateName": "FactHarbor likelihood score"
1372 },
1373 "itemReviewed": {
1374 "@type": "Claim",
1375 "author": {
1376 "@type": "Person",
1377 "name": "Claim author if known"
1378 },
1379 "datePublished": "YYYY-MM-DD if known",
1380 "appearance": {
1381 "@type": "CreativeWork",
1382 "url": "Original claim URL if from article"
1383 }
1384 }
1385 }
1386 {{/code}}
1387
1388 **FactHarbor-Specific Mapping:**
1389
1390 **Likelihood Score to Rating Scale:**
1391
1392 * 80-100% likelihood → 5 (Highly Supported)
1393 * 60-79% likelihood → 4 (Supported)
1394 * 40-59% likelihood → 3 (Mixed/Uncertain)
1395 * 20-39% likelihood → 2 (Questionable)
1396 * 0-19% likelihood → 1 (Refuted)
1397
1398 **Multiple Scenarios Handling:**
1399
1400 * If claim has multiple scenarios with different verdicts, generate **separate ClaimReview** for each scenario
1401 * Add `disambiguatingDescription` field explaining scenario context
1402 * Example: "Scenario: If interpreted as referring to 2023 data..."
1403
1404 ==== Implementation Requirements ====
1405
1406 1. **Auto-generate** on claim publication
1407 2. **Embed** in HTML `<head>` section as JSON-LD script
1408 3. **Validate** against Schema.org validator before publishing
1409 4. **Submit** to Google Search Console for indexing
1410 5. **Update** automatically when verdict changes (integrate with FR8: Time Evolution)
1411
1412 ==== Integration Points ====
1413
1414 * **FR7: Automated Verdicts** - Source of rating data and claim text
1415 * **FR8: Time Evolution** - Triggers schema updates when verdicts change
1416 * **FR11: Audit Trail** - Logs all schema generation and update events
1417
1418 ==== Resources ====
1419
1420 * ClaimReview Project: https://www.claimreviewproject.com
1421 * Schema.org ClaimReview: https://schema.org/ClaimReview
1422 * Google Fact Check Guidelines: https://developers.google.com/search/docs/appearance/fact-check
1423
1424 **Acceptance Criteria:**
1425
1426 * ✅ Passes Google Structured Data Testing Tool
1427 * ✅ Appears in Google Fact Check Explorer within 48 hours of publication
1428 * ✅ Valid JSON-LD syntax (no errors)
1429 * ✅ All required fields populated with correct data types
1430 * ✅ Handles multi-scenario claims correctly (separate ClaimReview per scenario)
1431
1432 === FR45: User Corrections Notification System ===
1433
1434 **Fulfills:** IFCN Principle 5 (Open & Honest Corrections), EFCSN compliance
1435
1436
1437 **Purpose:** When any claim analysis is corrected, notify users who previously viewed the claim to maintain transparency and build trust.
1438
1439 **Specification:**
1440
1441 ==== Component: Corrections Visibility Framework ====
1442
1443 **Correction Types:**
1444
1445 1. **Major Correction:** Verdict changes category (e.g., "Supported" → "Refuted")
1446 2. **Significant Correction:** Likelihood score changes >20%
1447 3. **Minor Correction:** Evidence additions, source quality updates
1448 4. **Scenario Addition:** New scenario added to existing claim
1449
1450 ==== Notification Mechanisms ====
1451
1452 *
1453 **
1454 **1. In-Page Banner:
1455
1456 Display prominent banner on claim page:
1457
1458 {{code}}
1459 [!] CORRECTION NOTICE
1460 This analysis was updated on [DATE]. [View what changed] [Dismiss]
1461
1462 Major changes:
1463 • Verdict changed from "Likely True (75%)" to "Uncertain (45%)"
1464 • New contradicting evidence added from [Source]
1465 • Scenario 2 updated with additional context
1466
1467 [See full correction log]
1468 {{/code}}
1469
1470 **2. Correction Log Page:**
1471
1472 * Public changelog at `/claims/{id}/corrections`
1473 * Displays for each correction:
1474 * Date/time of correction
1475 * What changed (before/after comparison)
1476 * Why changed (reason if provided)
1477 * Who made change (AKEL auto-update vs. contributor override)
1478
1479 **3. Email Notifications (opt-in):**
1480
1481 * Send to users who bookmarked or shared the claim
1482 * Subject: "FactHarbor Correction: [Claim title]"
1483 * Include summary of changes
1484 * Link to updated analysis
1485
1486 **4. RSS/API Feed:**
1487
1488 * Corrections feed at `/corrections.rss`
1489 * API endpoint: `GET /api/corrections?since={timestamp}`
1490 * Enables external monitoring by journalists and researchers
1491
1492 ==== Display Rules ====
1493
1494 * Show banner on **ALL pages** displaying the claim (search results, related claims, embeddings)
1495 * Banner persists for **30 days** after correction
1496 * **"Corrections" count badge** on claim card
1497 * **Timestamp** on every verdict: "Last updated: [datetime]"
1498
1499 ==== IFCN Compliance Requirements ====
1500
1501 * Corrections policy published at `/corrections-policy`
1502 * User can report suspected errors via `/report-error/{claim_id}`
1503 * Link to IFCN complaint process (if FactHarbor becomes signatory)
1504 * **Scrupulous transparency:** Never silently edit analyses
1505
1506 ==== Integration Points ====
1507
1508 * **FR8: Time Evolution** - Triggers corrections when verdicts change
1509 * **FR11: Audit Trail** - Source of correction data and change history
1510 * **NFR3: Transparency** - Public correction log demonstrates commitment
1511
1512 **Acceptance Criteria:**
1513
1514 * ✅ Banner appears within 60 seconds of correction
1515 * ✅ Correction log is permanent and publicly accessible
1516 * ✅ Email notifications deliver within 5 minutes
1517 * ✅ RSS feed updates in real-time
1518 * ✅ Mobile-responsive banner design
1519 * ✅ Accessible (screen reader compatible)
1520
1521 === FR46: Image Verification System ===
1522
1523 **Fulfills:** UN-27 (Visual Claim Verification)
1524
1525
1526 **Purpose:** Verify authenticity and context of images shared with claims to detect manipulation, misattribution, and out-of-context usage.
1527
1528 **Specification:**
1529
1530 ==== Component: Multi-Method Image Verification ====
1531
1532 **Method 1: Reverse Image Search**
1533
1534 **Purpose:** Find earlier uses of the image to verify context
1535
1536 **Implementation:**
1537
1538 * Integrate APIs:
1539 * **Google Vision AI** (reverse search)
1540 * **TinEye** (oldest known uses)
1541 * **Bing Visual Search** (broad coverage)
1542
1543 **Process:**
1544
1545 1. Extract image from claim or user upload
1546 2. Query multiple reverse search services
1547 3. Analyze results for:
1548
1549 * Earliest known publication
1550 * Original context (what was it really showing?)
1551 * Publication timeline
1552 * Geographic spread
1553
1554 **Output:**
1555 {{code}}Reverse Image Search Results:
1556
1557 Earliest known use: 2019-03-15 (5 years before claim)
1558 Original context: "Photo from 2019 flooding in Mumbai"
1559 This claim uses it for: "2024 hurricane damage in Florida"
1560
1561 ⚠️ Image is OUT OF CONTEXT
1562
1563 Found in 47 other articles:
1564 • 2019-03-15: Mumbai floods (original)
1565 • 2020-07-22: Bangladesh monsoon
1566 • 2024-10-15: Current claim (misattributed)
1567
1568 [View full timeline]{{/code}}
1569
1570 -----
1571
1572 **Method 2: AI Manipulation Detection**
1573
1574 **Purpose:** Detect deepfakes, face swaps, and digital alterations
1575
1576 **Implementation:**
1577
1578 * Integrate detection services:
1579 * **Sensity AI** (deepfake detection)
1580 * **Reality Defender** (multimodal analysis)
1581 * **AWS Rekognition** (face detection inconsistencies)
1582
1583 **Detection Categories:**
1584
1585 1. **Face Manipulation:**
1586
1587 * Deepfake face swaps
1588 * Expression manipulation
1589 * Identity replacement
1590
1591 2. **Image Manipulation:**
1592
1593 * Copy-paste artifacts
1594 * Clone stamp detection
1595 * Content-aware fill detection
1596 * JPEG compression inconsistencies
1597
1598 3. **AI Generation:**
1599
1600 * Detect fully AI-generated images
1601 * Identify generation artifacts
1602 * Check for model signatures
1603
1604 **Confidence Scoring:**
1605
1606 * **HIGH (80-100%):** Strong evidence of manipulation
1607 * **MEDIUM (50-79%):** Suspicious artifacts detected
1608 * **LOW (0-49%):** Minor inconsistencies or inconclusive
1609
1610 **Output:**
1611 {{code}}Manipulation Analysis:
1612
1613 Face Manipulation: LOW RISK (12%)
1614 Image Editing: MEDIUM RISK (64%)
1615 • Clone stamp artifacts detected in sky region
1616 • JPEG compression inconsistent between objects
1617
1618 AI Generation: LOW RISK (8%)
1619
1620 ⚠️ Possible manipulation detected. Manual review recommended.{{/code}}
1621
1622 -----
1623
1624 **Method 3: Metadata Analysis (EXIF)**
1625
1626 **Purpose:** Extract technical details that may reveal manipulation or misattribution
1627
1628 **Extracted Data:**
1629
1630 * **Camera/Device:** Make, model, software
1631 * **Timestamps:** Original date, modification dates
1632 * **Location:** GPS coordinates (if present)
1633 * **Editing History:** Software used, edit count
1634 * **File Properties:** Resolution, compression, format conversions
1635
1636 **Red Flags:**
1637
1638 * Metadata completely stripped (suspicious)
1639 * Timestamp conflicts with claimed date
1640 * GPS location conflicts with claimed location
1641 * Multiple edit rounds (hiding something?)
1642 * Creation date after modification date (impossible)
1643
1644 **Output:**
1645 {{code}}Image Metadata:
1646
1647 Camera: iPhone 14 Pro
1648 Original date: 2023-08-12 14:32:15
1649 Location: 40.7128°N, 74.0060°W (New York City)
1650 Modified: 2024-10-15 08:45:22
1651 Software: Adobe Photoshop 2024
1652
1653 ⚠️ Location conflicts with claim
1654 Claim says: "Taken in Los Angeles"
1655 EXIF says: New York City
1656
1657 ⚠️ Edited 14 months after capture{{/code}}
1658
1659 -----
1660
1661 ==== Verification Workflow ====
1662
1663 **Automatic Triggers:**
1664
1665 1. User submits claim with image
1666 2. Article being analyzed contains images
1667 3. Social media post includes photos
1668
1669 **Process:**
1670
1671 1. Extract images from content
1672 2. Run all 3 verification methods in parallel
1673 3. Aggregate results into confidence score
1674 4. Generate human-readable summary
1675 5. Display prominently in analysis
1676
1677 **Display Integration:**
1678
1679 Show image verification panel in claim analysis:
1680
1681 {{code}}
1682 📷 IMAGE VERIFICATION
1683
1684 [Image thumbnail]
1685
1686 ✅ Reverse Search: Original context verified
1687 ⚠️ Manipulation: Possible editing detected (64% confidence)
1688 ✅ Metadata: Consistent with claim details
1689
1690 Overall Assessment: CAUTION ADVISED
1691 This image may have been edited. Original context appears accurate.
1692
1693 [View detailed analysis]
1694 {{/code}}
1695
1696 ==== Integration Points ====
1697
1698 * **FR7: Automated Verdicts** - Image verification affects claim credibility
1699 * **FR4: Analysis Summary** - Image findings included in summary
1700 * **UN-27: Visual Claim Verification** - Direct fulfillment
1701
1702 ==== Cost Considerations ====
1703
1704 **API Costs (estimated per image):**
1705
1706 * Google Vision AI: $0.001-0.003
1707 * TinEye: $0.02 (commercial API)
1708 * Sensity AI: $0.05-0.10
1709 * AWS Rekognition: $0.001-0.002
1710
1711 **Total per image:** $0.07-0.15**
1712
1713 **Mitigation Strategies:**
1714
1715 * Cache results for duplicate images
1716 * Use free tier quotas where available
1717 * Prioritize higher-value claims for deep analysis
1718 * Offer premium verification as paid tier
1719
1720 **Acceptance Criteria:**
1721
1722 * ✅ Reverse image search finds original sources
1723 * ✅ Manipulation detection accuracy >80% on test dataset
1724 * ✅ EXIF extraction works for major image formats (JPEG, PNG, HEIC)
1725 * ✅ Results display within 10 seconds
1726 * ✅ Mobile-friendly image comparison interface
1727 * ✅ False positive rate <15%
1728
1729 === FR47: Archive.org Integration ===
1730
1731 **Importance:** CRITICAL
1732 **Fulfills:** Evidence persistence, FR5 (Evidence linking)
1733
1734 **Purpose:** Ensure evidence remains accessible even if original sources are deleted.
1735
1736 **Specification:**
1737
1738 **Automatic Archiving:**
1739
1740 When AKEL links evidence:
1741
1742 1. Check if URL already archived (Wayback Machine API)
1743 2. If not, submit for archiving (Save Page Now API)
1744 3. Store both original URL and archive URL
1745 4. Display both to users
1746
1747 **Archive Display:**
1748
1749 {{code}}
1750 Evidence Source: [Original URL]
1751 Archived: [Archive.org URL] (Captured: [date])
1752
1753 [View Original] [View Archive]
1754 {{/code}}
1755
1756 **Fallback Logic:**
1757
1758 * If original URL unavailable → Auto-redirect to archive
1759 * If archive unavailable → Display warning
1760 * If both unavailable → Flag for manual review
1761
1762 **API Integration:**
1763
1764 * Use Wayback Machine Availability API
1765 * Use Save Page Now API (SPNv2)
1766 * Rate limiting: 15 requests/minute (Wayback limit)
1767
1768 **Acceptance Criteria:**
1769
1770 * ✅ All evidence URLs auto-archived
1771 * ✅ Archive links displayed to users
1772 * ✅ Fallback to archive if original unavailable
1773 * ✅ API rate limits respected
1774 * ✅ Archive status visible in evidence display
1775
1776 == Category 4: Community Safety ==
1777
1778 FR48: Contributor Safety Framework ===
1779
1780 **Importance:** CRITICAL
1781 **Fulfills:** UN-28 (Safe contribution environment)
1782
1783 **Purpose:** Protect contributors from harassment, doxxing, and coordinated attacks.
1784
1785 **Specification:**
1786
1787 *
1788 **
1789 **1. Privacy Protection:
1790
1791 * **Optional Pseudonymity:** Contributors can use pseudonyms
1792 * **Email Privacy:** Emails never displayed publicly
1793 * **Profile Privacy:** Contributors control what's public
1794 * **IP Logging:** Only for abuse prevention, not public
1795
1796 **2. Harassment Prevention:**
1797
1798 * **Automated Toxicity Detection:** Flag abusive comments
1799 * **Personal Information Detection:** Auto-block doxxing attempts
1800 * **Coordinated Attack Detection:** Identify brigading patterns
1801 * **Rapid Response:** Moderator alerts for harassment
1802
1803 **3. Safety Features:**
1804
1805 * **Block Users:** Contributors can block harassers
1806 * **Private Contributions:** Option to contribute anonymously
1807 * **Report Harassment:** One-click harassment reporting
1808 * **Safety Resources:** Links to support resources
1809
1810 **4. Moderator Tools:**
1811
1812 * **Quick Ban:** Immediately block abusers
1813 * **Pattern Detection:** Identify coordinated attacks
1814 * **Appeal Process:** Fair review of moderation actions
1815 * **Escalation:** Serious threats escalated to authorities
1816
1817 **5. Trusted Contributor Protection:**
1818
1819 * **Enhanced Privacy:** Additional protection for high-profile contributors
1820 * **Verification:** Optional identity verification (not public)
1821 * **Legal Support:** Resources for contributors facing legal threats
1822
1823 **Acceptance Criteria:**
1824
1825 * ✅ Pseudonyms supported
1826 * ✅ Toxicity detection active
1827 * ✅ Doxxing auto-blocked
1828 * ✅ Harassment reporting functional
1829 * ✅ Moderator tools implemented
1830 * ✅ Safety policy published
1831
1832 == Category 5: Continuous Improvement ==
1833
1834 FR49: A/B Testing Framework ===
1835
1836 **Importance:** CRITICAL
1837 **Fulfills:** Continuous system improvement
1838
1839 **Purpose:** Test and measure improvements to AKEL prompts, algorithms, and workflows.
1840
1841 **Specification:**
1842
1843 **Test Capabilities:**
1844
1845 1. **Prompt Variations:**
1846
1847 * Test different claim extraction prompts
1848 * Test different verdict generation prompts
1849 * Measure: Accuracy, clarity, completeness
1850
1851 2. **Algorithm Variations:**
1852
1853 * Test different source scoring algorithms
1854 * Test different confidence calculations
1855 * Measure: Audit accuracy, user satisfaction
1856
1857 3. **Workflow Variations:**
1858
1859 * Test different quality gate thresholds
1860 * Test different risk tier assignments
1861 * Measure: Publication rate, quality scores
1862
1863 **Implementation:**
1864
1865 * **Traffic Split:** 50/50 or 90/10 splits
1866 * **Randomization:** Consistent per claim (not per user)
1867 * **Metrics Collection:** Automatic for all variants
1868 * **Statistical Significance:** Minimum sample size calculation
1869 * **Rollout:** Winner promoted to 100% traffic
1870
1871 **A/B Test Workflow:**
1872
1873 {{code}}
1874 1. Hypothesis: "New prompt improves claim extraction"
1875 2. Design test: Control vs. Variant
1876 3. Define metrics: Extraction accuracy, completeness
1877 4. Run test: 7-14 days, minimum 100 claims each
1878 5. Analyze results: Statistical significance?
1879 6. Decision: Deploy winner or iterate
1880 {{/code}}
1881
1882 **Acceptance Criteria:**
1883
1884 * ✅ A/B testing framework implemented
1885 * ✅ Can test prompt variations
1886 * ✅ Can test algorithm variations
1887 * ✅ Metrics automatically collected
1888 * ✅ Statistical significance calculated
1889 * ✅ Results inform system improvements
1890
1891 === FR54: Evidence Deduplication ===
1892
1893 **Importance:** CRITICAL (POC2/Beta)
1894 **Fulfills:** Accurate evidence counting, quality metrics
1895
1896 **Purpose:** Avoid counting the same source multiple times when it appears in different forms.
1897
1898 **Specification:**
1899
1900 **Deduplication Logic:**
1901
1902 1. **URL Normalization:**
1903
1904 * Remove tracking parameters (?utm_source=...)
1905 * Normalize http/https
1906 * Normalize www/non-www
1907 * Handle redirects
1908
1909 2. **Content Similarity:**
1910
1911 * If two sources have >90% text similarity → Same source
1912 * If one is subset of other → Same source
1913 * Use fuzzy matching for minor differences
1914
1915 3. **Cross-Domain Syndication:**
1916
1917 * Detect wire service content (AP, Reuters)
1918 * Mark as single source if syndicated
1919 * Count original publication only
1920
1921 **Display:**
1922
1923 {{code}}
1924 Evidence Sources (3 unique, 5 total):
1925
1926 1. Original Article (NYTimes)
1927 - Also appeared in: WashPost, Guardian (syndicated)
1928
1929 2. Research Paper (Nature)
1930
1931 3. Official Statement (WHO)
1932 {{/code}}
1933
1934 **Acceptance Criteria:**
1935
1936 * ✅ URL normalization works
1937 * ✅ Content similarity detected
1938 * ✅ Syndicated content identified
1939 * ✅ Unique vs. total counts accurate
1940 * ✅ Improves evidence quality metrics
1941
1942 == Additional Requirements (Lower Importance) ==
1943
1944 FR50: OSINT Toolkit Integration ===
1945
1946
1947
1948 **Fulfills:** Advanced media verification
1949
1950 **Purpose:** Integrate open-source intelligence tools for advanced verification.
1951
1952 **Tools to Integrate:**
1953
1954 * InVID/WeVerify (video verification)
1955 * Bellingcat toolkit
1956 * Additional TBD based on V1.0 learnings
1957
1958 === FR51: Video Verification System ===
1959
1960
1961
1962 **Fulfills:** UN-27 (Visual claims), advanced media verification
1963
1964 **Purpose:** Verify video-based claims.
1965
1966 **Specification:**
1967
1968 * Keyframe extraction
1969 * Reverse video search
1970 * Deepfake detection (AI-powered)
1971 * Metadata analysis
1972 * Acoustic signature analysis
1973
1974 === FR52: Interactive Detection Training ===
1975
1976
1977
1978 **Fulfills:** Media literacy education
1979
1980 **Purpose:** Teach users to identify misinformation.
1981
1982 **Specification:**
1983
1984 * Interactive tutorials
1985 * Practice exercises
1986 * Detection quizzes
1987 * Gamification elements
1988
1989 === FR53: Cross-Organizational Sharing ===
1990
1991
1992
1993 **Fulfills:** Collaboration with other fact-checkers
1994
1995 **Purpose:** Share findings with IFCN/EFCSN members.
1996
1997 **Specification:**
1998
1999 * API for fact-checking organizations
2000 * Structured data exchange
2001 * Privacy controls
2002 * Attribution requirements
2003
2004 == Summary ==
2005
2006 **V1.0 Critical Requirements (Must Have):**
2007
2008 * FR44: ClaimReview Schema ✅
2009 * FR45: Corrections Notification ✅
2010 * FR46: Image Verification ✅
2011 * FR47: Archive.org Integration ✅
2012 * FR48: Contributor Safety ✅
2013 * FR49: A/B Testing ✅
2014 * FR54: Evidence Deduplication ✅
2015 * NFR11: Quality Assurance Framework ✅
2016 * NFR12: Security Controls ✅
2017 * NFR13: Quality Metrics Dashboard ✅
2018
2019 **V1.1+ (Future):**
2020
2021 * FR50: OSINT Integration
2022 * FR51: Video Verification
2023 * FR52: Detection Training
2024 * FR53: Cross-Org Sharing
2025
2026 **Total:** 11 critical requirements for V1.0
2027
2028 === FR54: Evidence Deduplication ===
2029
2030
2031
2032 **Fulfills:** Accurate evidence counting, quality metrics
2033
2034 **Purpose:** Avoid counting the same source multiple times when it appears in different forms.
2035
2036 **Specification:**
2037
2038 **Deduplication Logic:**
2039
2040 1. **URL Normalization:**
2041
2042 * Remove tracking parameters (?utm_source=...)
2043 * Normalize http/https
2044 * Normalize www/non-www
2045 * Handle redirects
2046
2047 2. **Content Similarity:**
2048
2049 * If two sources have >90% text similarity → Same source
2050 * If one is subset of other → Same source
2051 * Use fuzzy matching for minor differences
2052
2053 3. **Cross-Domain Syndication:**
2054
2055 * Detect wire service content (AP, Reuters)
2056 * Mark as single source if syndicated
2057 * Count original publication only
2058
2059 **Display:**
2060
2061 {{code}}
2062 Evidence Sources (3 unique, 5 total):
2063
2064 1. Original Article (NYTimes)
2065 - Also appeared in: WashPost, Guardian (syndicated)
2066
2067 2. Research Paper (Nature)
2068
2069 3. Official Statement (WHO)
2070 {{/code}}
2071
2072 **Acceptance Criteria:**
2073
2074 * ✅ URL normalization works
2075 * ✅ Content similarity detected
2076 * ✅ Syndicated content identified
2077 * ✅ Unique vs. total counts accurate
2078 * ✅ Improves evidence quality metrics
2079
2080 == Additional Requirements (Lower Importance) ==
2081
2082 FR7: Automated Verdicts (Enhanced with Quality Gates) ===
2083
2084 **POC1+ Enhancement:**
2085
2086 After AKEL generates verdict, it passes through quality gates:
2087
2088 {{code}}
2089 Workflow:
2090 1. Extract claims
2091 ↓
2092 2. [GATE 1] Validate fact-checkable
2093 ↓
2094 3. Generate scenarios
2095 ↓
2096 4. Generate verdicts
2097 ↓
2098 5. [GATE 4] Validate confidence
2099 ↓
2100 6. Display to user
2101 {{/code}}
2102
2103 **Updated Verdict States:**
2104
2105 * PUBLISHED
2106 * INSUFFICIENT_EVIDENCE
2107 * NON_FACTUAL_CLAIM
2108 * PROCESSING
2109 * ERROR
2110
2111 === FR4: Analysis Summary (Enhanced with Quality Metadata) ===
2112
2113 **POC1+ Enhancement:**
2114
2115 Display quality indicators:
2116
2117 {{code}}
2118 Analysis Summary:
2119 Verifiable Claims: 3/5
2120 High Confidence Verdicts: 1
2121 Medium Confidence: 2
2122 Evidence Sources: 12
2123 Avg Source Quality: 0.73
2124 Quality Score: 8.5/10
2125 {{/code}}