Skip to Content

Wiki source code of Requirements

Last modified by Robert Schaub on 2025/12/23 11:03
Show last authors
1 = Requirements =
2
3 **This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.**
4
5 **Core Philosophy:** Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically.
6
7 == Navigation ==
8
9 * **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need from FactHarbor (drives these requirements)
10 * **This page** - How we fulfill those needs through system design
11
12 (% class="box infomessage" %)
13 (((
14 **How to read this page:**
15
16 1. **User Needs drive Requirements**: See [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] for what users need
17 2. **Requirements define implementation**: This page shows how we fulfill those needs
18 3. **Functional Requirements (FR)**: Specific features and capabilities
19 4. **Non-Functional Requirements (NFR)**: Quality attributes (performance, security, etc.)
20
21 Each requirement references which User Needs it fulfills.
22 )))
23
24 == 1. Roles ==
25
26 **Fulfills**: UN-12 (Submit claims), UN-13 (Cite verdicts), UN-14 (API access)
27
28 FactHarbor uses three simple roles plus a reputation system.
29
30 === 1.1 Reader ===
31
32 **Who**: Anyone (no login required)
33
34 **Can**:
35 * Browse and search claims
36 * View scenarios, evidence, verdicts, and confidence scores
37 * Flag issues or errors
38 * Use filters, search, and visualization tools
39 * Submit claims automatically (new claims added if not duplicates)
40
41 **Cannot**:
42 * Modify content
43 * Access edit history details
44
45 **User Needs served**: UN-1 (Trust assessment), UN-2 (Claim verification), UN-3 (Article summary with FactHarbor analysis summary), UN-4 (Social media fact-checking), UN-5 (Source tracing), UN-7 (Evidence transparency), UN-8 (Understanding disagreement), UN-12 (Submit claims), UN-17 (In-article highlighting)
46
47 === 1.2 Contributor ===
48
49 **Who**: Registered users (earns reputation through contributions)
50
51 **Can**:
52 * Everything a Reader can do
53 * Edit claims, evidence, and scenarios
54 * Add sources and citations
55 * Suggest improvements to AI-generated content
56 * Participate in discussions
57 * Earn reputation points for quality contributions
58
59 **Reputation System**:
60 * New contributors: Limited edit privileges
61 * Established contributors (established reputation): Full edit access
62 * Trusted contributors (substantial reputation): Can approve certain changes
63 * Reputation earned through: Accepted edits, helpful flags, quality contributions
64 * Reputation lost through: Reverted edits, invalid flags, abuse
65
66 **Cannot**:
67 * Delete or hide content (only moderators)
68 * Override moderation decisions
69
70 **User Needs served**: UN-13 (Cite and contribute)
71
72 === 1.3 Moderator ===
73
74 **Who**: Trusted community members with proven track record, appointed by governance board
75
76 **Can**:
77 * Review flagged content
78 * Hide harmful or abusive content
79 * Resolve disputes between contributors
80 * Issue warnings or temporary bans
81 * Make final decisions on content disputes
82 * Access full audit logs
83
84 **Cannot**:
85 * Change governance rules
86 * Permanently ban users without board approval
87 * Override technical quality gates
88
89 **Note**: Small team (3-5 initially), supported by automated moderation tools.
90
91 === 1.4 Domain Trusted Contributors (Optional, Task-Specific) ===
92
93 **Who**: Subject matter specialists invited for specific high-stakes disputes
94
95 **Not a permanent role**: Contacted externally when needed for contested claims in their domain
96
97 **When used**:
98 * Medical claims with life/safety implications
99 * Legal interpretations with significant impact
100 * Scientific claims with high controversy
101 * Technical claims requiring specialized knowledge
102
103 **Process**:
104 * Moderator identifies need for expert input
105 * Contact expert externally (don't require them to be users)
106 * Trusted Contributor provides written opinion with sources
107 * Opinion added to claim record
108 * Trusted Contributor acknowledged in claim
109
110 **User Needs served**: UN-16 (Expert validation status)
111
112 == 2. Content States ==
113
114 **Fulfills**: UN-1 (Trust indicators), UN-16 (Review status transparency)
115
116 FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping.
117
118 === 2.1 Published ===
119
120 **Status**: Visible to all users
121
122 **Includes**:
123 * AI-generated analyses (default state)
124 * User-contributed content
125 * Edited/improved content
126
127 **Quality Indicators** (displayed with content):
128 * **Confidence Score**: 0-100% (AI's confidence in analysis)
129 * **Source Quality Score**: 0-100% (based on source track record)
130 * **Controversy Flag**: If high dispute/edit activity
131 * **Completeness Score**: % of expected fields filled
132 * **Last Updated**: Date of most recent change
133 * **Edit Count**: Number of revisions
134 * **Review Status**: AI-generated / Human-reviewed / Expert-validated
135
136 **Automatic Warnings**:
137 * Confidence < 60%: "Low confidence - use caution"
138 * Source quality < 40%: "Sources may be unreliable"
139 * High controversy: "Disputed - multiple interpretations exist"
140 * Medical/Legal/Safety domain: "Seek professional advice"
141
142 **User Needs served**: UN-1 (Trust score), UN-9 (Methodology transparency), UN-15 (Evolution timeline), UN-16 (Review status)
143
144 === 2.2 Hidden ===
145
146 **Status**: Not visible to regular users (only to moderators)
147
148 **Reasons**:
149 * Spam or advertising
150 * Personal attacks or harassment
151 * Illegal content
152 * Privacy violations
153 * Deliberate misinformation (verified)
154 * Abuse or harmful content
155
156 **Process**:
157 * Automated detection flags for moderator review
158 * Moderator confirms and hides
159 * Original author notified with reason
160 * Can appeal to board if disputes moderator decision
161
162 **Note**: Content is hidden, not deleted (for audit trail)
163
164 == 3. Contribution Rules ==
165
166 === 3.1 All Contributors Must ===
167
168 * Provide sources for factual claims
169 * Use clear, neutral language in FactHarbor's own summaries
170 * Respect others and maintain civil discourse
171 * Accept community feedback constructively
172 * Focus on improving quality, not protecting ego
173
174 === 3.2 AKEL (AI System) ===
175
176 **AKEL is the primary system**. Human contributions supplement and train AKEL.
177
178 **AKEL Must**:
179 * Mark all outputs as AI-generated
180 * Display confidence scores prominently
181 * Provide source citations
182 * Flag uncertainty clearly
183 * Identify contradictions in evidence
184 * Learn from human corrections
185
186 **When AKEL Makes Errors**:
187 1. Capture the error pattern (what, why, how common)
188 2. Improve the system (better prompt, model, validation)
189 3. Re-process affected claims automatically
190 4. Measure improvement (did quality increase?)
191
192 **Human Role**: Train AKEL through corrections, not replace AKEL
193
194 === 3.3 Contributors Should ===
195
196 * Improve clarity and structure
197 * Add missing sources
198 * Flag errors for system improvement
199 * Suggest better ways to present information
200 * Participate in quality discussions
201
202 === 3.4 Moderators Must ===
203
204 * Be impartial
205 * Document moderation decisions
206 * Respond to appeals promptly
207 * Use automated tools to scale efforts
208 * Focus on abuse/harm, not routine quality control
209
210 == 4. Quality Standards ==
211
212 **Fulfills**: UN-5 (Source reliability), UN-6 (Publisher track records), UN-7 (Evidence transparency), UN-9 (Methodology transparency)
213
214 === 4.1 Source Requirements ===
215
216 **Track Record Over Credentials**:
217 * Sources evaluated by historical accuracy
218 * Correction policy matters
219 * Independence from conflicts of interest
220 * Methodology transparency
221
222 **Source Quality Database**:
223 * Automated tracking of source accuracy
224 * Correction frequency
225 * Reliability score (updated continuously)
226 * Users can see source track record
227
228 **No automatic trust** for government, academia, or media - all evaluated by track record.
229
230 **User Needs served**: UN-5 (Source provenance), UN-6 (Publisher reliability)
231
232 === 4.2 Claim Requirements ===
233
234 * Clear subject and assertion
235 * Verifiable with available information
236 * Sourced (or explicitly marked as needing sources)
237 * Neutral language in FactHarbor summaries
238 * Appropriate context provided
239
240 **User Needs served**: UN-2 (Claim extraction and verification)
241
242 === 4.3 Evidence Requirements ===
243
244 * Publicly accessible (or explain why not)
245 * Properly cited with attribution
246 * Relevant to claim being evaluated
247 * Original source preferred over secondary
248
249 **User Needs served**: UN-7 (Evidence transparency)
250
251 === 4.4 Confidence Scoring ===
252
253 **Automated confidence calculation based on**:
254 * Source quality scores
255 * Evidence consistency
256 * Contradiction detection
257 * Completeness of analysis
258 * Historical accuracy of similar claims
259
260 **Thresholds**:
261 * < 40%: Too low to publish (needs improvement)
262 * 40-60%: Published with "Low confidence" warning
263 * 60-80%: Published as standard
264 * 80-100%: Published as "High confidence"
265
266 **User Needs served**: UN-1 (Trust assessment), UN-9 (Methodology transparency)
267
268 == 5. Automated Risk Scoring ==
269
270 **Fulfills**: UN-10 (Manipulation detection), UN-16 (Appropriate review level)
271
272 **Replace manual risk tiers with continuous automated scoring**.
273
274 === 5.1 Risk Score Calculation ===
275
276 **Factors** (weighted algorithm):
277 * **Domain sensitivity**: Medical, legal, safety auto-flagged higher
278 * **Potential impact**: Views, citations, spread
279 * **Controversy level**: Flags, disputes, edit wars
280 * **Uncertainty**: Low confidence, contradictory evidence
281 * **Source reliability**: Track record of sources used
282
283 **Score**: 0-100 (higher = more risk)
284
285 === 5.2 Automated Actions ===
286
287 * **Score > 80**: Flag for moderator review before publication
288 * **Score 60-80**: Publish with prominent warnings
289 * **Score 40-60**: Publish with standard warnings
290 * **Score < 40**: Publish normally
291
292 **Continuous monitoring**: Risk score recalculated as new information emerges
293
294 **User Needs served**: UN-10 (Detect manipulation tactics), UN-16 (Review status)
295
296 == 6. System Improvement Process ==
297
298 **Core principle**: Fix the system, not just the data.
299
300 === 6.1 Error Capture ===
301
302 **When users flag errors or make corrections**:
303 1. What was wrong? (categorize)
304 2. What should it have been?
305 3. Why did the system fail? (root cause)
306 4. How common is this pattern?
307 5. Store in ErrorPattern table (improvement queue)
308
309 === 6.2 Continuous Improvement Cycle ===
310
311 1. **Review**: Analyze top error patterns
312 2. **Develop**: Create fix (prompt, model, validation)
313 3. **Test**: Validate fix on sample claims
314 4. **Deploy**: Roll out if quality improves
315 5. **Re-process**: Automatically update affected claims
316 6. **Monitor**: Track quality metrics
317
318 === 6.3 Quality Metrics Dashboard ===
319
320 **Track continuously**:
321 * Error rate by category
322 * Source quality distribution
323 * Confidence score trends
324 * User flag rate (issues found)
325 * Correction acceptance rate
326 * Re-work rate
327 * Claims processed per hour
328
329 **Goal**: continuous improvement in error rate
330
331 == 7. Automated Quality Monitoring ==
332
333 **Replace manual audit sampling with automated monitoring**.
334
335 === 7.1 Continuous Metrics ===
336
337 * **Source quality**: Track record database
338 * **Consistency**: Contradiction detection
339 * **Clarity**: Readability scores
340 * **Completeness**: Field validation
341 * **Accuracy**: User corrections tracked
342
343 === 7.2 Anomaly Detection ===
344
345 **Automated alerts for**:
346 * Sudden quality drops
347 * Unusual patterns
348 * Contradiction clusters
349 * Source reliability changes
350 * User behavior anomalies
351
352 === 7.3 Targeted Review ===
353
354 * Review only flagged items
355 * Random sampling for calibration (not quotas)
356 * Learn from corrections to improve automation
357
358 == 8. Functional Requirements ==
359
360 This section defines specific features that fulfill user needs.
361
362 === 8.1 Claim Intake & Normalization ===
363
364 ==== FR1 — Claim Intake ====
365
366 **Fulfills**: UN-2 (Claim extraction), UN-4 (Quick fact-checking), UN-12 (Submit claims)
367
368 * Users submit claims via simple form or API
369 * Claims can be text, URL, or image
370 * Duplicate detection (semantic similarity)
371 * Auto-categorization by domain
372
373 ==== FR2 — Claim Normalization ====
374
375 **Fulfills**: UN-2 (Claim verification)
376
377 * Standardize to clear assertion format
378 * Extract key entities (who, what, when, where)
379 * Identify claim type (factual, predictive, evaluative)
380 * Link to existing similar claims
381
382 ==== FR3 — Claim Classification ====
383
384 **Fulfills**: UN-11 (Filtered research)
385
386 * Domain: Politics, Science, Health, etc.
387 * Type: Historical fact, current stat, prediction, etc.
388 * Risk score: Automated calculation
389 * Complexity: Simple, moderate, complex
390
391 === 8.2 Scenario System ===
392
393 ==== FR4 — Scenario Generation ====
394
395 **Fulfills**: UN-2 (Context-dependent verification), UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
396
397 **Automated scenario creation**:
398 * AKEL analyzes claim and generates likely scenarios (use-cases and contexts)
399 * Each scenario includes: assumptions, definitions, boundaries, evidence context
400 * Users can flag incorrect scenarios
401 * System learns from corrections
402
403 **Key Concept**: Scenarios represent different interpretations or contexts (e.g., "Clinical trials with healthy adults" vs. "Real-world data with diverse populations")
404
405 ==== FR5 — Evidence Linking ====
406
407 **Fulfills**: UN-5 (Source tracing), UN-7 (Evidence transparency)
408
409 * Automated evidence discovery from sources
410 * Relevance scoring
411 * Contradiction detection
412 * Source quality assessment
413
414 ==== FR6 — Scenario Comparison ====
415
416 **Fulfills**: UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
417
418 * Side-by-side comparison interface
419 * Highlight key differences between scenarios
420 * Show evidence supporting each scenario
421 * Display confidence scores per scenario
422
423 === 8.3 Verdicts & Analysis ===
424
425 ==== FR7 — Automated Verdicts ====
426
427 **Fulfills**: UN-1 (Trust score), UN-2 (Verification verdicts), UN-3 (Article summary with FactHarbor analysis summary), UN-13 (Cite verdicts)
428
429 * AKEL generates verdict based on evidence within each scenario
430 * **Likelihood range** displayed (e.g., "0.70-0.85 (likely true)") - NOT binary true/false
431 * **Uncertainty factors** explicitly listed (e.g., "Small sample sizes", "Long-term effects unknown")
432 * Confidence score displayed prominently
433 * Source quality indicators shown
434 * Contradictions noted
435 * Uncertainty acknowledged
436
437 **Key Innovation**: Detailed probabilistic verdicts with explicit uncertainty, not binary judgments
438
439 ==== FR8 — Time Evolution ====
440
441 **Fulfills**: UN-15 (Verdict evolution timeline)
442
443 * Claims and verdicts update as new evidence emerges
444 * Version history maintained for all verdicts
445 * Changes highlighted
446 * Confidence score trends visible
447 * Users can see "as of date X, what did we know?"
448
449 === 8.4 User Interface & Presentation ===
450
451 ==== FR12 — Two-Panel Summary View (Article Summary with FactHarbor Analysis Summary) ====
452
453 **Fulfills**: UN-3 (Article Summary with FactHarbor Analysis Summary)
454
455 **Purpose**: Provide side-by-side comparison of what a document claims vs. FactHarbor's complete analysis of its credibility
456
457 **Left Panel: Article Summary**:
458 * Document title, source, and claimed credibility
459 * "The Big Picture" - main thesis or position change
460 * "Key Findings" - structured summary of document's main claims
461 * "Reasoning" - document's explanation for positions
462 * "Conclusion" - document's bottom line
463
464 **Right Panel: FactHarbor Analysis Summary**:
465 * FactHarbor's independent source credibility assessment
466 * Claim-by-claim verdicts with confidence scores
467 * Methodology assessment (strengths, limitations)
468 * Overall verdict on document quality
469 * Analysis ID for reference
470
471 **Design Principles**:
472 * No scrolling required - both panels visible simultaneously
473 * Visual distinction between "what they say" and "FactHarbor's analysis"
474 * Color coding for verdicts (supported, uncertain, refuted)
475 * Confidence percentages clearly visible
476 * Mobile responsive (panels stack vertically on small screens)
477
478 **Implementation Notes**:
479 * Generated automatically by AKEL for every analyzed document
480 * Updates when verdict evolves (maintains version history)
481 * Exportable as standalone summary report
482 * Shareable via permanent URL
483
484 ==== FR13 — In-Article Claim Highlighting ====
485
486 **Fulfills**: UN-17 (In-article claim highlighting)
487
488 **Purpose**: Enable readers to quickly assess claim credibility while reading by visually highlighting factual claims with color-coded indicators
489
490 ==== Visual Example: Article with Highlighted Claims ====
491
492 (% class="box" %)
493 (((
494 **Article: "New Study Shows Benefits of Mediterranean Diet"**
495
496 A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.
497
498 (% class="box successmessage" style="margin:10px 0;" %)
499 (((
500 🟢 **Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups**
501
502 (% style="font-size:0.9em; color:#666;" %)
503 ↑ WELL SUPPORTED • 87% confidence
504 [[Click for evidence details →]]
505 (%%)
506 )))
507
508 The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
509
510 (% class="box warningmessage" style="margin:10px 0;" %)
511 (((
512 🟡 **Some experts believe this diet can completely prevent heart attacks**
513
514 (% style="font-size:0.9em; color:#666;" %)
515 ↑ UNCERTAIN • 45% confidence
516 Overstated - evidence shows risk reduction, not prevention
517 [[Click for details →]]
518 (%%)
519 )))
520
521 Dr. Maria Rodriguez, lead researcher, recommends incorporating more olive oil, fish, and vegetables into daily meals.
522
523 (% class="box errormessage" style="margin:10px 0;" %)
524 (((
525 🔴 **The study proves that saturated fats cause heart disease**
526
527 (% style="font-size:0.9em; color:#666;" %)
528 ↑ REFUTED • 15% confidence
529 Claim not supported by study design; correlation ≠ causation
530 [[Click for counter-evidence →]]
531 (%%)
532 )))
533
534 Participants also reported feeling more energetic and experiencing better sleep quality, though these were secondary measures.
535 )))
536
537 **Legend:**
538 * 🟢 = Well-supported claim (confidence ≥75%)
539 * 🟡 = Uncertain claim (confidence 40-74%)
540 * 🔴 = Refuted/unsupported claim (confidence <40%)
541 * Plain text = Non-factual content (context, opinions, recommendations)
542
543 ==== Tooltip on Hover/Click ====
544
545 (% class="box infomessage" %)
546 (((
547 **FactHarbor Analysis**
548
549 **Claim:**
550 "Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease"
551
552 **Verdict:** WELL SUPPORTED
553 **Confidence:** 87%
554
555 **Evidence Summary:**
556 * Meta-analysis of 12 RCTs confirms 23-28% risk reduction
557 * Consistent findings across multiple populations
558 * Published in peer-reviewed journal (high credibility)
559
560 **Uncertainty Factors:**
561 * Exact percentage varies by study (20-30% range)
562
563 [[View Full Analysis →]]
564 )))
565
566 **Color-Coding System**:
567 * **Green**: Well-supported claims (confidence ≥75%, strong evidence)
568 * **Yellow/Orange**: Uncertain claims (confidence 40-74%, conflicting or limited evidence)
569 * **Red**: Refuted or unsupported claims (confidence <40%, contradicted by evidence)
570 * **Gray/Neutral**: Non-factual content (opinions, questions, procedural text)
571
572 ==== Interactive Highlighting Example (Detailed View) ====
573
574 (% style="width:100%; border-collapse:collapse;" %)
575 |=**Article Text**|=**Status**|=**Analysis**
576 |(((A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Context - no highlighting
577 |(((//Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups//)))|(% style="background-color:#D4EDDA; text-align:center; padding:8px;" %)🟢 **WELL SUPPORTED**|(((
578 **87% confidence**
579
580 Meta-analysis of 12 RCTs confirms 23-28% risk reduction
581
582 [[View Full Analysis]]
583 )))
584 |(((The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Methodology - no highlighting
585 |(((//Some experts believe this diet can completely prevent heart attacks//)))|(% style="background-color:#FFF3CD; text-align:center; padding:8px;" %)🟡 **UNCERTAIN**|(((
586 **45% confidence**
587
588 Overstated - evidence shows risk reduction, not prevention
589
590 [[View Details]]
591 )))
592 |(((Dr. Rodriguez recommends incorporating more olive oil, fish, and vegetables into daily meals.)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Recommendation - no highlighting
593 |(((//The study proves that saturated fats cause heart disease//)))|(% style="background-color:#F8D7DA; text-align:center; padding:8px;" %)🔴 **REFUTED**|(((
594 **15% confidence**
595
596 Claim not supported by study; correlation ≠ causation
597
598 [[View Counter-Evidence]]
599 )))
600
601 **Design Notes:**
602 * Highlighted claims use italics to distinguish from plain text
603 * Color backgrounds match XWiki message box colors (success/warning/error)
604 * Status column shows verdict prominently
605 * Analysis column provides quick summary with link to details
606
607 **User Actions**:
608 * **Hover** over highlighted claim → Tooltip appears
609 * **Click** highlighted claim → Detailed analysis modal/panel
610 * **Toggle** button to turn highlighting on/off
611 * **Keyboard**: Tab through highlighted claims
612
613 **Interaction Design**:
614 * Hover/click on highlighted claim → Show tooltip with:
615 * Claim text
616 * Verdict (e.g., "WELL SUPPORTED")
617 * Confidence score (e.g., "85%")
618 * Brief evidence summary
619 * Link to detailed analysis
620 * Toggle highlighting on/off (user preference)
621 * Adjustable color intensity for accessibility
622
623 **Technical Requirements**:
624 * Real-time highlighting as page loads (non-blocking)
625 * Claim boundary detection (start/end of assertion)
626 * Handle nested or overlapping claims
627 * Preserve original article formatting
628 * Work with various content formats (HTML, plain text, PDFs)
629
630 **Performance Requirements**:
631 * Highlighting renders within 500ms of page load
632 * No perceptible delay in reading experience
633 * Efficient DOM manipulation (avoid reflows)
634
635 **Accessibility**:
636 * Color-blind friendly palette (use patterns/icons in addition to color)
637 * Screen reader compatible (ARIA labels for claim credibility)
638 * Keyboard navigation to highlighted claims
639
640 **Implementation Notes**:
641 * Claims extracted and analyzed by AKEL during initial processing
642 * Highlighting data stored as annotations with byte offsets
643 * Client-side rendering of highlights based on verdict data
644 * Mobile responsive (tap instead of hover)
645
646 === 8.5 Workflow & Moderation ===
647
648 ==== FR9 — Publication Workflow ====
649
650 **Fulfills**: UN-1 (Fast access to verified content), UN-16 (Clear review status)
651
652 **Simple flow**:
653 1. Claim submitted
654 2. AKEL processes (automated)
655 3. If confidence > threshold: Publish (labeled as AI-generated)
656 4. If confidence < threshold: Flag for improvement
657 5. If risk score > threshold: Flag for moderator
658
659 **No multi-stage approval process**
660
661 ==== FR10 — Moderation ====
662
663 **Focus on abuse, not routine quality**:
664 * Automated abuse detection
665 * Moderators handle flags
666 * Quick response to harmful content
667 * Minimal involvement in routine content
668
669 ==== FR11 — Audit Trail ====
670
671 **Fulfills**: UN-14 (API access to histories), UN-15 (Evolution tracking)
672
673 * All edits logged
674 * Version history public
675 * Moderation decisions documented
676 * System improvements tracked
677
678 == 9. Non-Functional Requirements ==
679
680 === 9.1 NFR1 — Performance ===
681
682 **Fulfills**: UN-4 (Fast fact-checking), UN-11 (Responsive filtering)
683
684 * Claim processing: < 30 seconds
685 * Search response: < 2 seconds
686 * Page load: < 3 seconds
687 * 99% uptime
688
689 === 9.2 NFR2 — Scalability ===
690
691 **Fulfills**: UN-14 (API access at scale)
692
693 * Handle 10,000 claims initially
694 * Scale to 1M+ claims
695 * Support 100K+ concurrent users
696 * Automated processing scales linearly
697
698 === 9.3 NFR3 — Transparency ===
699
700 **Fulfills**: UN-7 (Evidence transparency), UN-9 (Methodology transparency), UN-13 (Citable verdicts), UN-15 (Evolution visibility)
701
702 * All algorithms open source
703 * All data exportable
704 * All decisions documented
705 * Quality metrics public
706
707 === 9.4 NFR4 — Security & Privacy ===
708
709 * Follow [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
710 * Secure authentication
711 * Data encryption
712 * Regular security audits
713
714 === 9.5 NFR5 — Maintainability ===
715
716 * Modular architecture
717 * Automated testing
718 * Continuous integration
719 * Comprehensive documentation
720
721 === NFR11: AKEL Quality Assurance Framework ===
722
723 **Fulfills:** AI safety, IFCN methodology transparency
724
725 **Specification:**
726
727 Multi-layer AI quality gates to detect hallucinations, low-confidence results, and logical inconsistencies.
728
729 ==== Quality Gate 1: Claim Extraction Validation ====
730
731 **Purpose:** Ensure extracted claims are factual assertions (not opinions/predictions)
732
733 **Checks:**
734 1. **Factual Statement Test:** Is this verifiable? (Yes/No)
735 2. **Opinion Detection:** Contains hedging language? ("I think", "probably", "best")
736 3. **Future Prediction Test:** Makes claims about future events?
737 4. **Specificity Score:** Contains specific entities, numbers, dates?
738
739 **Thresholds:**
740 * Factual: Must be "Yes"
741 * Opinion markers: <2 hedging phrases
742 * Specificity: ≥3 specific elements
743
744 **Action if Failed:** Flag as "Non-verifiable", do NOT generate verdict
745
746 ==== Quality Gate 2: Evidence Relevance Validation ====
747
748 **Purpose:** Ensure AI-linked evidence actually relates to claim
749
750 **Checks:**
751 1. **Semantic Similarity Score:** Evidence vs. claim (embeddings)
752 2. **Entity Overlap:** Shared people/places/things?
753 3. **Topic Relevance:** Discusses claim subject?
754
755 **Thresholds:**
756 * Similarity: ≥0.6 (cosine similarity)
757 * Entity overlap: ≥1 shared entity
758 * Topic relevance: ≥0.5
759
760 **Action if Failed:** Discard irrelevant evidence
761
762 ==== Quality Gate 3: Scenario Coherence Check ====
763
764 **Purpose:** Validate scenario assumptions are logical and complete
765
766 **Checks:**
767 1. **Completeness:** All required fields populated
768 2. **Internal Consistency:** Assumptions don't contradict
769 3. **Distinguishability:** Scenarios meaningfully different
770
771 **Thresholds:**
772 * Required fields: 100%
773 * Contradiction score: <0.3
774 * Scenario similarity: <0.8
775
776 **Action if Failed:** Merge duplicates, reduce confidence -20%
777
778 ==== Quality Gate 4: Verdict Confidence Assessment ====
779
780 **Purpose:** Only publish high-confidence verdicts
781
782 **Checks:**
783 1. **Evidence Count:** Minimum 2 sources
784 2. **Source Quality:** Average reliability ≥0.6
785 3. **Evidence Agreement:** Supporting vs. contradicting ≥0.6
786 4. **Uncertainty Factors:** Hedging in reasoning
787
788 **Confidence Tiers:**
789 * **HIGH (80-100%):** ≥3 sources, ≥0.7 quality, ≥80% agreement
790 * **MEDIUM (50-79%):** ≥2 sources, ≥0.6 quality, ≥60% agreement
791 * **LOW (0-49%):** <2 sources OR low quality/agreement
792 * **INSUFFICIENT:** <2 sources → DO NOT PUBLISH
793
794 **Implementation Phases:**
795 * **POC1:** Gates 1 & 4 only (basic validation)
796 * **POC2:** All 4 gates (complete framework)
797 * **V1.0:** Hardened with <5% hallucination rate
798
799 **Acceptance Criteria:**
800 * ✅ All gates operational
801 * ✅ Hallucination rate <5%
802 * ✅ Quality metrics public
803
804 === NFR12: Security Controls ===
805
806 **Fulfills:** Data protection, system integrity, user privacy, production readiness
807
808 **Phase:** Beta 0 (essential), V1.0 (complete) **BLOCKER**
809
810 **Purpose:** Protect FactHarbor systems, user data, and operations from security threats, ensuring production-grade security posture.
811
812 **Specification:**
813
814 ==== API Security ====
815
816 **Rate Limiting:**
817 * **Analysis endpoints:** 100 requests/hour per IP
818 * **Read endpoints:** 1,000 requests/hour per IP
819 * **Search:** 500 requests/hour per IP
820 * **Authenticated users:** 5x higher limits
821 * **Burst protection:** Max 10 requests/second
822
823 **Authentication & Authorization:**
824 * **API Keys:** Required for programmatic access
825 * **JWT tokens:** For user sessions (1-hour expiry)
826 * **OAuth2:** For third-party integrations
827 * **Role-Based Access Control (RBAC):**
828 * Public: Read-only access to published claims
829 * Contributor: Submit claims, provide evidence
830 * Moderator: Review contributions, manage quality
831 * Admin: System configuration, user management
832
833 **CORS Policies:**
834 * Whitelist approved domains only
835 * No wildcard origins in production
836 * Credentials required for sensitive endpoints
837
838 **Input Sanitization:**
839 * Validate all user input against schemas
840 * Sanitize HTML/JavaScript in text submissions
841 * Prevent SQL injection (use parameterized queries)
842 * Prevent command injection (no shell execution of user input)
843 * Max request size: 10MB
844 * File upload restrictions: Whitelist file types, scan for malware
845
846 ---
847
848 ==== Data Security ====
849
850 **Encryption at Rest:**
851 * Database encryption using AES-256
852 * Encrypted backups
853 * Key management via cloud provider KMS (AWS KMS, Google Cloud KMS)
854 * Regular key rotation (90-day cycle)
855
856 **Encryption in Transit:**
857 * HTTPS/TLS 1.3 only (no TLS 1.0/1.1)
858 * Strong cipher suites only
859 * HSTS (HTTP Strict Transport Security) enabled
860 * Certificate pinning for mobile apps
861
862 **Secure Credential Storage:**
863 * Passwords hashed with bcrypt (cost factor 12+)
864 * API keys encrypted in database
865 * Secrets stored in environment variables (never in code)
866 * Use secrets manager (AWS Secrets Manager, HashiCorp Vault)
867
868 **Data Privacy:**
869 * Minimal data collection (privacy by design)
870 * User data deletion on request (GDPR compliance)
871 * PII encryption in database
872 * Anonymize logs (no PII in log files)
873
874 ---
875
876 ==== Application Security ====
877
878 **OWASP Top 10 Compliance:**
879
880 1. **Broken Access Control:** RBAC implementation, path traversal prevention
881 2. **Cryptographic Failures:** Strong encryption, secure key management
882 3. **Injection:** Parameterized queries, input validation
883 4. **Insecure Design:** Security review of all features
884 5. **Security Misconfiguration:** Hardened defaults, security headers
885 6. **Vulnerable Components:** Dependency scanning (see below)
886 7. **Authentication Failures:** Strong password policy, MFA support
887 8. **Data Integrity Failures:** Signature verification, checksums
888 9. **Security Logging Failures:** Comprehensive audit logs
889 10. **Server-Side Request Forgery:** URL validation, whitelist domains
890
891 **Security Headers:**
892 * `Content-Security-Policy`: Strict CSP to prevent XSS
893 * `X-Frame-Options`: DENY (prevent clickjacking)
894 * `X-Content-Type-Options`: nosniff
895 * `Referrer-Policy`: strict-origin-when-cross-origin
896 * `Permissions-Policy`: Restrict browser features
897
898 **Dependency Vulnerability Scanning:**
899 * **Tools:** Snyk, Dependabot, npm audit, pip-audit
900 * **Frequency:** Daily automated scans
901 * **Action:** Patch critical vulnerabilities within 24 hours
902 * **Policy:** No known high/critical CVEs in production
903
904 **Security Audits:**
905 * **Internal:** Quarterly security reviews
906 * **External:** Annual penetration testing by certified firm
907 * **Bug Bounty:** Public bug bounty program (V1.1+)
908 * **Compliance:** SOC 2 Type II certification target (V1.5)
909
910 ---
911
912 ==== Operational Security ====
913
914 **DDoS Protection:**
915 * CloudFlare or AWS Shield
916 * Rate limiting at CDN layer
917 * Automatic IP blocking for abuse patterns
918
919 **Monitoring & Alerting:**
920 * Real-time security event monitoring
921 * Alerts for:
922 * Failed login attempts (>5 in 10 minutes)
923 * API abuse patterns
924 * Unusual data access patterns
925 * Security scan detections
926 * Integration with SIEM (Security Information and Event Management)
927
928 **Incident Response:**
929 * Documented incident response plan
930 * Security incident classification (P1-P4)
931 * On-call rotation for security issues
932 * Post-mortem for all security incidents
933 * Public disclosure policy (coordinated disclosure)
934
935 **Backup & Recovery:**
936 * Daily encrypted backups
937 * 30-day retention period
938 * Tested recovery procedures (quarterly)
939 * Disaster recovery plan (RTO: 4 hours, RPO: 1 hour)
940
941 ---
942
943 ==== Compliance & Standards ====
944
945 **GDPR Compliance:**
946 * User consent management
947 * Right to access data
948 * Right to deletion
949 * Data portability
950 * Privacy policy published
951
952 **Accessibility:**
953 * WCAG 2.1 AA compliance
954 * Screen reader compatibility
955 * Keyboard navigation
956 * Alt text for images
957
958 **Browser Support:**
959 * Modern browsers only (Chrome/Edge/Firefox/Safari latest 2 versions)
960 * No IE11 support
961
962 **Acceptance Criteria:**
963
964 * ✅ Passes OWASP ZAP security scan (no high/critical findings)
965 * ✅ All dependencies with known vulnerabilities patched
966 * ✅ Penetration test completed with no critical findings
967 * ✅ Rate limiting blocks abuse attempts
968 * ✅ Encryption at rest and in transit verified
969 * ✅ Security headers scored A+ on securityheaders.com
970 * ✅ Incident response plan documented and tested
971 * ✅ 95% uptime over 30-day period
972
973
974 === NFR13: Quality Metrics Transparency ===
975
976 **Fulfills:** User trust, transparency, continuous improvement, IFCN methodology transparency
977
978 **Phase:** POC2 (internal), Beta 0 (public), V1.0 (real-time)
979
980 **Purpose:** Provide transparent, measurable quality metrics that demonstrate AKEL's performance and build user trust in automated fact-checking.
981
982 **Specification:**
983
984 ==== Component: Public Quality Dashboard ====
985
986 **Core Metrics to Display:**
987
988 **1. Verdict Quality Metrics**
989
990 **TIGERScore (Fact-Checking Quality):**
991 * **Definition:** Measures how well generated verdicts match expert fact-checker judgments
992 * **Scale:** 0-100 (higher is better)
993 * **Calculation:** Using TIGERScore framework (Truth-conditional accuracy, Informativeness, Generality, Evaluativeness, Relevance)
994 * **Target:** Average ≥80 for production release
995 * **Display:**
996 {{code}}
997 Verdict Quality (TIGERScore):
998 Overall: 84.2 ▲ (+2.1 from last month)
999
1000 Distribution:
1001 Excellent (>80): 67%
1002 Good (60-80): 28%
1003 Needs Improvement (<60): 5%
1004
1005 Trend: [Graph showing improvement over time]
1006 {{/code}}
1007
1008 **2. Hallucination & Faithfulness Metrics**
1009
1010 **AlignScore (Faithfulness to Evidence):**
1011 * **Definition:** Measures how well verdicts align with actual evidence content
1012 * **Scale:** 0-1 (higher is better)
1013 * **Purpose:** Detect AI hallucinations (making claims not supported by evidence)
1014 * **Target:** Average ≥0.85, hallucination rate <5%
1015 * **Display:**
1016 {{code}}
1017 Evidence Faithfulness (AlignScore):
1018 Average: 0.87 ▼ (-0.02 from last month)
1019
1020 Hallucination Rate: 4.2%
1021 - Claims without evidence support: 3.1%
1022 - Misrepresented evidence: 1.1%
1023
1024 Action: Prompt engineering review scheduled
1025 {{/code}}
1026
1027 **3. Evidence Quality Metrics**
1028
1029 **Source Reliability:**
1030 * Average source quality score (0-1 scale)
1031 * Distribution of high/medium/low quality sources
1032 * Publisher track record trends
1033
1034 **Evidence Coverage:**
1035 * Average number of sources per claim
1036 * Percentage of claims with ≥2 sources (EFCSN minimum)
1037 * Geographic diversity of sources
1038
1039 **Display:**
1040 {{code}}
1041 Evidence Quality:
1042
1043 Average Sources per Claim: 4.2
1044 Claims with ≥2 sources: 94% (EFCSN compliant)
1045
1046 Source Quality Distribution:
1047 High quality (>0.8): 48%
1048 Medium quality (0.5-0.8): 43%
1049 Low quality (<0.5): 9%
1050
1051 Geographic Diversity: 23 countries represented
1052 {{/code}}
1053
1054 **4. Contributor Consensus Metrics** (when human reviewers involved)
1055
1056 **Inter-Rater Reliability (IRR):**
1057 * **Calculation:** Cohen's Kappa or Fleiss' Kappa for multiple raters
1058 * **Scale:** 0-1 (higher is better)
1059 * **Interpretation:**
1060 * >0.8: Almost perfect agreement
1061 * 0.6-0.8: Substantial agreement
1062 * 0.4-0.6: Moderate agreement
1063 * <0.4: Poor agreement
1064 * **Target:** Maintain ≥0.7 (substantial agreement)
1065
1066 **Display:**
1067 {{code}}
1068 Contributor Consensus:
1069
1070 Inter-Rater Reliability (IRR): 0.73 (Substantial agreement)
1071 - Verdict agreement: 78%
1072 - Evidence quality agreement: 71%
1073 - Scenario structure agreement: 69%
1074
1075 Cases requiring moderator review: 12
1076 Moderator override rate: 8%
1077 {{/code}}
1078
1079 ---
1080
1081 ==== Quality Dashboard Implementation ====
1082
1083 **Dashboard Location:** `/quality-metrics`
1084
1085 **Update Frequency:**
1086 * **POC2:** Weekly manual updates
1087 * **Beta 0:** Daily automated updates
1088 * **V1.0:** Real-time metrics (updated hourly)
1089
1090 **Dashboard Sections:**
1091
1092 1. **Overview:** Key metrics at a glance
1093 2. **Verdict Quality:** TIGERScore trends and distributions
1094 3. **Evidence Analysis:** Source quality and coverage
1095 4. **AI Performance:** Hallucination rates, AlignScore
1096 5. **Human Oversight:** Contributor consensus, review rates
1097 6. **System Health:** Processing times, error rates, uptime
1098
1099 **Example Dashboard Layout:**
1100
1101 {{code}}
1102 ┌─────────────────────────────────────────────────────────────┐
1103 │ FactHarbor Quality Metrics Last updated: │
1104 │ Public Dashboard 2 hours ago │
1105 └─────────────────────────────────────────────────────────────┘
1106
1107 📊 KEY METRICS
1108 ─────────────────────────────────────────────────────────────
1109 TIGERScore (Verdict Quality): 84.2 ▲ (+2.1)
1110 AlignScore (Faithfulness): 0.87 ▼ (-0.02)
1111 Hallucination Rate: 4.2% ✓ (Target: <5%)
1112 Average Sources per Claim: 4.2 ▲ (+0.3)
1113
1114 📈 TRENDS (30 days)
1115 ─────────────────────────────────────────────────────────────
1116 [Graph: TIGERScore trending upward]
1117 [Graph: Hallucination rate declining]
1118 [Graph: Evidence quality stable]
1119
1120 ⚠️ IMPROVEMENT TARGETS
1121 ─────────────────────────────────────────────────────────────
1122 1. Reduce hallucination rate to <3% (Current: 4.2%)
1123 2. Increase TIGERScore average to >85 (Current: 84.2)
1124 3. Maintain IRR >0.75 (Current: 0.73)
1125
1126 📄 DETAILED REPORTS
1127 ─────────────────────────────────────────────────────────────
1128 • Monthly Quality Report (PDF)
1129 • Methodology Documentation
1130 • AKEL Performance Analysis
1131 • Contributor Agreement Analysis
1132
1133 {{/code}}
1134
1135 ---
1136
1137 ==== Continuous Improvement Feedback Loop ====
1138
1139 **How Metrics Inform AKEL Improvements:**
1140
1141 1. **Identify Weak Areas:**
1142 * Low TIGERScore → Review prompt engineering
1143 * High hallucination → Strengthen evidence grounding
1144 * Low IRR → Clarify evaluation criteria
1145
1146 2. **A/B Testing Integration:**
1147 * Test prompt variations
1148 * Measure impact on quality metrics
1149 * Deploy winners automatically
1150
1151 3. **Alert Thresholds:**
1152 * TIGERScore drops below 75 → Alert team
1153 * Hallucination rate exceeds 7% → Pause auto-publishing
1154 * IRR below 0.6 → Moderator training needed
1155
1156 4. **Monthly Quality Reviews:**
1157 * Analyze trends
1158 * Identify systematic issues
1159 * Plan prompt improvements
1160 * Update AKEL models
1161
1162 ---
1163
1164 ==== Metric Calculation Details ====
1165
1166 **TIGERScore Implementation:**
1167 * Reference: https://github.com/TIGER-AI-Lab/TIGERScore
1168 * Input: Generated verdict + reference verdict (from expert)
1169 * Output: 0-100 score across 5 dimensions
1170 * Requires: Test set of expert-reviewed claims (minimum 100)
1171
1172 **AlignScore Implementation:**
1173 * Reference: https://github.com/yuh-zha/AlignScore
1174 * Input: Generated verdict + source evidence text
1175 * Output: 0-1 faithfulness score
1176 * Calculation: Semantic alignment between claim and evidence
1177
1178 **Source Quality Scoring:**
1179 * Use existing source reliability database (e.g., NewsGuard, MBFC)
1180 * Factor in: Publication history, corrections record, transparency
1181 * Scale: 0-1 (weighted average across sources)
1182
1183 ---
1184
1185 ==== Integration Points ====
1186
1187 * **NFR11: AKEL Quality Assurance** - Metrics validate quality gate effectiveness
1188 * **FR49: A/B Testing** - Metrics measure test success
1189 * **FR11: Audit Trail** - Source of quality data
1190 * **NFR3: Transparency** - Public metrics build trust
1191
1192 **Acceptance Criteria:**
1193
1194 * ✅ All core metrics implemented and calculating correctly
1195 * ✅ Dashboard updates daily (Beta 0) or hourly (V1.0)
1196 * ✅ Alerts trigger when metrics degrade beyond thresholds
1197 * ✅ Monthly quality report auto-generates
1198 * ✅ Dashboard is publicly accessible (no login required)
1199 * ✅ Mobile-responsive dashboard design
1200 * ✅ Metrics inform quarterly AKEL improvement planning
1201
1202
1203
1204
1205 == 13. Requirements Traceability ==
1206
1207 For full traceability matrix showing which requirements fulfill which user needs, see:
1208
1209 * [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] - Section 8 includes comprehensive mapping tables
1210
1211 == 14. Related Pages ==
1212
1213 **Non-Functional Requirements (see Section 9):**
1214 * [[NFR11 — AKEL Quality Assurance Framework>>#NFR11]]
1215 * [[NFR12 — Security Controls>>#NFR12]]
1216 * [[NFR13 — Quality Metrics Transparency>>#NFR13]]
1217
1218 **Other Requirements:**
1219 * [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]
1220 * [[V1.0 Requirements>>FactHarbor.Specification.Requirements.V10.]]
1221 * [[Gap Analysis>>FactHarbor.Specification.Requirements.GapAnalysis]]
1222
1223 * **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need (drives these requirements)
1224 * [[Architecture>>FactHarbor.Specification.Architecture.WebHome]] - How requirements are implemented
1225 * [[Data Model>>FactHarbor.Specification.Data Model.WebHome]] - Data structures supporting requirements
1226 * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] - User interaction workflows
1227 * [[AKEL>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements
1228 * [[Global Rules>>FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]]
1229 * [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
1230
1231 = V0.9.70 Additional Requirements =
1232
1233 == Functional Requirements (Additional) ==
1234
1235 === FR44: ClaimReview Schema Implementation ===
1236
1237 **Fulfills:** UN-13 (Cite FactHarbor Verdicts), UN-14 (API Access for Integration), UN-26 (Search Engine Visibility)
1238
1239 **Phase:** V1.0
1240
1241 **Purpose:** Generate valid ClaimReview structured data for every published analysis to enable Google/Bing search visibility and fact-check discovery.
1242
1243 **Specification:**
1244
1245 ==== Component: Schema.org Markup Generator ====
1246
1247 FactHarbor must generate valid ClaimReview structured data following Schema.org specifications for every published claim analysis.
1248
1249 **Required JSON-LD Schema:**
1250
1251 {{code language="json"}}
1252 {
1253 "@context": "https://schema.org",
1254 "@type": "ClaimReview",
1255 "datePublished": "YYYY-MM-DD",
1256 "url": "https://factharbor.org/claims/{claim_id}",
1257 "claimReviewed": "The exact claim text",
1258 "author": {
1259 "@type": "Organization",
1260 "name": "FactHarbor",
1261 "url": "https://factharbor.org"
1262 },
1263 "reviewRating": {
1264 "@type": "Rating",
1265 "ratingValue": "1-5",
1266 "bestRating": "5",
1267 "worstRating": "1",
1268 "alternateName": "FactHarbor likelihood score"
1269 },
1270 "itemReviewed": {
1271 "@type": "Claim",
1272 "author": {
1273 "@type": "Person",
1274 "name": "Claim author if known"
1275 },
1276 "datePublished": "YYYY-MM-DD if known",
1277 "appearance": {
1278 "@type": "CreativeWork",
1279 "url": "Original claim URL if from article"
1280 }
1281 }
1282 }
1283 {{/code}}
1284
1285 **FactHarbor-Specific Mapping:**
1286
1287 **Likelihood Score to Rating Scale:**
1288 * 80-100% likelihood → 5 (Highly Supported)
1289 * 60-79% likelihood → 4 (Supported)
1290 * 40-59% likelihood → 3 (Mixed/Uncertain)
1291 * 20-39% likelihood → 2 (Questionable)
1292 * 0-19% likelihood → 1 (Refuted)
1293
1294 **Multiple Scenarios Handling:**
1295 * If claim has multiple scenarios with different verdicts, generate **separate ClaimReview** for each scenario
1296 * Add `disambiguatingDescription` field explaining scenario context
1297 * Example: "Scenario: If interpreted as referring to 2023 data..."
1298
1299 ==== Implementation Requirements ====
1300
1301 1. **Auto-generate** on claim publication
1302 2. **Embed** in HTML `<head>` section as JSON-LD script
1303 3. **Validate** against Schema.org validator before publishing
1304 4. **Submit** to Google Search Console for indexing
1305 5. **Update** automatically when verdict changes (integrate with FR8: Time Evolution)
1306
1307 ==== Integration Points ====
1308
1309 * **FR7: Automated Verdicts** - Source of rating data and claim text
1310 * **FR8: Time Evolution** - Triggers schema updates when verdicts change
1311 * **FR11: Audit Trail** - Logs all schema generation and update events
1312
1313 ==== Resources ====
1314
1315 * ClaimReview Project: https://www.claimreviewproject.com
1316 * Schema.org ClaimReview: https://schema.org/ClaimReview
1317 * Google Fact Check Guidelines: https://developers.google.com/search/docs/appearance/fact-check
1318
1319 **Acceptance Criteria:**
1320
1321 * ✅ Passes Google Structured Data Testing Tool
1322 * ✅ Appears in Google Fact Check Explorer within 48 hours of publication
1323 * ✅ Valid JSON-LD syntax (no errors)
1324 * ✅ All required fields populated with correct data types
1325 * ✅ Handles multi-scenario claims correctly (separate ClaimReview per scenario)
1326
1327
1328 === FR45: User Corrections Notification System ===
1329
1330 **Fulfills:** IFCN Principle 5 (Open & Honest Corrections), EFCSN compliance
1331
1332 **Phase:** Beta 0 (basic), V1.0 (complete) **BLOCKER**
1333
1334 **Purpose:** When any claim analysis is corrected, notify users who previously viewed the claim to maintain transparency and build trust.
1335
1336 **Specification:**
1337
1338 ==== Component: Corrections Visibility Framework ====
1339
1340 **Correction Types:**
1341
1342 1. **Major Correction:** Verdict changes category (e.g., "Supported" → "Refuted")
1343 2. **Significant Correction:** Likelihood score changes >20%
1344 3. **Minor Correction:** Evidence additions, source quality updates
1345 4. **Scenario Addition:** New scenario added to existing claim
1346
1347 ==== Notification Mechanisms ====
1348
1349 **1. In-Page Banner:**
1350
1351 Display prominent banner on claim page:
1352
1353 {{code}}
1354 [!] CORRECTION NOTICE
1355 This analysis was updated on [DATE]. [View what changed] [Dismiss]
1356
1357 Major changes:
1358 • Verdict changed from "Likely True (75%)" to "Uncertain (45%)"
1359 • New contradicting evidence added from [Source]
1360 • Scenario 2 updated with additional context
1361
1362 [See full correction log]
1363 {{/code}}
1364
1365 **2. Correction Log Page:**
1366
1367 * Public changelog at `/claims/{id}/corrections`
1368 * Displays for each correction:
1369 * Date/time of correction
1370 * What changed (before/after comparison)
1371 * Why changed (reason if provided)
1372 * Who made change (AKEL auto-update vs. contributor override)
1373
1374 **3. Email Notifications (opt-in):**
1375
1376 * Send to users who bookmarked or shared the claim
1377 * Subject: "FactHarbor Correction: [Claim title]"
1378 * Include summary of changes
1379 * Link to updated analysis
1380
1381 **4. RSS/API Feed:**
1382
1383 * Corrections feed at `/corrections.rss`
1384 * API endpoint: `GET /api/corrections?since={timestamp}`
1385 * Enables external monitoring by journalists and researchers
1386
1387 ==== Display Rules ====
1388
1389 * Show banner on **ALL pages** displaying the claim (search results, related claims, embeddings)
1390 * Banner persists for **30 days** after correction
1391 * **"Corrections" count badge** on claim card
1392 * **Timestamp** on every verdict: "Last updated: [datetime]"
1393
1394 ==== IFCN Compliance Requirements ====
1395
1396 * Corrections policy published at `/corrections-policy`
1397 * User can report suspected errors via `/report-error/{claim_id}`
1398 * Link to IFCN complaint process (if FactHarbor becomes signatory)
1399 * **Scrupulous transparency:** Never silently edit analyses
1400
1401 ==== Integration Points ====
1402
1403 * **FR8: Time Evolution** - Triggers corrections when verdicts change
1404 * **FR11: Audit Trail** - Source of correction data and change history
1405 * **NFR3: Transparency** - Public correction log demonstrates commitment
1406
1407 **Acceptance Criteria:**
1408
1409 * ✅ Banner appears within 60 seconds of correction
1410 * ✅ Correction log is permanent and publicly accessible
1411 * ✅ Email notifications deliver within 5 minutes
1412 * ✅ RSS feed updates in real-time
1413 * ✅ Mobile-responsive banner design
1414 * ✅ Accessible (screen reader compatible)
1415
1416
1417 === FR46: Image Verification System ===
1418
1419 **Fulfills:** UN-27 (Visual Claim Verification)
1420
1421 **Phase:** Beta 0 (basic), V1.0 (extended)
1422
1423 **Purpose:** Verify authenticity and context of images shared with claims to detect manipulation, misattribution, and out-of-context usage.
1424
1425 **Specification:**
1426
1427 ==== Component: Multi-Method Image Verification ====
1428
1429 **Method 1: Reverse Image Search**
1430
1431 **Purpose:** Find earlier uses of the image to verify context
1432
1433 **Implementation:**
1434 * Integrate APIs:
1435 * **Google Vision AI** (reverse search)
1436 * **TinEye** (oldest known uses)
1437 * **Bing Visual Search** (broad coverage)
1438
1439 **Process:**
1440 1. Extract image from claim or user upload
1441 2. Query multiple reverse search services
1442 3. Analyze results for:
1443 * Earliest known publication
1444 * Original context (what was it really showing?)
1445 * Publication timeline
1446 * Geographic spread
1447
1448 **Output:**
1449 {{code}}
1450 Reverse Image Search Results:
1451
1452 Earliest known use: 2019-03-15 (5 years before claim)
1453 Original context: "Photo from 2019 flooding in Mumbai"
1454 This claim uses it for: "2024 hurricane damage in Florida"
1455
1456 ⚠️ Image is OUT OF CONTEXT
1457
1458 Found in 47 other articles:
1459 • 2019-03-15: Mumbai floods (original)
1460 • 2020-07-22: Bangladesh monsoon
1461 • 2024-10-15: Current claim (misattributed)
1462
1463 [View full timeline]
1464 {{/code}}
1465
1466 ---
1467
1468 **Method 2: AI Manipulation Detection**
1469
1470 **Purpose:** Detect deepfakes, face swaps, and digital alterations
1471
1472 **Implementation:**
1473 * Integrate detection services:
1474 * **Sensity AI** (deepfake detection)
1475 * **Reality Defender** (multimodal analysis)
1476 * **AWS Rekognition** (face detection inconsistencies)
1477
1478 **Detection Categories:**
1479 1. **Face Manipulation:**
1480 * Deepfake face swaps
1481 * Expression manipulation
1482 * Identity replacement
1483
1484 2. **Image Manipulation:**
1485 * Copy-paste artifacts
1486 * Clone stamp detection
1487 * Content-aware fill detection
1488 * JPEG compression inconsistencies
1489
1490 3. **AI Generation:**
1491 * Detect fully AI-generated images
1492 * Identify generation artifacts
1493 * Check for model signatures
1494
1495 **Confidence Scoring:**
1496 * **HIGH (80-100%):** Strong evidence of manipulation
1497 * **MEDIUM (50-79%):** Suspicious artifacts detected
1498 * **LOW (0-49%):** Minor inconsistencies or inconclusive
1499
1500 **Output:**
1501 {{code}}
1502 Manipulation Analysis:
1503
1504 Face Manipulation: LOW RISK (12%)
1505 Image Editing: MEDIUM RISK (64%)
1506 • Clone stamp artifacts detected in sky region
1507 • JPEG compression inconsistent between objects
1508
1509 AI Generation: LOW RISK (8%)
1510
1511 ⚠️ Possible manipulation detected. Manual review recommended.
1512 {{/code}}
1513
1514 ---
1515
1516 **Method 3: Metadata Analysis (EXIF)**
1517
1518 **Purpose:** Extract technical details that may reveal manipulation or misattribution
1519
1520 **Extracted Data:**
1521 * **Camera/Device:** Make, model, software
1522 * **Timestamps:** Original date, modification dates
1523 * **Location:** GPS coordinates (if present)
1524 * **Editing History:** Software used, edit count
1525 * **File Properties:** Resolution, compression, format conversions
1526
1527 **Red Flags:**
1528 * Metadata completely stripped (suspicious)
1529 * Timestamp conflicts with claimed date
1530 * GPS location conflicts with claimed location
1531 * Multiple edit rounds (hiding something?)
1532 * Creation date after modification date (impossible)
1533
1534 **Output:**
1535 {{code}}
1536 Image Metadata:
1537
1538 Camera: iPhone 14 Pro
1539 Original date: 2023-08-12 14:32:15
1540 Location: 40.7128°N, 74.0060°W (New York City)
1541 Modified: 2024-10-15 08:45:22
1542 Software: Adobe Photoshop 2024
1543
1544 ⚠️ Location conflicts with claim
1545 Claim says: "Taken in Los Angeles"
1546 EXIF says: New York City
1547
1548 ⚠️ Edited 14 months after capture
1549 {{/code}}
1550
1551 ---
1552
1553 ==== Verification Workflow ====
1554
1555 **Automatic Triggers:**
1556 1. User submits claim with image
1557 2. Article being analyzed contains images
1558 3. Social media post includes photos
1559
1560 **Process:**
1561 1. Extract images from content
1562 2. Run all 3 verification methods in parallel
1563 3. Aggregate results into confidence score
1564 4. Generate human-readable summary
1565 5. Display prominently in analysis
1566
1567 **Display Integration:**
1568
1569 Show image verification panel in claim analysis:
1570
1571 {{code}}
1572 📷 IMAGE VERIFICATION
1573
1574 [Image thumbnail]
1575
1576 ✅ Reverse Search: Original context verified
1577 ⚠️ Manipulation: Possible editing detected (64% confidence)
1578 ✅ Metadata: Consistent with claim details
1579
1580 Overall Assessment: CAUTION ADVISED
1581 This image may have been edited. Original context appears accurate.
1582
1583 [View detailed analysis]
1584 {{/code}}
1585
1586 ==== Integration Points ====
1587
1588 * **FR7: Automated Verdicts** - Image verification affects claim credibility
1589 * **FR4: Analysis Summary** - Image findings included in summary
1590 * **UN-27: Visual Claim Verification** - Direct fulfillment
1591
1592 ==== Cost Considerations ====
1593
1594 **API Costs (estimated per image):**
1595 * Google Vision AI: $0.001-0.003
1596 * TinEye: $0.02 (commercial API)
1597 * Sensity AI: $0.05-0.10
1598 * AWS Rekognition: $0.001-0.002
1599
1600 **Total per image:** ~$0.07-0.15
1601
1602 **Mitigation Strategies:**
1603 * Cache results for duplicate images
1604 * Use free tier quotas where available
1605 * Prioritize higher-value claims for deep analysis
1606 * Offer premium verification as paid tier
1607
1608 **Acceptance Criteria:**
1609
1610 * ✅ Reverse image search finds original sources
1611 * ✅ Manipulation detection accuracy >80% on test dataset
1612 * ✅ EXIF extraction works for major image formats (JPEG, PNG, HEIC)
1613 * ✅ Results display within 10 seconds
1614 * ✅ Mobile-friendly image comparison interface
1615 * ✅ False positive rate <15%
1616
1617
1618 === FR47: Archive.org Integration ===
1619
1620 **Priority:** CRITICAL
1621 **Fulfills:** Evidence persistence, FR5 (Evidence linking)
1622 **Phase:** V1.0
1623
1624 **Purpose:** Ensure evidence remains accessible even if original sources are deleted.
1625
1626 **Specification:**
1627
1628 **Automatic Archiving:**
1629
1630 When AKEL links evidence:
1631 1. Check if URL already archived (Wayback Machine API)
1632 2. If not, submit for archiving (Save Page Now API)
1633 3. Store both original URL and archive URL
1634 4. Display both to users
1635
1636 **Archive Display:**
1637
1638 {{code}}
1639 Evidence Source: [Original URL]
1640 Archived: [Archive.org URL] (Captured: [date])
1641
1642 [View Original] [View Archive]
1643 {{/code}}
1644
1645 **Fallback Logic:**
1646
1647 * If original URL unavailable → Auto-redirect to archive
1648 * If archive unavailable → Display warning
1649 * If both unavailable → Flag for manual review
1650
1651 **API Integration:**
1652
1653 * Use Wayback Machine Availability API
1654 * Use Save Page Now API (SPNv2)
1655 * Rate limiting: 15 requests/minute (Wayback limit)
1656
1657 **Acceptance Criteria:**
1658
1659 * ✅ All evidence URLs auto-archived
1660 * ✅ Archive links displayed to users
1661 * ✅ Fallback to archive if original unavailable
1662 * ✅ API rate limits respected
1663 * ✅ Archive status visible in evidence display
1664
1665
1666 == Category 4: Community Safety ===== FR48: Contributor Safety Framework ===
1667
1668 **Priority:** CRITICAL
1669 **Fulfills:** UN-28 (Safe contribution environment)
1670 **Phase:** V1.0
1671
1672 **Purpose:** Protect contributors from harassment, doxxing, and coordinated attacks.
1673
1674 **Specification:**
1675
1676 **1. Privacy Protection:**
1677
1678 * **Optional Pseudonymity:** Contributors can use pseudonyms
1679 * **Email Privacy:** Emails never displayed publicly
1680 * **Profile Privacy:** Contributors control what's public
1681 * **IP Logging:** Only for abuse prevention, not public
1682
1683 **2. Harassment Prevention:**
1684
1685 * **Automated Toxicity Detection:** Flag abusive comments
1686 * **Personal Information Detection:** Auto-block doxxing attempts
1687 * **Coordinated Attack Detection:** Identify brigading patterns
1688 * **Rapid Response:** Moderator alerts for harassment
1689
1690 **3. Safety Features:**
1691
1692 * **Block Users:** Contributors can block harassers
1693 * **Private Contributions:** Option to contribute anonymously
1694 * **Report Harassment:** One-click harassment reporting
1695 * **Safety Resources:** Links to support resources
1696
1697 **4. Moderator Tools:**
1698
1699 * **Quick Ban:** Immediately block abusers
1700 * **Pattern Detection:** Identify coordinated attacks
1701 * **Appeal Process:** Fair review of moderation actions
1702 * **Escalation:** Serious threats escalated to authorities
1703
1704 **5. Trusted Contributor Protection:**
1705
1706 * **Enhanced Privacy:** Additional protection for high-profile contributors
1707 * **Verification:** Optional identity verification (not public)
1708 * **Legal Support:** Resources for contributors facing legal threats
1709
1710 **Acceptance Criteria:**
1711
1712 * ✅ Pseudonyms supported
1713 * ✅ Toxicity detection active
1714 * ✅ Doxxing auto-blocked
1715 * ✅ Harassment reporting functional
1716 * ✅ Moderator tools implemented
1717 * ✅ Safety policy published
1718
1719
1720 == Category 5: Continuous Improvement ===== FR49: A/B Testing Framework ===
1721
1722 **Priority:** CRITICAL
1723 **Fulfills:** Continuous system improvement
1724 **Phase:** V1.0
1725
1726 **Purpose:** Test and measure improvements to AKEL prompts, algorithms, and workflows.
1727
1728 **Specification:**
1729
1730 **Test Capabilities:**
1731
1732 1. **Prompt Variations:**
1733 * Test different claim extraction prompts
1734 * Test different verdict generation prompts
1735 * Measure: Accuracy, clarity, completeness
1736
1737 2. **Algorithm Variations:**
1738 * Test different source scoring algorithms
1739 * Test different confidence calculations
1740 * Measure: Audit accuracy, user satisfaction
1741
1742 3. **Workflow Variations:**
1743 * Test different quality gate thresholds
1744 * Test different risk tier assignments
1745 * Measure: Publication rate, quality scores
1746
1747 **Implementation:**
1748
1749 * **Traffic Split:** 50/50 or 90/10 splits
1750 * **Randomization:** Consistent per claim (not per user)
1751 * **Metrics Collection:** Automatic for all variants
1752 * **Statistical Significance:** Minimum sample size calculation
1753 * **Rollout:** Winner promoted to 100% traffic
1754
1755 **A/B Test Workflow:**
1756
1757 {{code}}
1758 1. Hypothesis: "New prompt improves claim extraction"
1759 2. Design test: Control vs. Variant
1760 3. Define metrics: Extraction accuracy, completeness
1761 4. Run test: 7-14 days, minimum 100 claims each
1762 5. Analyze results: Statistical significance?
1763 6. Decision: Deploy winner or iterate
1764 {{/code}}
1765
1766 **Acceptance Criteria:**
1767
1768 * ✅ A/B testing framework implemented
1769 * ✅ Can test prompt variations
1770 * ✅ Can test algorithm variations
1771 * ✅ Metrics automatically collected
1772 * ✅ Statistical significance calculated
1773 * ✅ Results inform system improvements
1774
1775
1776 === FR54: Evidence Deduplication ===
1777
1778 **Priority:** CRITICAL (POC2/Beta)
1779 **Fulfills:** Accurate evidence counting, quality metrics
1780 **Phase:** POC2, Beta 0, V1.0
1781
1782 **Purpose:** Avoid counting the same source multiple times when it appears in different forms.
1783
1784 **Specification:**
1785
1786 **Deduplication Logic:**
1787
1788 1. **URL Normalization:**
1789 * Remove tracking parameters (?utm_source=...)
1790 * Normalize http/https
1791 * Normalize www/non-www
1792 * Handle redirects
1793
1794 2. **Content Similarity:**
1795 * If two sources have >90% text similarity → Same source
1796 * If one is subset of other → Same source
1797 * Use fuzzy matching for minor differences
1798
1799 3. **Cross-Domain Syndication:**
1800 * Detect wire service content (AP, Reuters)
1801 * Mark as single source if syndicated
1802 * Count original publication only
1803
1804 **Display:**
1805
1806 {{code}}
1807 Evidence Sources (3 unique, 5 total):
1808
1809 1. Original Article (NYTimes)
1810 - Also appeared in: WashPost, Guardian (syndicated)
1811
1812 2. Research Paper (Nature)
1813
1814 3. Official Statement (WHO)
1815 {{/code}}
1816
1817 **Acceptance Criteria:**
1818
1819 * ✅ URL normalization works
1820 * ✅ Content similarity detected
1821 * ✅ Syndicated content identified
1822 * ✅ Unique vs. total counts accurate
1823 * ✅ Improves evidence quality metrics
1824
1825
1826 == Additional Requirements (Lower Priority) ===== FR50: OSINT Toolkit Integration ===
1827
1828
1829
1830 **Fulfills:** Advanced media verification
1831 **Phase:** V1.1
1832
1833 **Purpose:** Integrate open-source intelligence tools for advanced verification.
1834
1835 **Tools to Integrate:**
1836 * InVID/WeVerify (video verification)
1837 * Bellingcat toolkit
1838 * Additional TBD based on V1.0 learnings
1839
1840 === FR51: Video Verification System ===
1841
1842
1843
1844 **Fulfills:** UN-27 (Visual claims), advanced media verification
1845 **Phase:** V1.1
1846
1847 **Purpose:** Verify video-based claims.
1848
1849 **Specification:**
1850 * Keyframe extraction
1851 * Reverse video search
1852 * Deepfake detection (AI-powered)
1853 * Metadata analysis
1854 * Acoustic signature analysis
1855
1856 === FR52: Interactive Detection Training ===
1857
1858
1859
1860 **Fulfills:** Media literacy education
1861 **Phase:** V1.5
1862
1863 **Purpose:** Teach users to identify misinformation.
1864
1865 **Specification:**
1866 * Interactive tutorials
1867 * Practice exercises
1868 * Detection quizzes
1869 * Gamification elements
1870
1871 === FR53: Cross-Organizational Sharing ===
1872
1873
1874
1875 **Fulfills:** Collaboration with other fact-checkers
1876 **Phase:** V1.5
1877
1878 **Purpose:** Share findings with IFCN/EFCSN members.
1879
1880 **Specification:**
1881 * API for fact-checking organizations
1882 * Structured data exchange
1883 * Privacy controls
1884 * Attribution requirements
1885
1886
1887 == Summary ==
1888
1889 **V1.0 Critical Requirements (Must Have):**
1890
1891 * FR44: ClaimReview Schema ✅
1892 * FR45: Corrections Notification ✅
1893 * FR46: Image Verification ✅
1894 * FR47: Archive.org Integration ✅
1895 * FR48: Contributor Safety ✅
1896 * FR49: A/B Testing ✅
1897 * FR54: Evidence Deduplication ✅
1898 * NFR11: Quality Assurance Framework ✅
1899 * NFR12: Security Controls ✅
1900 * NFR13: Quality Metrics Dashboard ✅
1901
1902 **V1.1+ (Future):**
1903
1904 * FR50: OSINT Integration
1905 * FR51: Video Verification
1906 * FR52: Detection Training
1907 * FR53: Cross-Org Sharing
1908
1909
1910 **Total:** 11 critical requirements for V1.0
1911
1912 === FR54: Evidence Deduplication ===
1913
1914
1915
1916 **Fulfills:** Accurate evidence counting, quality metrics
1917 **Phase:** POC2, Beta 0, V1.0
1918
1919 **Purpose:** Avoid counting the same source multiple times when it appears in different forms.
1920
1921 **Specification:**
1922
1923 **Deduplication Logic:**
1924
1925 1. **URL Normalization:**
1926 * Remove tracking parameters (?utm_source=...)
1927 * Normalize http/https
1928 * Normalize www/non-www
1929 * Handle redirects
1930
1931 2. **Content Similarity:**
1932 * If two sources have >90% text similarity → Same source
1933 * If one is subset of other → Same source
1934 * Use fuzzy matching for minor differences
1935
1936 3. **Cross-Domain Syndication:**
1937 * Detect wire service content (AP, Reuters)
1938 * Mark as single source if syndicated
1939 * Count original publication only
1940
1941 **Display:**
1942
1943 {{code}}
1944 Evidence Sources (3 unique, 5 total):
1945
1946 1. Original Article (NYTimes)
1947 - Also appeared in: WashPost, Guardian (syndicated)
1948
1949 2. Research Paper (Nature)
1950
1951 3. Official Statement (WHO)
1952 {{/code}}
1953
1954 **Acceptance Criteria:**
1955
1956 * ✅ URL normalization works
1957 * ✅ Content similarity detected
1958 * ✅ Syndicated content identified
1959 * ✅ Unique vs. total counts accurate
1960 * ✅ Improves evidence quality metrics
1961
1962
1963 == Additional Requirements (Lower Priority) ===== FR7: Automated Verdicts (Enhanced with Quality Gates) ===
1964
1965 **POC1+ Enhancement:**
1966
1967 After AKEL generates verdict, it passes through quality gates:
1968
1969 {{code}}
1970 Workflow:
1971 1. Extract claims
1972 ↓
1973 2. [GATE 1] Validate fact-checkable
1974 ↓
1975 3. Generate scenarios
1976 ↓
1977 4. Generate verdicts
1978 ↓
1979 5. [GATE 4] Validate confidence
1980 ↓
1981 6. Display to user
1982 {{/code}}
1983
1984 **Updated Verdict States:**
1985 * PUBLISHED
1986 * INSUFFICIENT_EVIDENCE
1987 * NON_FACTUAL_CLAIM
1988 * PROCESSING
1989 * ERROR
1990
1991 === FR4: Analysis Summary (Enhanced with Quality Metadata) ===
1992
1993 **POC1+ Enhancement:**
1994
1995 Display quality indicators:
1996
1997 {{code}}
1998 Analysis Summary:
1999 Verifiable Claims: 3/5
2000 High Confidence Verdicts: 1
2001 Medium Confidence: 2
2002 Evidence Sources: 12
2003 Avg Source Quality: 0.73
2004 Quality Score: 8.5/10
2005 {{/code}}