Changes for page Requirements

Last modified by Robert Schaub on 2026/02/08 21:32

From version 1.1
edited by Robert Schaub
on 2025/12/18 12:03
Change comment: Imported from XAR
To version 7.5
edited by Robert Schaub
on 2026/01/20 20:26
Change comment: Renamed back-links.

Summary

Details

Page properties
Content
... ... @@ -1,22 +1,61 @@
1 1  = Requirements =
2 -This page defines **Roles**, **Content States**, **Rules**, and **System Principles** for FactHarbor.
2 +
3 +{{info}}
4 +**Phase Assignments:** See [[Requirements Roadmap Matrix>>Archive.FactHarbor.Roadmap.Requirements-Roadmap-Matrix.WebHome]] for which requirements are implemented in which phases.
5 +{{/info}}
6 +
7 +**This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.**
8 +
3 3  **Core Philosophy:** Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically.
10 +
11 +== Navigation ==
12 +
13 +* **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need from FactHarbor (drives these requirements)
14 +* **This page** - How we fulfill those needs through system design
15 +
16 +(% class="box infomessage" %)
17 +(((
18 +**How to read this page:**
19 +
20 +1. **User Needs drive Requirements**: See [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] for what users need
21 +2. **Requirements define implementation**: This page shows how we fulfill those needs
22 +3. **Functional Requirements (FR)**: Specific features and capabilities
23 +4. **Non-Functional Requirements (NFR)**: Quality attributes (performance, security, etc.)
24 +
25 +Each requirement references which User Needs it fulfills.
26 +)))
27 +
4 4  == 1. Roles ==
29 +
30 +**Fulfills**: UN-12 (Submit claims), UN-13 (Cite verdicts), UN-14 (API access)
31 +
5 5  FactHarbor uses three simple roles plus a reputation system.
33 +
6 6  === 1.1 Reader ===
35 +
7 7  **Who**: Anyone (no login required)
37 +
8 8  **Can**:
39 +
9 9  * Browse and search claims
10 10  * View scenarios, evidence, verdicts, and confidence scores
11 11  * Flag issues or errors
12 12  * Use filters, search, and visualization tools
13 13  * Submit claims automatically (new claims added if not duplicates)
45 +
14 14  **Cannot**:
47 +
15 15  * Modify content
16 16  * Access edit history details
50 +
51 +**User Needs served**: UN-1 (Trust assessment), UN-2 (Claim verification), UN-3 (Article summary with FactHarbor analysis summary), UN-4 (Social media fact-checking), UN-5 (Source tracing), UN-7 (Evidence transparency), UN-8 (Understanding disagreement), UN-12 (Submit claims), UN-17 (In-article highlighting)
52 +
17 17  === 1.2 Contributor ===
54 +
18 18  **Who**: Registered users (earns reputation through contributions)
56 +
19 19  **Can**:
58 +
20 20  * Everything a Reader can do
21 21  * Edit claims, evidence, and scenarios
22 22  * Add sources and citations
... ... @@ -23,18 +23,28 @@
23 23  * Suggest improvements to AI-generated content
24 24  * Participate in discussions
25 25  * Earn reputation points for quality contributions
65 +
26 26  **Reputation System**:
67 +
27 27  * New contributors: Limited edit privileges
28 28  * Established contributors (established reputation): Full edit access
29 29  * Trusted contributors (substantial reputation): Can approve certain changes
30 30  * Reputation earned through: Accepted edits, helpful flags, quality contributions
31 31  * Reputation lost through: Reverted edits, invalid flags, abuse
73 +
32 32  **Cannot**:
75 +
33 33  * Delete or hide content (only moderators)
34 34  * Override moderation decisions
78 +
79 +**User Needs served**: UN-13 (Cite and contribute)
80 +
35 35  === 1.3 Moderator ===
82 +
36 36  **Who**: Trusted community members with proven track record, appointed by governance board
84 +
37 37  **Can**:
86 +
38 38  * Review flagged content
39 39  * Hide harmful or abusive content
40 40  * Resolve disputes between contributors
... ... @@ -41,34 +41,56 @@
41 41  * Issue warnings or temporary bans
42 42  * Make final decisions on content disputes
43 43  * Access full audit logs
93 +
44 44  **Cannot**:
95 +
45 45  * Change governance rules
46 46  * Permanently ban users without board approval
47 47  * Override technical quality gates
99 +
48 48  **Note**: Small team (3-5 initially), supported by automated moderation tools.
101 +
49 49  === 1.4 Domain Trusted Contributors (Optional, Task-Specific) ===
103 +
50 50  **Who**: Subject matter specialists invited for specific high-stakes disputes
105 +
51 51  **Not a permanent role**: Contacted externally when needed for contested claims in their domain
107 +
52 52  **When used**:
109 +
53 53  * Medical claims with life/safety implications
54 54  * Legal interpretations with significant impact
55 55  * Scientific claims with high controversy
56 56  * Technical claims requiring specialized knowledge
114 +
57 57  **Process**:
116 +
58 58  * Moderator identifies need for expert input
59 59  * Contact expert externally (don't require them to be users)
60 60  * Trusted Contributor provides written opinion with sources
61 61  * Opinion added to claim record
62 62  * Trusted Contributor acknowledged in claim
122 +
123 +**User Needs served**: UN-16 (Expert validation status)
124 +
63 63  == 2. Content States ==
126 +
127 +**Fulfills**: UN-1 (Trust indicators), UN-16 (Review status transparency)
128 +
64 64  FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping.
130 +
65 65  === 2.1 Published ===
132 +
66 66  **Status**: Visible to all users
134 +
67 67  **Includes**:
136 +
68 68  * AI-generated analyses (default state)
69 69  * User-contributed content
70 70  * Edited/improved content
140 +
71 71  **Quality Indicators** (displayed with content):
142 +
72 72  * **Confidence Score**: 0-100% (AI's confidence in analysis)
73 73  * **Source Quality Score**: 0-100% (based on source track record)
74 74  * **Controversy Flag**: If high dispute/edit activity
... ... @@ -75,14 +75,23 @@
75 75  * **Completeness Score**: % of expected fields filled
76 76  * **Last Updated**: Date of most recent change
77 77  * **Edit Count**: Number of revisions
149 +* **Review Status**: AI-generated / Human-reviewed / Expert-validated
150 +
78 78  **Automatic Warnings**:
152 +
79 79  * Confidence < 60%: "Low confidence - use caution"
80 80  * Source quality < 40%: "Sources may be unreliable"
81 81  * High controversy: "Disputed - multiple interpretations exist"
82 82  * Medical/Legal/Safety domain: "Seek professional advice"
157 +
158 +**User Needs served**: UN-1 (Trust score), UN-9 (Methodology transparency), ~~UN-15 (Evolution timeline - Deferred)~~, UN-16 (Review status)
159 +
83 83  === 2.2 Hidden ===
161 +
84 84  **Status**: Not visible to regular users (only to moderators)
163 +
85 85  **Reasons**:
165 +
86 86  * Spam or advertising
87 87  * Personal attacks or harassment
88 88  * Illegal content
... ... @@ -89,22 +89,32 @@
89 89  * Privacy violations
90 90  * Deliberate misinformation (verified)
91 91  * Abuse or harmful content
172 +
92 92  **Process**:
174 +
93 93  * Automated detection flags for moderator review
94 94  * Moderator confirms and hides
95 95  * Original author notified with reason
96 96  * Can appeal to board if disputes moderator decision
179 +
97 97  **Note**: Content is hidden, not deleted (for audit trail)
181 +
98 98  == 3. Contribution Rules ==
183 +
99 99  === 3.1 All Contributors Must ===
185 +
100 100  * Provide sources for factual claims
101 101  * Use clear, neutral language in FactHarbor's own summaries
102 102  * Respect others and maintain civil discourse
103 103  * Accept community feedback constructively
104 104  * Focus on improving quality, not protecting ego
191 +
105 105  === 3.2 AKEL (AI System) ===
193 +
106 106  **AKEL is the primary system**. Human contributions supplement and train AKEL.
195 +
107 107  **AKEL Must**:
197 +
108 108  * Mark all outputs as AI-generated
109 109  * Display confidence scores prominently
110 110  * Provide source citations
... ... @@ -111,86 +111,139 @@
111 111  * Flag uncertainty clearly
112 112  * Identify contradictions in evidence
113 113  * Learn from human corrections
204 +
114 114  **When AKEL Makes Errors**:
206 +
115 115  1. Capture the error pattern (what, why, how common)
116 116  2. Improve the system (better prompt, model, validation)
117 117  3. Re-process affected claims automatically
118 118  4. Measure improvement (did quality increase?)
211 +
119 119  **Human Role**: Train AKEL through corrections, not replace AKEL
213 +
120 120  === 3.3 Contributors Should ===
215 +
121 121  * Improve clarity and structure
122 122  * Add missing sources
123 123  * Flag errors for system improvement
124 124  * Suggest better ways to present information
125 125  * Participate in quality discussions
221 +
126 126  === 3.4 Moderators Must ===
223 +
127 127  * Be impartial
128 128  * Document moderation decisions
129 129  * Respond to appeals promptly
130 130  * Use automated tools to scale efforts
131 131  * Focus on abuse/harm, not routine quality control
229 +
132 132  == 4. Quality Standards ==
231 +
232 +**Fulfills**: UN-5 (Source reliability), UN-6 (Publisher track records), UN-7 (Evidence transparency), UN-9 (Methodology transparency)
233 +
133 133  === 4.1 Source Requirements ===
235 +
134 134  **Track Record Over Credentials**:
237 +
135 135  * Sources evaluated by historical accuracy
136 136  * Correction policy matters
137 137  * Independence from conflicts of interest
138 138  * Methodology transparency
242 +
139 139  **Source Quality Database**:
244 +
140 140  * Automated tracking of source accuracy
141 141  * Correction frequency
142 142  * Reliability score (updated continuously)
143 143  * Users can see source track record
249 +
144 144  **No automatic trust** for government, academia, or media - all evaluated by track record.
251 +
252 +**User Needs served**: UN-5 (Source provenance), UN-6 (Publisher reliability)
253 +
145 145  === 4.2 Claim Requirements ===
255 +
146 146  * Clear subject and assertion
147 147  * Verifiable with available information
148 148  * Sourced (or explicitly marked as needing sources)
149 149  * Neutral language in FactHarbor summaries
150 150  * Appropriate context provided
261 +
262 +**User Needs served**: UN-2 (Claim extraction and verification)
263 +
151 151  === 4.3 Evidence Requirements ===
265 +
152 152  * Publicly accessible (or explain why not)
153 153  * Properly cited with attribution
154 154  * Relevant to claim being evaluated
155 155  * Original source preferred over secondary
270 +
271 +**User Needs served**: UN-7 (Evidence transparency)
272 +
156 156  === 4.4 Confidence Scoring ===
274 +
157 157  **Automated confidence calculation based on**:
276 +
158 158  * Source quality scores
159 159  * Evidence consistency
160 160  * Contradiction detection
161 161  * Completeness of analysis
162 162  * Historical accuracy of similar claims
282 +
163 163  **Thresholds**:
284 +
164 164  * < 40%: Too low to publish (needs improvement)
165 165  * 40-60%: Published with "Low confidence" warning
166 166  * 60-80%: Published as standard
167 167  * 80-100%: Published as "High confidence"
289 +
290 +**User Needs served**: UN-1 (Trust assessment), UN-9 (Methodology transparency)
291 +
168 168  == 5. Automated Risk Scoring ==
293 +
294 +**Fulfills**: UN-10 (Manipulation detection), UN-16 (Appropriate review level)
295 +
169 169  **Replace manual risk tiers with continuous automated scoring**.
297 +
170 170  === 5.1 Risk Score Calculation ===
299 +
171 171  **Factors** (weighted algorithm):
301 +
172 172  * **Domain sensitivity**: Medical, legal, safety auto-flagged higher
173 173  * **Potential impact**: Views, citations, spread
174 174  * **Controversy level**: Flags, disputes, edit wars
175 175  * **Uncertainty**: Low confidence, contradictory evidence
176 176  * **Source reliability**: Track record of sources used
307 +
177 177  **Score**: 0-100 (higher = more risk)
309 +
178 178  === 5.2 Automated Actions ===
311 +
179 179  * **Score > 80**: Flag for moderator review before publication
180 180  * **Score 60-80**: Publish with prominent warnings
181 181  * **Score 40-60**: Publish with standard warnings
182 182  * **Score < 40**: Publish normally
316 +
183 183  **Continuous monitoring**: Risk score recalculated as new information emerges
318 +
319 +**User Needs served**: UN-10 (Detect manipulation tactics), UN-16 (Review status)
320 +
184 184  == 6. System Improvement Process ==
322 +
185 185  **Core principle**: Fix the system, not just the data.
324 +
186 186  === 6.1 Error Capture ===
326 +
187 187  **When users flag errors or make corrections**:
328 +
188 188  1. What was wrong? (categorize)
189 189  2. What should it have been?
190 190  3. Why did the system fail? (root cause)
191 191  4. How common is this pattern?
192 192  5. Store in ErrorPattern table (improvement queue)
193 -=== 6.2 Weekly Improvement Cycle ===
334 +
335 +=== 6.2 Continuous Improvement Cycle ===
336 +
194 194  1. **Review**: Analyze top error patterns
195 195  2. **Develop**: Create fix (prompt, model, validation)
196 196  3. **Test**: Validate fix on sample claims
... ... @@ -197,8 +197,11 @@
197 197  4. **Deploy**: Roll out if quality improves
198 198  5. **Re-process**: Automatically update affected claims
199 199  6. **Monitor**: Track quality metrics
343 +
200 200  === 6.3 Quality Metrics Dashboard ===
345 +
201 201  **Track continuously**:
347 +
202 202  * Error rate by category
203 203  * Source quality distribution
204 204  * Confidence score trends
... ... @@ -206,158 +206,1753 @@
206 206  * Correction acceptance rate
207 207  * Re-work rate
208 208  * Claims processed per hour
209 -**Goal**: 10% monthly improvement in error rate
355 +
356 +**Goal**: continuous improvement in error rate
357 +
210 210  == 7. Automated Quality Monitoring ==
359 +
211 211  **Replace manual audit sampling with automated monitoring**.
361 +
212 212  === 7.1 Continuous Metrics ===
363 +
213 213  * **Source quality**: Track record database
214 214  * **Consistency**: Contradiction detection
215 215  * **Clarity**: Readability scores
216 216  * **Completeness**: Field validation
217 217  * **Accuracy**: User corrections tracked
369 +
218 218  === 7.2 Anomaly Detection ===
371 +
219 219  **Automated alerts for**:
373 +
220 220  * Sudden quality drops
221 221  * Unusual patterns
222 222  * Contradiction clusters
223 223  * Source reliability changes
224 224  * User behavior anomalies
379 +
225 225  === 7.3 Targeted Review ===
381 +
226 226  * Review only flagged items
227 227  * Random sampling for calibration (not quotas)
228 228  * Learn from corrections to improve automation
229 -== 8. Claim Intake & Normalization ==
230 -=== 8.1 FR1 – Claim Intake ===
385 +
386 +== 8. Functional Requirements ==
387 +
388 +This section defines specific features that fulfill user needs.
389 +
390 +=== 8.1 Claim Intake & Normalization ===
391 +
392 +==== FR1 — Claim Intake ====
393 +
394 +**Fulfills**: UN-2 (Claim extraction), UN-4 (Quick fact-checking), UN-12 (Submit claims)
395 +
231 231  * Users submit claims via simple form or API
232 232  * Claims can be text, URL, or image
233 233  * Duplicate detection (semantic similarity)
234 234  * Auto-categorization by domain
235 -=== 8.2 FR2 – Claim Normalization ===
400 +
401 +==== FR2 — Claim Normalization ====
402 +
403 +**Fulfills**: UN-2 (Claim verification)
404 +
236 236  * Standardize to clear assertion format
237 237  * Extract key entities (who, what, when, where)
238 238  * Identify claim type (factual, predictive, evaluative)
239 239  * Link to existing similar claims
240 -=== 8.3 FR3 – Claim Classification ===
409 +
410 +==== FR3 — Claim Classification ====
411 +
412 +**Fulfills**: UN-11 (Filtered research)
413 +
241 241  * Domain: Politics, Science, Health, etc.
242 242  * Type: Historical fact, current stat, prediction, etc.
243 243  * Risk score: Automated calculation
244 244  * Complexity: Simple, moderate, complex
245 -== 9. Scenario System ==
246 -=== 9.1 FR4 – Scenario Generation ===
418 +
419 +=== 8.2 Scenario System ===
420 +
421 +==== FR4 — Scenario Generation ====
422 +
423 +**Fulfills**: UN-2 (Context-dependent verification), UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
424 +
247 247  **Automated scenario creation**:
248 -* AKEL analyzes claim and generates likely scenarios
249 -* Each scenario includes: assumptions, evidence, conclusion
426 +
427 +* AKEL analyzes claim and generates likely scenarios (use-cases and contexts)
428 +* Each scenario includes: assumptions, definitions, boundaries, evidence context
250 250  * Users can flag incorrect scenarios
251 251  * System learns from corrections
252 -=== 9.2 FR5 – Evidence Linking ===
431 +
432 +**Key Concept**: Scenarios represent different interpretations or contexts (e.g., "Clinical trials with healthy adults" vs. "Real-world data with diverse populations")
433 +
434 +==== FR5 — Evidence Linking ====
435 +
436 +**Fulfills**: UN-5 (Source tracing), UN-7 (Evidence transparency)
437 +
253 253  * Automated evidence discovery from sources
254 254  * Relevance scoring
255 255  * Contradiction detection
256 256  * Source quality assessment
257 -=== 9.3 FR6 – Scenario Comparison ===
442 +
443 +==== FR6 — Scenario Comparison ====
444 +
445 +**Fulfills**: UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
446 +
258 258  * Side-by-side comparison interface
259 -* Highlight key differences
260 -* Show evidence supporting each
261 -* Display confidence scores
262 -== 10. Verdicts & Analysis ==
263 -=== 10.1 FR7 – Automated Verdicts ===
264 -* AKEL generates verdict based on evidence
448 +* Highlight key differences between scenarios
449 +* Show evidence supporting each scenario
450 +* Display confidence scores per scenario
451 +
452 +=== 8.3 Verdicts & Analysis ===
453 +
454 +==== FR7 — Automated Verdicts ====
455 +
456 +**Fulfills**: UN-1 (Trust score), UN-2 (Verification verdicts), UN-3 (Article summary with FactHarbor analysis summary), UN-13 (Cite verdicts)
457 +
458 +* AKEL generates verdict based on evidence within each scenario
459 +* **Likelihood range** displayed (e.g., "0.70-0.85 (likely true)") - NOT binary true/false
460 +* **Uncertainty factors** explicitly listed (e.g., "Small sample sizes", "Long-term effects unknown")
265 265  * Confidence score displayed prominently
266 -* Source quality indicators
462 +* Source quality indicators shown
267 267  * Contradictions noted
268 268  * Uncertainty acknowledged
269 -=== 10.2 FR8 – Time Evolution ===
270 -* Claims update as new evidence emerges
271 -* Version history maintained
465 +
466 +**Key Innovation**: Detailed probabilistic verdicts with explicit uncertainty, not binary judgments
467 +
468 +==== FR8 — Time Evolution ====
469 +
470 +{{warning}}
471 +**Status:** Deferred (Not in V1.0)
472 +
473 +This requirement has been **dropped from the current architecture and design**. Versioned entities have been replaced with simple edit history tracking only. Full evolution timeline functionality is deferred to future releases beyond V1.0.
474 +{{/warning}}
475 +
476 +**Fulfills**: UN-15 (Verdict evolution timeline)
477 +
478 +* Claims and verdicts update as new evidence emerges
479 +* Version history maintained for all verdicts
272 272  * Changes highlighted
273 273  * Confidence score trends visible
274 -== 11. Workflow & Moderation ==
275 -=== 11.1 FR9 – Publication Workflow ===
482 +* Users can see "as of date X, what did we know?"
483 +
484 +=== 8.4 User Interface & Presentation ===
485 +
486 +==== FR12 — Two-Panel Summary View (Article Summary with FactHarbor Analysis Summary) ====
487 +
488 +**Fulfills**: UN-3 (Article Summary with FactHarbor Analysis Summary)
489 +
490 +**Purpose**: Provide side-by-side comparison of what a document claims vs. FactHarbor's complete analysis of its credibility
491 +
492 +**Left Panel: Article Summary**:
493 +
494 +* Document title, source, and claimed credibility
495 +* "The Big Picture" - main thesis or position change
496 +* "Key Findings" - structured summary of document's main claims
497 +* "Reasoning" - document's explanation for positions
498 +* "Conclusion" - document's bottom line
499 +
500 +**Right Panel: FactHarbor Analysis Summary**:
501 +
502 +* FactHarbor's independent source credibility assessment
503 +* Claim-by-claim verdicts with confidence scores
504 +* Methodology assessment (strengths, limitations)
505 +* Overall verdict on document quality
506 +* Analysis ID for reference
507 +
508 +**Design Principles**:
509 +
510 +* No scrolling required - both panels visible simultaneously
511 +* Visual distinction between "what they say" and "FactHarbor's analysis"
512 +* Color coding for verdicts (supported, uncertain, refuted)
513 +* Confidence percentages clearly visible
514 +* Mobile responsive (panels stack vertically on small screens)
515 +
516 +**Implementation Notes**:
517 +
518 +* Generated automatically by AKEL for every analyzed document
519 +* Updates when verdict evolves (maintains version history)
520 +* Exportable as standalone summary report
521 +* Shareable via permanent URL
522 +
523 +==== FR13 — In-Article Claim Highlighting ====
524 +
525 +**Fulfills**: UN-17 (In-article claim highlighting)
526 +
527 +**Purpose**: Enable readers to quickly assess claim credibility while reading by visually highlighting factual claims with color-coded indicators
528 +
529 +==== Visual Example: Article with Highlighted Claims ====
530 +
531 +(% class="box" %)
532 +(((
533 +**Article: "New Study Shows Benefits of Mediterranean Diet"**
534 +
535 +A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.
536 +
537 +(% class="box successmessage" style="margin:10px 0;" %)
538 +(((
539 +🟢 **Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups**
540 +
541 +(% style="font-size:0.9em; color:#666;" %)
542 +↑ WELL SUPPORTED • 87% confidence
543 +[[Click for evidence details →]]
544 +
545 +
546 +)))
547 +
548 +The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
549 +
550 +(% class="box warningmessage" style="margin:10px 0;" %)
551 +(((
552 +🟡 **Some experts believe this diet can completely prevent heart attacks**
553 +
554 +(% style="font-size:0.9em; color:#666;" %)
555 +↑ UNCERTAIN • 45% confidence
556 +Overstated - evidence shows risk reduction, not prevention
557 +[[Click for details →]]
558 +
559 +
560 +)))
561 +
562 +Dr. Maria Rodriguez, lead researcher, recommends incorporating more olive oil, fish, and vegetables into daily meals.
563 +
564 +(% class="box errormessage" style="margin:10px 0;" %)
565 +(((
566 +🔴 **The study proves that saturated fats cause heart disease**
567 +
568 +(% style="font-size:0.9em; color:#666;" %)
569 +↑ REFUTED • 15% confidence
570 +Claim not supported by study design; correlation ≠ causation
571 +[[Click for counter-evidence →]]
572 +
573 +
574 +)))
575 +
576 +Participants also reported feeling more energetic and experiencing better sleep quality, though these were secondary measures.
577 +)))
578 +
579 +**Legend:**
580 +
581 +* 🟢 = Well-supported claim (confidence ≥75%)
582 +* 🟡 = Uncertain claim (confidence 40-74%)
583 +* 🔴 = Refuted/unsupported claim (confidence <40%)
584 +* Plain text = Non-factual content (context, opinions, recommendations)
585 +
586 +==== Tooltip on Hover/Click ====
587 +
588 +(% class="box infomessage" %)
589 +(((
590 +**FactHarbor Analysis**
591 +
592 +**Claim:**
593 +"Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease"
594 +
595 +**Verdict:** WELL SUPPORTED
596 +**Confidence:** 87%
597 +
598 +**Evidence Summary:**
599 +
600 +* Meta-analysis of 12 RCTs confirms 23-28% risk reduction
601 +* Consistent findings across multiple populations
602 +* Published in peer-reviewed journal (high credibility)
603 +
604 +**Uncertainty Factors:**
605 +
606 +* Exact percentage varies by study (20-30% range)
607 +
608 +[[View Full Analysis →]]
609 +)))
610 +
611 +**Color-Coding System**:
612 +
613 +* **Green**: Well-supported claims (confidence ≥75%, strong evidence)
614 +* **Yellow/Orange**: Uncertain claims (confidence 40-74%, conflicting or limited evidence)
615 +* **Red**: Refuted or unsupported claims (confidence <40%, contradicted by evidence)
616 +* **Gray/Neutral**: Non-factual content (opinions, questions, procedural text)
617 +
618 +==== Interactive Highlighting Example (Detailed View) ====
619 +
620 +(% style="width:100%; border-collapse:collapse;" %)
621 +|=**Article Text**|=**Status**|=**Analysis**
622 +|(((
623 +A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.
624 +)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Context - no highlighting
625 +|(((
626 +//Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups//
627 +)))|(% style="background-color:#D4EDDA; text-align:center; padding:8px;" %)🟢 **WELL SUPPORTED**|(((
628 +**87% confidence**
629 +
630 +Meta-analysis of 12 RCTs confirms 23-28% risk reduction
631 +
632 +[[View Full Analysis]]
633 +)))
634 +|(((
635 +The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
636 +)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Methodology - no highlighting
637 +|(((
638 +//Some experts believe this diet can completely prevent heart attacks//
639 +)))|(% style="background-color:#FFF3CD; text-align:center; padding:8px;" %)🟡 **UNCERTAIN**|(((
640 +**45% confidence**
641 +
642 +Overstated - evidence shows risk reduction, not prevention
643 +
644 +[[View Details]]
645 +)))
646 +|(((
647 +Dr. Rodriguez recommends incorporating more olive oil, fish, and vegetables into daily meals.
648 +)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Recommendation - no highlighting
649 +|(((
650 +//The study proves that saturated fats cause heart disease//
651 +)))|(% style="background-color:#F8D7DA; text-align:center; padding:8px;" %)🔴 **REFUTED**|(((
652 +**15% confidence**
653 +
654 +Claim not supported by study; correlation ≠ causation
655 +
656 +[[View Counter-Evidence]]
657 +)))
658 +
659 +**Design Notes:**
660 +
661 +* Highlighted claims use italics to distinguish from plain text
662 +* Color backgrounds match XWiki message box colors (success/warning/error)
663 +* Status column shows verdict prominently
664 +* Analysis column provides quick summary with link to details
665 +
666 +**User Actions**:
667 +
668 +* **Hover** over highlighted claim → Tooltip appears
669 +* **Click** highlighted claim → Detailed analysis modal/panel
670 +* **Toggle** button to turn highlighting on/off
671 +* **Keyboard**: Tab through highlighted claims
672 +
673 +**Interaction Design**:
674 +
675 +* Hover/click on highlighted claim → Show tooltip with:
676 +* Claim text
677 +* Verdict (e.g., "WELL SUPPORTED")
678 +* Confidence score (e.g., "85%")
679 +* Brief evidence summary
680 +* Link to detailed analysis
681 +* Toggle highlighting on/off (user preference)
682 +* Adjustable color intensity for accessibility
683 +
684 +**Technical Requirements**:
685 +
686 +* Real-time highlighting as page loads (non-blocking)
687 +* Claim boundary detection (start/end of assertion)
688 +* Handle nested or overlapping claims
689 +* Preserve original article formatting
690 +* Work with various content formats (HTML, plain text, PDFs)
691 +
692 +**Performance Requirements**:
693 +
694 +* Highlighting renders within 500ms of page load
695 +* No perceptible delay in reading experience
696 +* Efficient DOM manipulation (avoid reflows)
697 +
698 +**Accessibility**:
699 +
700 +* Color-blind friendly palette (use patterns/icons in addition to color)
701 +* Screen reader compatible (ARIA labels for claim credibility)
702 +* Keyboard navigation to highlighted claims
703 +
704 +**Implementation Notes**:
705 +
706 +* Claims extracted and analyzed by AKEL during initial processing
707 +* Highlighting data stored as annotations with byte offsets
708 +* Client-side rendering of highlights based on verdict data
709 +* Mobile responsive (tap instead of hover)
710 +
711 +=== 8.5 Workflow & Moderation ===
712 +
713 +==== FR9 — Publication Workflow ====
714 +
715 +**Fulfills**: UN-1 (Fast access to verified content), UN-16 (Clear review status)
716 +
276 276  **Simple flow**:
718 +
277 277  1. Claim submitted
278 278  2. AKEL processes (automated)
279 -3. If confidence > threshold: Publish
721 +3. If confidence > threshold: Publish (labeled as AI-generated)
280 280  4. If confidence < threshold: Flag for improvement
281 281  5. If risk score > threshold: Flag for moderator
724 +
282 282  **No multi-stage approval process**
283 -=== 11.2 FR10 – Moderation ===
726 +
727 +==== FR10 — Moderation ====
728 +
284 284  **Focus on abuse, not routine quality**:
730 +
285 285  * Automated abuse detection
286 286  * Moderators handle flags
287 287  * Quick response to harmful content
288 288  * Minimal involvement in routine content
289 -=== 11.3 FR11 – Audit Trail ===
735 +
736 +==== FR11 — Audit Trail ====
737 +
738 +**Fulfills**: UN-14 (API access to histories), UN-15 (Evolution tracking)
739 +
290 290  * All edits logged
291 291  * Version history public
292 292  * Moderation decisions documented
293 293  * System improvements tracked
294 -== 12. Technical Requirements ==
295 -=== 12.1 NFR1 – Performance ===
744 +
745 +== 9. Non-Functional Requirements ==
746 +
747 +=== 9.1 NFR1 — Performance ===
748 +
749 +**Fulfills**: UN-4 (Fast fact-checking), UN-11 (Responsive filtering)
750 +
296 296  * Claim processing: < 30 seconds
297 297  * Search response: < 2 seconds
298 298  * Page load: < 3 seconds
299 299  * 99% uptime
300 -=== 12.2 NFR2 – Scalability ===
755 +
756 +=== 9.2 NFR2 — Scalability ===
757 +
758 +**Fulfills**: UN-14 (API access at scale)
759 +
301 301  * Handle 10,000 claims initially
302 302  * Scale to 1M+ claims
303 303  * Support 100K+ concurrent users
304 304  * Automated processing scales linearly
305 -=== 12.3 NFR3 – Transparency ===
764 +
765 +=== 9.3 NFR3 — Transparency ===
766 +
767 +**Fulfills**: UN-7 (Evidence transparency), UN-9 (Methodology transparency), UN-13 (Citable verdicts), UN-15 (Evolution visibility)
768 +
306 306  * All algorithms open source
307 307  * All data exportable
308 308  * All decisions documented
309 309  * Quality metrics public
310 -=== 12.4 NFR4 – Security & Privacy ===
773 +
774 +=== 9.4 NFR4 — Security & Privacy ===
775 +
311 311  * Follow [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
312 312  * Secure authentication
313 313  * Data encryption
314 314  * Regular security audits
315 -=== 12.5 NFR5 – Maintainability ===
780 +
781 +=== 9.5 NFR5 — Maintainability ===
782 +
316 316  * Modular architecture
317 317  * Automated testing
318 318  * Continuous integration
319 319  * Comprehensive documentation
320 -== 13. MVP Scope ==
321 -**Phase 1 (Months 1-3): Read-Only MVP**
322 -Build:
323 -* Automated claim analysis
324 -* Confidence scoring
325 -* Source evaluation
326 -* Browse/search interface
327 -* User flagging system
328 -**Goal**: Prove AI quality before adding user editing
329 -**Phase 2 (Months 4-6): User Contributions**
330 -Add only if needed:
331 -* Simple editing (Wikipedia-style)
332 -* Reputation system
333 -* Basic moderation
334 -**Phase 3 (Months 7-12): Refinement**
335 -* Continuous quality improvement
336 -* Feature additions based on real usage
337 -* Scale infrastructure
338 -**Deferred**:
339 -* Federation (until multiple successful instances exist)
340 -* Complex contribution workflows (focus on automation)
341 -* Extensive role hierarchy (keep simple)
342 -== 14. Success Metrics ==
343 -**System Quality** (track weekly):
344 -* Error rate by category (target: -10%/month)
345 -* Average confidence score (target: increase)
346 -* Source quality distribution (target: more high-quality)
347 -* Contradiction detection rate (target: increase)
348 -**Efficiency** (track monthly):
349 -* Claims processed per hour (target: increase)
350 -* Human hours per claim (target: decrease)
351 -* Automation coverage (target: >90%)
352 -* Re-work rate (target: <5%)
353 -**User Satisfaction** (track quarterly):
354 -* User flag rate (issues found)
355 -* Correction acceptance rate (flags valid)
356 -* Return user rate
357 -* Trust indicators (surveys)
358 -== 15. Related Pages ==
359 -* [[Architecture>>FactHarbor.Specification.Architecture.WebHome]]
360 -* [[Data Model>>FactHarbor.Specification.Data Model.WebHome]]
361 -* [[Workflows>>FactHarbor.Specification.Workflows.WebHome]]
362 -* [[Global Rules>>FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]]
787 +
788 +=== NFR11: AKEL Quality Assurance Framework ===
789 +
790 +**Fulfills:** AI safety, IFCN methodology transparency
791 +
792 +**Specification:**
793 +
794 +Multi-layer AI quality gates to detect hallucinations, low-confidence results, and logical inconsistencies.
795 +
796 +==== Quality Gate 1: Claim Extraction Validation ====
797 +
798 +**Purpose:** Ensure extracted claims are factual assertions (not opinions/predictions)
799 +
800 +**Checks:**
801 +
802 +1. **Factual Statement Test:** Is this verifiable? (Yes/No)
803 +2. **Opinion Detection:** Contains hedging language? ("I think", "probably", "best")
804 +3. **Future Prediction Test:** Makes claims about future events?
805 +4. **Specificity Score:** Contains specific entities, numbers, dates?
806 +
807 +**Thresholds:**
808 +
809 +* Factual: Must be "Yes"
810 +* Opinion markers: <2 hedging phrases
811 +* Specificity: ≥3 specific elements
812 +
813 +**Action if Failed:** Flag as "Non-verifiable", do NOT generate verdict
814 +
815 +==== Quality Gate 2: Evidence Relevance Validation ====
816 +
817 +**Purpose:** Ensure AI-linked evidence actually relates to claim
818 +
819 +**Checks:**
820 +
821 +1. **Semantic Similarity Score:** Evidence vs. claim (embeddings)
822 +2. **Entity Overlap:** Shared people/places/things?
823 +3. **Topic Relevance:** Discusses claim subject?
824 +
825 +**Thresholds:**
826 +
827 +* Similarity: ≥0.6 (cosine similarity)
828 +* Entity overlap: ≥1 shared entity
829 +* Topic relevance: ≥0.5
830 +
831 +**Action if Failed:** Discard irrelevant evidence
832 +
833 +==== Quality Gate 3: Scenario Coherence Check ====
834 +
835 +**Purpose:** Validate scenario assumptions are logical and complete
836 +
837 +**Checks:**
838 +
839 +1. **Completeness:** All required fields populated
840 +2. **Internal Consistency:** Assumptions don't contradict
841 +3. **Distinguishability:** Scenarios meaningfully different
842 +
843 +**Thresholds:**
844 +
845 +* Required fields: 100%
846 +* Contradiction score: <0.3
847 +* Scenario similarity: <0.8
848 +
849 +**Action if Failed:** Merge duplicates, reduce confidence -20%
850 +
851 +==== Quality Gate 4: Verdict Confidence Assessment ====
852 +
853 +**Purpose:** Only publish high-confidence verdicts
854 +
855 +**Checks:**
856 +
857 +1. **Evidence Count:** Minimum 2 sources
858 +2. **Source Quality:** Average reliability ≥0.6
859 +3. **Evidence Agreement:** Supporting vs. contradicting ≥0.6
860 +4. **Uncertainty Factors:** Hedging in reasoning
861 +
862 +**Confidence Tiers:**
863 +
864 +* **HIGH (80-100%):** ≥3 sources, ≥0.7 quality, ≥80% agreement
865 +* **MEDIUM (50-79%):** ≥2 sources, ≥0.6 quality, ≥60% agreement
866 +* **LOW (0-49%):** <2 sources OR low quality/agreement
867 +* **INSUFFICIENT:** <2 sources → DO NOT PUBLISH
868 +
869 +**Implementation Phases:**
870 +
871 +* **POC1:** Gates 1 & 4 only (basic validation)
872 +* **POC2:** All 4 gates (complete framework)
873 +* **V1.0:** Hardened with <5% hallucination rate
874 +
875 +**Acceptance Criteria:**
876 +
877 +* ✅ All gates operational
878 +* ✅ Hallucination rate <5%
879 +* ✅ Quality metrics public
880 +
881 +=== NFR12: Security Controls ===
882 +
883 +**Fulfills:** Data protection, system integrity, user privacy, production readiness
884 +
885 +**Purpose:** Protect FactHarbor systems, user data, and operations from security threats, ensuring production-grade security posture.
886 +
887 +**Specification:**
888 +
889 +==== API Security ====
890 +
891 +**Rate Limiting:**
892 +
893 +* **Analysis endpoints:** 100 requests/hour per IP
894 +* **Read endpoints:** 1,000 requests/hour per IP
895 +* **Search:** 500 requests/hour per IP
896 +* **Authenticated users:** 5x higher limits
897 +* **Burst protection:** Max 10 requests/second
898 +
899 +**Authentication & Authorization:**
900 +
901 +* **API Keys:** Required for programmatic access
902 +* **JWT tokens:** For user sessions (1-hour expiry)
903 +* **OAuth2:** For third-party integrations
904 +* **Role-Based Access Control (RBAC):**
905 +* Public: Read-only access to published claims
906 +* Contributor: Submit claims, provide evidence
907 +* Moderator: Review contributions, manage quality
908 +* Admin: System configuration, user management
909 +
910 +**CORS Policies:**
911 +
912 +* Whitelist approved domains only
913 +* No wildcard origins in production
914 +* Credentials required for sensitive endpoints
915 +
916 +**Input Sanitization:**
917 +
918 +* Validate all user input against schemas
919 +* Sanitize HTML/JavaScript in text submissions
920 +* Prevent SQL injection (use parameterized queries)
921 +* Prevent command injection (no shell execution of user input)
922 +* Max request size: 10MB
923 +* File upload restrictions: Whitelist file types, scan for malware
924 +
925 +----
926 +
927 +==== Data Security ====
928 +
929 +**Encryption at Rest:**
930 +
931 +* Database encryption using AES-256
932 +* Encrypted backups
933 +* Key management via cloud provider KMS (AWS KMS, Google Cloud KMS)
934 +* Regular key rotation (90-day cycle)
935 +
936 +**Encryption in Transit:**
937 +
938 +* HTTPS/TLS 1.3 only (no TLS 1.0/1.1)
939 +* Strong cipher suites only
940 +* HSTS (HTTP Strict Transport Security) enabled
941 +* Certificate pinning for mobile apps
942 +
943 +**Secure Credential Storage:**
944 +
945 +* Passwords hashed with bcrypt (cost factor 12+)
946 +* API keys encrypted in database
947 +* Secrets stored in environment variables (never in code)
948 +* Use secrets manager (AWS Secrets Manager, HashiCorp Vault)
949 +
950 +**Data Privacy:**
951 +
952 +* Minimal data collection (privacy by design)
953 +* User data deletion on request (GDPR compliance)
954 +* PII encryption in database
955 +* Anonymize logs (no PII in log files)
956 +
957 +----
958 +
959 +==== Application Security ====
960 +
961 +**OWASP Top 10 Compliance:**
962 +
963 +1. **Broken Access Control:** RBAC implementation, path traversal prevention
964 +2. **Cryptographic Failures:** Strong encryption, secure key management
965 +3. **Injection:** Parameterized queries, input validation
966 +4. **Insecure Design:** Security review of all features
967 +5. **Security Misconfiguration:** Hardened defaults, security headers
968 +6. **Vulnerable Components:** Dependency scanning (see below)
969 +7. **Authentication Failures:** Strong password policy, MFA support
970 +8. **Data Integrity Failures:** Signature verification, checksums
971 +9. **Security Logging Failures:** Comprehensive audit logs
972 +10. **Server-Side Request Forgery:** URL validation, whitelist domains
973 +
974 +**Security Headers:**
975 +
976 +* `Content-Security-Policy`: Strict CSP to prevent XSS
977 +* `X-Frame-Options`: DENY (prevent clickjacking)
978 +* `X-Content-Type-Options`: nosniff
979 +* `Referrer-Policy`: strict-origin-when-cross-origin
980 +* `Permissions-Policy`: Restrict browser features
981 +
982 +**Dependency Vulnerability Scanning:**
983 +
984 +* **Tools:** Snyk, Dependabot, npm audit, pip-audit
985 +* **Frequency:** Daily automated scans
986 +* **Action:** Patch critical vulnerabilities within 24 hours
987 +* **Policy:** No known high/critical CVEs in production
988 +
989 +**Security Audits:**
990 +
991 +* **Internal:** Quarterly security reviews
992 +* **External:** Annual penetration testing by certified firm
993 +* **Bug Bounty:** Public bug bounty program (V1.1+)
994 +* **Compliance:** SOC 2 Type II certification target (V1.5)
995 +
996 +----
997 +
998 +==== Operational Security ====
999 +
1000 +**DDoS Protection:**
1001 +
1002 +* CloudFlare or AWS Shield
1003 +* Rate limiting at CDN layer
1004 +* Automatic IP blocking for abuse patterns
1005 +
1006 +**Monitoring & Alerting:**
1007 +
1008 +* Real-time security event monitoring
1009 +* Alerts for:
1010 +* Failed login attempts (>5 in 10 minutes)
1011 +* API abuse patterns
1012 +* Unusual data access patterns
1013 +* Security scan detections
1014 +* Integration with SIEM (Security Information and Event Management)
1015 +
1016 +**Incident Response:**
1017 +
1018 +* Documented incident response plan
1019 +* Security incident classification (P1-P4)
1020 +* On-call rotation for security issues
1021 +* Post-mortem for all security incidents
1022 +* Public disclosure policy (coordinated disclosure)
1023 +
1024 +**Backup & Recovery:**
1025 +
1026 +* Daily encrypted backups
1027 +* 30-day retention period
1028 +* Tested recovery procedures (quarterly)
1029 +* Disaster recovery plan (RTO: 4 hours, RPO: 1 hour)
1030 +
1031 +----
1032 +
1033 +==== Compliance & Standards ====
1034 +
1035 +**GDPR Compliance:**
1036 +
1037 +* User consent management
1038 +* Right to access data
1039 +* Right to deletion
1040 +* Data portability
1041 +* Privacy policy published
1042 +
1043 +**Accessibility:**
1044 +
1045 +* WCAG 2.1 AA compliance
1046 +* Screen reader compatibility
1047 +* Keyboard navigation
1048 +* Alt text for images
1049 +
1050 +**Browser Support:**
1051 +
1052 +* Modern browsers only (Chrome/Edge/Firefox/Safari latest 2 versions)
1053 +* No IE11 support
1054 +
1055 +**Acceptance Criteria:**
1056 +
1057 +* ✅ Passes OWASP ZAP security scan (no high/critical findings)
1058 +* ✅ All dependencies with known vulnerabilities patched
1059 +* ✅ Penetration test completed with no critical findings
1060 +* ✅ Rate limiting blocks abuse attempts
1061 +* ✅ Encryption at rest and in transit verified
1062 +* ✅ Security headers scored A+ on securityheaders.com
1063 +* ✅ Incident response plan documented and tested
1064 +* ✅ 95% uptime over 30-day period
1065 +
1066 +=== NFR13: Quality Metrics Transparency ===
1067 +
1068 +**Fulfills:** User trust, transparency, continuous improvement, IFCN methodology transparency
1069 +
1070 +**Purpose:** Provide transparent, measurable quality metrics that demonstrate AKEL's performance and build user trust in automated fact-checking.
1071 +
1072 +**Specification:**
1073 +
1074 +==== Component: Public Quality Dashboard ====
1075 +
1076 +**Core Metrics to Display:**
1077 +
1078 +* \\
1079 +** \\
1080 +**1. Verdict Quality Metrics
1081 +
1082 +**TIGERScore (Fact-Checking Quality):**
1083 +
1084 +* **Definition:** Measures how well generated verdicts match expert fact-checker judgments
1085 +* **Scale:** 0-100 (higher is better)
1086 +* **Calculation:** Using TIGERScore framework (Truth-conditional accuracy, Informativeness, Generality, Evaluativeness, Relevance)
1087 +* **Target:** Average ≥80 for production release
1088 +* **Display:**
1089 +{{code}}Verdict Quality (TIGERScore):
1090 +Overall: 84.2 ▲ (+2.1 from last month)
1091 +
1092 +Distribution:
1093 + Excellent (>80): 67%
1094 + Good (60-80): 28%
1095 + Needs Improvement (<60): 5%
1096 +
1097 +Trend: [Graph showing improvement over time]{{/code}}
1098 +
1099 +**2. Hallucination & Faithfulness Metrics**
1100 +
1101 +**AlignScore (Faithfulness to Evidence):**
1102 +
1103 +* **Definition:** Measures how well verdicts align with actual evidence content
1104 +* **Scale:** 0-1 (higher is better)
1105 +* **Purpose:** Detect AI hallucinations (making claims not supported by evidence)
1106 +* **Target:** Average ≥0.85, hallucination rate <5%
1107 +* **Display:**
1108 +{{code}}Evidence Faithfulness (AlignScore):
1109 +Average: 0.87 ▼ (-0.02 from last month)
1110 +
1111 +Hallucination Rate: 4.2%
1112 + - Claims without evidence support: 3.1%
1113 + - Misrepresented evidence: 1.1%
1114 +
1115 +Action: Prompt engineering review scheduled{{/code}}
1116 +
1117 +**3. Evidence Quality Metrics**
1118 +
1119 +**Source Reliability:**
1120 +
1121 +* Average source quality score (0-1 scale)
1122 +* Distribution of high/medium/low quality sources
1123 +* Publisher track record trends
1124 +
1125 +**Evidence Coverage:**
1126 +
1127 +* Average number of sources per claim
1128 +* Percentage of claims with ≥2 sources (EFCSN minimum)
1129 +* Geographic diversity of sources
1130 +
1131 +**Display:**
1132 +{{code}}Evidence Quality:
1133 +
1134 +Average Sources per Claim: 4.2
1135 +Claims with ≥2 sources: 94% (EFCSN compliant)
1136 +
1137 +Source Quality Distribution:
1138 + High quality (>0.8): 48%
1139 + Medium quality (0.5-0.8): 43%
1140 + Low quality (<0.5): 9%
1141 +
1142 +Geographic Diversity: 23 countries represented{{/code}}
1143 +
1144 +**4. Contributor Consensus Metrics** (when human reviewers involved)
1145 +
1146 +**Inter-Rater Reliability (IRR):**
1147 +
1148 +* **Calculation:** Cohen's Kappa or Fleiss' Kappa for multiple raters
1149 +* **Scale:** 0-1 (higher is better)
1150 +* **Interpretation:**
1151 +* >0.8: Almost perfect agreement
1152 +* 0.6-0.8: Substantial agreement
1153 +* 0.4-0.6: Moderate agreement
1154 +* <0.4: Poor agreement
1155 +* **Target:** Maintain ≥0.7 (substantial agreement)
1156 +
1157 +**Display:**
1158 +{{code}}Contributor Consensus:
1159 +
1160 +Inter-Rater Reliability (IRR): 0.73 (Substantial agreement)
1161 + - Verdict agreement: 78%
1162 + - Evidence quality agreement: 71%
1163 + - Scenario structure agreement: 69%
1164 +
1165 +Cases requiring moderator review: 12
1166 +Moderator override rate: 8%{{/code}}
1167 +
1168 +----
1169 +
1170 +==== Quality Dashboard Implementation ====
1171 +
1172 +**Dashboard Location:** `/quality-metrics`
1173 +
1174 +**Update Frequency:**
1175 +
1176 +* **POC2:** Weekly manual updates
1177 +* **Beta 0:** Daily automated updates
1178 +* **V1.0:** Real-time metrics (updated hourly)
1179 +
1180 +**Dashboard Sections:**
1181 +
1182 +1. **Overview:** Key metrics at a glance
1183 +2. **Verdict Quality:** TIGERScore trends and distributions
1184 +3. **Evidence Analysis:** Source quality and coverage
1185 +4. **AI Performance:** Hallucination rates, AlignScore
1186 +5. **Human Oversight:** Contributor consensus, review rates
1187 +6. **System Health:** Processing times, error rates, uptime
1188 +
1189 +**Example Dashboard Layout:**
1190 +
1191 +{{code}}
1192 +┌─────────────────────────────────────────────────────────────┐
1193 +│ FactHarbor Quality Metrics Last updated: │
1194 +│ Public Dashboard 2 hours ago │
1195 +└─────────────────────────────────────────────────────────────┘
1196 +
1197 +📊 KEY METRICS
1198 +─────────────────────────────────────────────────────────────
1199 +TIGERScore (Verdict Quality): 84.2 ▲ (+2.1)
1200 +AlignScore (Faithfulness): 0.87 ▼ (-0.02)
1201 +Hallucination Rate: 4.2% ✓ (Target: <5%)
1202 +Average Sources per Claim: 4.2 ▲ (+0.3)
1203 +
1204 +📈 TRENDS (30 days)
1205 +─────────────────────────────────────────────────────────────
1206 +[Graph: TIGERScore trending upward]
1207 +[Graph: Hallucination rate declining]
1208 +[Graph: Evidence quality stable]
1209 +
1210 +⚠️ IMPROVEMENT TARGETS
1211 +─────────────────────────────────────────────────────────────
1212 +1. Reduce hallucination rate to <3% (Current: 4.2%)
1213 +2. Increase TIGERScore average to >85 (Current: 84.2)
1214 +3. Maintain IRR >0.75 (Current: 0.73)
1215 +
1216 +📄 DETAILED REPORTS
1217 +─────────────────────────────────────────────────────────────
1218 +• Monthly Quality Report (PDF)
1219 +• Methodology Documentation
1220 +• AKEL Performance Analysis
1221 +• Contributor Agreement Analysis
1222 +
1223 +{{/code}}
1224 +
1225 +----
1226 +
1227 +==== Continuous Improvement Feedback Loop ====
1228 +
1229 +**How Metrics Inform AKEL Improvements:**
1230 +
1231 +1. **Identify Weak Areas:**
1232 +
1233 +* Low TIGERScore → Review prompt engineering
1234 +* High hallucination → Strengthen evidence grounding
1235 +* Low IRR → Clarify evaluation criteria
1236 +
1237 +2. **A/B Testing Integration:**
1238 +
1239 +* Test prompt variations
1240 +* Measure impact on quality metrics
1241 +* Deploy winners automatically
1242 +
1243 +3. **Alert Thresholds:**
1244 +
1245 +* TIGERScore drops below 75 → Alert team
1246 +* Hallucination rate exceeds 7% → Pause auto-publishing
1247 +* IRR below 0.6 → Moderator training needed
1248 +
1249 +4. **Monthly Quality Reviews:**
1250 +
1251 +* Analyze trends
1252 +* Identify systematic issues
1253 +* Plan prompt improvements
1254 +* Update AKEL models
1255 +
1256 +----
1257 +
1258 +==== Metric Calculation Details ====
1259 +
1260 +**TIGERScore Implementation:**
1261 +
1262 +* Reference: https://github.com/TIGER-AI-Lab/TIGERScore
1263 +* Input: Generated verdict + reference verdict (from expert)
1264 +* Output: 0-100 score across 5 dimensions
1265 +* Requires: Test set of expert-reviewed claims (minimum 100)
1266 +
1267 +**AlignScore Implementation:**
1268 +
1269 +* Reference: https://github.com/yuh-zha/AlignScore
1270 +* Input: Generated verdict + source evidence text
1271 +* Output: 0-1 faithfulness score
1272 +* Calculation: Semantic alignment between claim and evidence
1273 +
1274 +**Source Quality Scoring:**
1275 +
1276 +* Use existing source reliability database (e.g., NewsGuard, MBFC)
1277 +* Factor in: Publication history, corrections record, transparency
1278 +* Scale: 0-1 (weighted average across sources)
1279 +
1280 +----
1281 +
1282 +==== Integration Points ====
1283 +
1284 +* **NFR11: AKEL Quality Assurance** - Metrics validate quality gate effectiveness
1285 +* **FR49: A/B Testing** - Metrics measure test success
1286 +* **FR11: Audit Trail** - Source of quality data
1287 +* **NFR3: Transparency** - Public metrics build trust
1288 +
1289 +**Acceptance Criteria:**
1290 +
1291 +* ✅ All core metrics implemented and calculating correctly
1292 +* ✅ Dashboard updates daily (Beta 0) or hourly (V1.0)
1293 +* ✅ Alerts trigger when metrics degrade beyond thresholds
1294 +* ✅ Monthly quality report auto-generates
1295 +* ✅ Dashboard is publicly accessible (no login required)
1296 +* ✅ Mobile-responsive dashboard design
1297 +* ✅ Metrics inform quarterly AKEL improvement planning
1298 +
1299 +== 13. Requirements Traceability ==
1300 +
1301 +For full traceability matrix showing which requirements fulfill which user needs, see:
1302 +
1303 +* [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] - Section 8 includes comprehensive mapping tables
1304 +
1305 +== 14. Related Pages ==
1306 +
1307 +**Non-Functional Requirements (see Section 9):**
1308 +
1309 +* [[NFR11 — AKEL Quality Assurance Framework>>#NFR11]]
1310 +* [[NFR12 — Security Controls>>#NFR12]]
1311 +* [[NFR13 — Quality Metrics Transparency>>#NFR13]]
1312 +
1313 +**Other Requirements:**
1314 +
1315 +* [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]
1316 +* [[V1.0 Requirements>>FactHarbor.Specification.Requirements.V10.]]
1317 +* [[Gap Analysis>>FactHarbor.Specification.Requirements.GapAnalysis]]
1318 +
1319 +* **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need (drives these requirements)
1320 +* [[Architecture>>Archive.FactHarbor.Specification.Architecture.WebHome]] - How requirements are implemented
1321 +* [[Data Model>>FactHarbor.Specification.Data Model.WebHome]] - Data structures supporting requirements
1322 +* [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] - User interaction workflows
1323 +* [[AKEL>>Archive.FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements
1324 +* [[Global Rules>>Archive.FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]]
363 363  * [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
1326 +
1327 += V0.9.70 Additional Requirements =
1328 +
1329 +== Functional Requirements (Additional) ==
1330 +
1331 +=== FR44: ClaimReview Schema Implementation ===
1332 +
1333 +**Fulfills:** UN-13 (Cite FactHarbor Verdicts), UN-14 (API Access for Integration), UN-26 (Search Engine Visibility)
1334 +
1335 +**Purpose:** Generate valid ClaimReview structured data for every published analysis to enable Google/Bing search visibility and fact-check discovery.
1336 +
1337 +**Specification:**
1338 +
1339 +==== Component: Schema.org Markup Generator ====
1340 +
1341 +FactHarbor must generate valid ClaimReview structured data following Schema.org specifications for every published claim analysis.
1342 +
1343 +**Required JSON-LD Schema:**
1344 +
1345 +{{code language="json"}}
1346 +{
1347 + "@context": "https://schema.org",
1348 + "@type": "ClaimReview",
1349 + "datePublished": "YYYY-MM-DD",
1350 + "url": "https://factharbor.org/claims/{claim_id}",
1351 + "claimReviewed": "The exact claim text",
1352 + "author": {
1353 + "@type": "Organization",
1354 + "name": "FactHarbor",
1355 + "url": "https://factharbor.org"
1356 + },
1357 + "reviewRating": {
1358 + "@type": "Rating",
1359 + "ratingValue": "1-5",
1360 + "bestRating": "5",
1361 + "worstRating": "1",
1362 + "alternateName": "FactHarbor likelihood score"
1363 + },
1364 + "itemReviewed": {
1365 + "@type": "Claim",
1366 + "author": {
1367 + "@type": "Person",
1368 + "name": "Claim author if known"
1369 + },
1370 + "datePublished": "YYYY-MM-DD if known",
1371 + "appearance": {
1372 + "@type": "CreativeWork",
1373 + "url": "Original claim URL if from article"
1374 + }
1375 + }
1376 +}
1377 +{{/code}}
1378 +
1379 +**FactHarbor-Specific Mapping:**
1380 +
1381 +**Likelihood Score to Rating Scale:**
1382 +
1383 +* 80-100% likelihood → 5 (Highly Supported)
1384 +* 60-79% likelihood → 4 (Supported)
1385 +* 40-59% likelihood → 3 (Mixed/Uncertain)
1386 +* 20-39% likelihood → 2 (Questionable)
1387 +* 0-19% likelihood → 1 (Refuted)
1388 +
1389 +**Multiple Scenarios Handling:**
1390 +
1391 +* If claim has multiple scenarios with different verdicts, generate **separate ClaimReview** for each scenario
1392 +* Add `disambiguatingDescription` field explaining scenario context
1393 +* Example: "Scenario: If interpreted as referring to 2023 data..."
1394 +
1395 +==== Implementation Requirements ====
1396 +
1397 +1. **Auto-generate** on claim publication
1398 +2. **Embed** in HTML `<head>` section as JSON-LD script
1399 +3. **Validate** against Schema.org validator before publishing
1400 +4. **Submit** to Google Search Console for indexing
1401 +5. **Update** automatically when verdict changes (integrate with FR8: Time Evolution)
1402 +
1403 +==== Integration Points ====
1404 +
1405 +* **FR7: Automated Verdicts** - Source of rating data and claim text
1406 +* **FR8: Time Evolution** - Triggers schema updates when verdicts change
1407 +* **FR11: Audit Trail** - Logs all schema generation and update events
1408 +
1409 +==== Resources ====
1410 +
1411 +* ClaimReview Project: https://www.claimreviewproject.com
1412 +* Schema.org ClaimReview: https://schema.org/ClaimReview
1413 +* Google Fact Check Guidelines: https://developers.google.com/search/docs/appearance/fact-check
1414 +
1415 +**Acceptance Criteria:**
1416 +
1417 +* ✅ Passes Google Structured Data Testing Tool
1418 +* ✅ Appears in Google Fact Check Explorer within 48 hours of publication
1419 +* ✅ Valid JSON-LD syntax (no errors)
1420 +* ✅ All required fields populated with correct data types
1421 +* ✅ Handles multi-scenario claims correctly (separate ClaimReview per scenario)
1422 +
1423 +=== FR45: User Corrections Notification System ===
1424 +
1425 +**Fulfills:** IFCN Principle 5 (Open & Honest Corrections), EFCSN compliance
1426 +
1427 +**Purpose:** When any claim analysis is corrected, notify users who previously viewed the claim to maintain transparency and build trust.
1428 +
1429 +**Specification:**
1430 +
1431 +==== Component: Corrections Visibility Framework ====
1432 +
1433 +**Correction Types:**
1434 +
1435 +1. **Major Correction:** Verdict changes category (e.g., "Supported" → "Refuted")
1436 +2. **Significant Correction:** Likelihood score changes >20%
1437 +3. **Minor Correction:** Evidence additions, source quality updates
1438 +4. **Scenario Addition:** New scenario added to existing claim
1439 +
1440 +==== Notification Mechanisms ====
1441 +
1442 +* \\
1443 +** \\
1444 +**1. In-Page Banner:
1445 +
1446 +Display prominent banner on claim page:
1447 +
1448 +{{code}}
1449 +[!] CORRECTION NOTICE
1450 +This analysis was updated on [DATE]. [View what changed] [Dismiss]
1451 +
1452 +Major changes:
1453 +• Verdict changed from "Likely True (75%)" to "Uncertain (45%)"
1454 +• New contradicting evidence added from [Source]
1455 +• Scenario 2 updated with additional context
1456 +
1457 +[See full correction log]
1458 +{{/code}}
1459 +
1460 +**2. Correction Log Page:**
1461 +
1462 +* Public changelog at `/claims/{id}/corrections`
1463 +* Displays for each correction:
1464 +* Date/time of correction
1465 +* What changed (before/after comparison)
1466 +* Why changed (reason if provided)
1467 +* Who made change (AKEL auto-update vs. contributor override)
1468 +
1469 +**3. Email Notifications (opt-in):**
1470 +
1471 +* Send to users who bookmarked or shared the claim
1472 +* Subject: "FactHarbor Correction: [Claim title]"
1473 +* Include summary of changes
1474 +* Link to updated analysis
1475 +
1476 +**4. RSS/API Feed:**
1477 +
1478 +* Corrections feed at `/corrections.rss`
1479 +* API endpoint: `GET /api/corrections?since={timestamp}`
1480 +* Enables external monitoring by journalists and researchers
1481 +
1482 +==== Display Rules ====
1483 +
1484 +* Show banner on **ALL pages** displaying the claim (search results, related claims, embeddings)
1485 +* Banner persists for **30 days** after correction
1486 +* **"Corrections" count badge** on claim card
1487 +* **Timestamp** on every verdict: "Last updated: [datetime]"
1488 +
1489 +==== IFCN Compliance Requirements ====
1490 +
1491 +* Corrections policy published at `/corrections-policy`
1492 +* User can report suspected errors via `/report-error/{claim_id}`
1493 +* Link to IFCN complaint process (if FactHarbor becomes signatory)
1494 +* **Scrupulous transparency:** Never silently edit analyses
1495 +
1496 +==== Integration Points ====
1497 +
1498 +* **FR8: Time Evolution** - Triggers corrections when verdicts change
1499 +* **FR11: Audit Trail** - Source of correction data and change history
1500 +* **NFR3: Transparency** - Public correction log demonstrates commitment
1501 +
1502 +**Acceptance Criteria:**
1503 +
1504 +* ✅ Banner appears within 60 seconds of correction
1505 +* ✅ Correction log is permanent and publicly accessible
1506 +* ✅ Email notifications deliver within 5 minutes
1507 +* ✅ RSS feed updates in real-time
1508 +* ✅ Mobile-responsive banner design
1509 +* ✅ Accessible (screen reader compatible)
1510 +
1511 +=== FR46: Image Verification System ===
1512 +
1513 +**Fulfills:** UN-27 (Visual Claim Verification)
1514 +
1515 +**Purpose:** Verify authenticity and context of images shared with claims to detect manipulation, misattribution, and out-of-context usage.
1516 +
1517 +**Specification:**
1518 +
1519 +==== Component: Multi-Method Image Verification ====
1520 +
1521 +**Method 1: Reverse Image Search**
1522 +
1523 +**Purpose:** Find earlier uses of the image to verify context
1524 +
1525 +**Implementation:**
1526 +
1527 +* Integrate APIs:
1528 +* **Google Vision AI** (reverse search)
1529 +* **TinEye** (oldest known uses)
1530 +* **Bing Visual Search** (broad coverage)
1531 +
1532 +**Process:**
1533 +
1534 +1. Extract image from claim or user upload
1535 +2. Query multiple reverse search services
1536 +3. Analyze results for:
1537 +
1538 +* Earliest known publication
1539 +* Original context (what was it really showing?)
1540 +* Publication timeline
1541 +* Geographic spread
1542 +
1543 +**Output:**
1544 +{{code}}Reverse Image Search Results:
1545 +
1546 +Earliest known use: 2019-03-15 (5 years before claim)
1547 +Original context: "Photo from 2019 flooding in Mumbai"
1548 +This claim uses it for: "2024 hurricane damage in Florida"
1549 +
1550 +⚠️ Image is OUT OF CONTEXT
1551 +
1552 +Found in 47 other articles:
1553 +• 2019-03-15: Mumbai floods (original)
1554 +• 2020-07-22: Bangladesh monsoon
1555 +• 2024-10-15: Current claim (misattributed)
1556 +
1557 +[View full timeline]{{/code}}
1558 +
1559 +----
1560 +
1561 +**Method 2: AI Manipulation Detection**
1562 +
1563 +**Purpose:** Detect deepfakes, face swaps, and digital alterations
1564 +
1565 +**Implementation:**
1566 +
1567 +* Integrate detection services:
1568 +* **Sensity AI** (deepfake detection)
1569 +* **Reality Defender** (multimodal analysis)
1570 +* **AWS Rekognition** (face detection inconsistencies)
1571 +
1572 +**Detection Categories:**
1573 +
1574 +1. **Face Manipulation:**
1575 +
1576 +* Deepfake face swaps
1577 +* Expression manipulation
1578 +* Identity replacement
1579 +
1580 +2. **Image Manipulation:**
1581 +
1582 +* Copy-paste artifacts
1583 +* Clone stamp detection
1584 +* Content-aware fill detection
1585 +* JPEG compression inconsistencies
1586 +
1587 +3. **AI Generation:**
1588 +
1589 +* Detect fully AI-generated images
1590 +* Identify generation artifacts
1591 +* Check for model signatures
1592 +
1593 +**Confidence Scoring:**
1594 +
1595 +* **HIGH (80-100%):** Strong evidence of manipulation
1596 +* **MEDIUM (50-79%):** Suspicious artifacts detected
1597 +* **LOW (0-49%):** Minor inconsistencies or inconclusive
1598 +
1599 +**Output:**
1600 +{{code}}Manipulation Analysis:
1601 +
1602 +Face Manipulation: LOW RISK (12%)
1603 +Image Editing: MEDIUM RISK (64%)
1604 + • Clone stamp artifacts detected in sky region
1605 + • JPEG compression inconsistent between objects
1606 +
1607 +AI Generation: LOW RISK (8%)
1608 +
1609 +⚠️ Possible manipulation detected. Manual review recommended.{{/code}}
1610 +
1611 +----
1612 +
1613 +**Method 3: Metadata Analysis (EXIF)**
1614 +
1615 +**Purpose:** Extract technical details that may reveal manipulation or misattribution
1616 +
1617 +**Extracted Data:**
1618 +
1619 +* **Camera/Device:** Make, model, software
1620 +* **Timestamps:** Original date, modification dates
1621 +* **Location:** GPS coordinates (if present)
1622 +* **Editing History:** Software used, edit count
1623 +* **File Properties:** Resolution, compression, format conversions
1624 +
1625 +**Red Flags:**
1626 +
1627 +* Metadata completely stripped (suspicious)
1628 +* Timestamp conflicts with claimed date
1629 +* GPS location conflicts with claimed location
1630 +* Multiple edit rounds (hiding something?)
1631 +* Creation date after modification date (impossible)
1632 +
1633 +**Output:**
1634 +{{code}}Image Metadata:
1635 +
1636 +Camera: iPhone 14 Pro
1637 +Original date: 2023-08-12 14:32:15
1638 +Location: 40.7128°N, 74.0060°W (New York City)
1639 +Modified: 2024-10-15 08:45:22
1640 +Software: Adobe Photoshop 2024
1641 +
1642 +⚠️ Location conflicts with claim
1643 +Claim says: "Taken in Los Angeles"
1644 +EXIF says: New York City
1645 +
1646 +⚠️ Edited 14 months after capture{{/code}}
1647 +
1648 +----
1649 +
1650 +==== Verification Workflow ====
1651 +
1652 +**Automatic Triggers:**
1653 +
1654 +1. User submits claim with image
1655 +2. Article being analyzed contains images
1656 +3. Social media post includes photos
1657 +
1658 +**Process:**
1659 +
1660 +1. Extract images from content
1661 +2. Run all 3 verification methods in parallel
1662 +3. Aggregate results into confidence score
1663 +4. Generate human-readable summary
1664 +5. Display prominently in analysis
1665 +
1666 +**Display Integration:**
1667 +
1668 +Show image verification panel in claim analysis:
1669 +
1670 +{{code}}
1671 +📷 IMAGE VERIFICATION
1672 +
1673 +[Image thumbnail]
1674 +
1675 +✅ Reverse Search: Original context verified
1676 +⚠️ Manipulation: Possible editing detected (64% confidence)
1677 +✅ Metadata: Consistent with claim details
1678 +
1679 +Overall Assessment: CAUTION ADVISED
1680 +This image may have been edited. Original context appears accurate.
1681 +
1682 +[View detailed analysis]
1683 +{{/code}}
1684 +
1685 +==== Integration Points ====
1686 +
1687 +* **FR7: Automated Verdicts** - Image verification affects claim credibility
1688 +* **FR4: Analysis Summary** - Image findings included in summary
1689 +* **UN-27: Visual Claim Verification** - Direct fulfillment
1690 +
1691 +==== Cost Considerations ====
1692 +
1693 +**API Costs (estimated per image):**
1694 +
1695 +* Google Vision AI: $0.001-0.003
1696 +* TinEye: $0.02 (commercial API)
1697 +* Sensity AI: $0.05-0.10
1698 +* AWS Rekognition: $0.001-0.002
1699 +
1700 +**Total per image:** $0.07-0.15**
1701 +
1702 +**Mitigation Strategies:**
1703 +
1704 +* Cache results for duplicate images
1705 +* Use free tier quotas where available
1706 +* Prioritize higher-value claims for deep analysis
1707 +* Offer premium verification as paid tier
1708 +
1709 +**Acceptance Criteria:**
1710 +
1711 +* ✅ Reverse image search finds original sources
1712 +* ✅ Manipulation detection accuracy >80% on test dataset
1713 +* ✅ EXIF extraction works for major image formats (JPEG, PNG, HEIC)
1714 +* ✅ Results display within 10 seconds
1715 +* ✅ Mobile-friendly image comparison interface
1716 +* ✅ False positive rate <15%
1717 +
1718 +=== FR47: Archive.org Integration ===
1719 +
1720 +**Importance:** CRITICAL
1721 +**Fulfills:** Evidence persistence, FR5 (Evidence linking)
1722 +
1723 +**Purpose:** Ensure evidence remains accessible even if original sources are deleted.
1724 +
1725 +**Specification:**
1726 +
1727 +**Automatic Archiving:**
1728 +
1729 +When AKEL links evidence:
1730 +
1731 +1. Check if URL already archived (Wayback Machine API)
1732 +2. If not, submit for archiving (Save Page Now API)
1733 +3. Store both original URL and archive URL
1734 +4. Display both to users
1735 +
1736 +**Archive Display:**
1737 +
1738 +{{code}}
1739 +Evidence Source: [Original URL]
1740 +Archived: [Archive.org URL] (Captured: [date])
1741 +
1742 +[View Original] [View Archive]
1743 +{{/code}}
1744 +
1745 +**Fallback Logic:**
1746 +
1747 +* If original URL unavailable → Auto-redirect to archive
1748 +* If archive unavailable → Display warning
1749 +* If both unavailable → Flag for manual review
1750 +
1751 +**API Integration:**
1752 +
1753 +* Use Wayback Machine Availability API
1754 +* Use Save Page Now API (SPNv2)
1755 +* Rate limiting: 15 requests/minute (Wayback limit)
1756 +
1757 +**Acceptance Criteria:**
1758 +
1759 +* ✅ All evidence URLs auto-archived
1760 +* ✅ Archive links displayed to users
1761 +* ✅ Fallback to archive if original unavailable
1762 +* ✅ API rate limits respected
1763 +* ✅ Archive status visible in evidence display
1764 +
1765 +== Category 4: Community Safety ==
1766 +
1767 + FR48: Contributor Safety Framework ===
1768 +
1769 +**Importance:** CRITICAL
1770 +**Fulfills:** UN-28 (Safe contribution environment)
1771 +
1772 +**Purpose:** Protect contributors from harassment, doxxing, and coordinated attacks.
1773 +
1774 +**Specification:**
1775 +
1776 +* \\
1777 +** \\
1778 +**1. Privacy Protection:
1779 +
1780 +* **Optional Pseudonymity:** Contributors can use pseudonyms
1781 +* **Email Privacy:** Emails never displayed publicly
1782 +* **Profile Privacy:** Contributors control what's public
1783 +* **IP Logging:** Only for abuse prevention, not public
1784 +
1785 +**2. Harassment Prevention:**
1786 +
1787 +* **Automated Toxicity Detection:** Flag abusive comments
1788 +* **Personal Information Detection:** Auto-block doxxing attempts
1789 +* **Coordinated Attack Detection:** Identify brigading patterns
1790 +* **Rapid Response:** Moderator alerts for harassment
1791 +
1792 +**3. Safety Features:**
1793 +
1794 +* **Block Users:** Contributors can block harassers
1795 +* **Private Contributions:** Option to contribute anonymously
1796 +* **Report Harassment:** One-click harassment reporting
1797 +* **Safety Resources:** Links to support resources
1798 +
1799 +**4. Moderator Tools:**
1800 +
1801 +* **Quick Ban:** Immediately block abusers
1802 +* **Pattern Detection:** Identify coordinated attacks
1803 +* **Appeal Process:** Fair review of moderation actions
1804 +* **Escalation:** Serious threats escalated to authorities
1805 +
1806 +**5. Trusted Contributor Protection:**
1807 +
1808 +* **Enhanced Privacy:** Additional protection for high-profile contributors
1809 +* **Verification:** Optional identity verification (not public)
1810 +* **Legal Support:** Resources for contributors facing legal threats
1811 +
1812 +**Acceptance Criteria:**
1813 +
1814 +* ✅ Pseudonyms supported
1815 +* ✅ Toxicity detection active
1816 +* ✅ Doxxing auto-blocked
1817 +* ✅ Harassment reporting functional
1818 +* ✅ Moderator tools implemented
1819 +* ✅ Safety policy published
1820 +
1821 +== Category 5: Continuous Improvement ==
1822 +
1823 + FR49: A/B Testing Framework ===
1824 +
1825 +**Importance:** CRITICAL
1826 +**Fulfills:** Continuous system improvement
1827 +
1828 +**Purpose:** Test and measure improvements to AKEL prompts, algorithms, and workflows.
1829 +
1830 +**Specification:**
1831 +
1832 +**Test Capabilities:**
1833 +
1834 +1. **Prompt Variations:**
1835 +
1836 +* Test different claim extraction prompts
1837 +* Test different verdict generation prompts
1838 +* Measure: Accuracy, clarity, completeness
1839 +
1840 +2. **Algorithm Variations:**
1841 +
1842 +* Test different source scoring algorithms
1843 +* Test different confidence calculations
1844 +* Measure: Audit accuracy, user satisfaction
1845 +
1846 +3. **Workflow Variations:**
1847 +
1848 +* Test different quality gate thresholds
1849 +* Test different risk tier assignments
1850 +* Measure: Publication rate, quality scores
1851 +
1852 +**Implementation:**
1853 +
1854 +* **Traffic Split:** 50/50 or 90/10 splits
1855 +* **Randomization:** Consistent per claim (not per user)
1856 +* **Metrics Collection:** Automatic for all variants
1857 +* **Statistical Significance:** Minimum sample size calculation
1858 +* **Rollout:** Winner promoted to 100% traffic
1859 +
1860 +**A/B Test Workflow:**
1861 +
1862 +{{code}}
1863 +1. Hypothesis: "New prompt improves claim extraction"
1864 +2. Design test: Control vs. Variant
1865 +3. Define metrics: Extraction accuracy, completeness
1866 +4. Run test: 7-14 days, minimum 100 claims each
1867 +5. Analyze results: Statistical significance?
1868 +6. Decision: Deploy winner or iterate
1869 +{{/code}}
1870 +
1871 +**Acceptance Criteria:**
1872 +
1873 +* ✅ A/B testing framework implemented
1874 +* ✅ Can test prompt variations
1875 +* ✅ Can test algorithm variations
1876 +* ✅ Metrics automatically collected
1877 +* ✅ Statistical significance calculated
1878 +* ✅ Results inform system improvements
1879 +
1880 +=== FR54: Evidence Deduplication ===
1881 +
1882 +**Importance:** CRITICAL (POC2/Beta)
1883 +**Fulfills:** Accurate evidence counting, quality metrics
1884 +
1885 +**Purpose:** Avoid counting the same source multiple times when it appears in different forms.
1886 +
1887 +**Specification:**
1888 +
1889 +**Deduplication Logic:**
1890 +
1891 +1. **URL Normalization:**
1892 +
1893 +* Remove tracking parameters (?utm_source=...)
1894 +* Normalize http/https
1895 +* Normalize www/non-www
1896 +* Handle redirects
1897 +
1898 +2. **Content Similarity:**
1899 +
1900 +* If two sources have >90% text similarity → Same source
1901 +* If one is subset of other → Same source
1902 +* Use fuzzy matching for minor differences
1903 +
1904 +3. **Cross-Domain Syndication:**
1905 +
1906 +* Detect wire service content (AP, Reuters)
1907 +* Mark as single source if syndicated
1908 +* Count original publication only
1909 +
1910 +**Display:**
1911 +
1912 +{{code}}
1913 +Evidence Sources (3 unique, 5 total):
1914 +
1915 +1. Original Article (NYTimes)
1916 + - Also appeared in: WashPost, Guardian (syndicated)
1917 +
1918 +2. Research Paper (Nature)
1919 +
1920 +3. Official Statement (WHO)
1921 +{{/code}}
1922 +
1923 +**Acceptance Criteria:**
1924 +
1925 +* ✅ URL normalization works
1926 +* ✅ Content similarity detected
1927 +* ✅ Syndicated content identified
1928 +* ✅ Unique vs. total counts accurate
1929 +* ✅ Improves evidence quality metrics
1930 +
1931 +== Additional Requirements (Lower Importance) ==
1932 +
1933 + FR50: OSINT Toolkit Integration ===
1934 +
1935 +**Fulfills:** Advanced media verification
1936 +
1937 +**Purpose:** Integrate open-source intelligence tools for advanced verification.
1938 +
1939 +**Tools to Integrate:**
1940 +
1941 +* InVID/WeVerify (video verification)
1942 +* Bellingcat toolkit
1943 +* Additional TBD based on V1.0 learnings
1944 +
1945 +=== FR51: Video Verification System ===
1946 +
1947 +**Fulfills:** UN-27 (Visual claims), advanced media verification
1948 +
1949 +**Purpose:** Verify video-based claims.
1950 +
1951 +**Specification:**
1952 +
1953 +* Keyframe extraction
1954 +* Reverse video search
1955 +* Deepfake detection (AI-powered)
1956 +* Metadata analysis
1957 +* Acoustic signature analysis
1958 +
1959 +=== FR52: Interactive Detection Training ===
1960 +
1961 +**Fulfills:** Media literacy education
1962 +
1963 +**Purpose:** Teach users to identify misinformation.
1964 +
1965 +**Specification:**
1966 +
1967 +* Interactive tutorials
1968 +* Practice exercises
1969 +* Detection quizzes
1970 +* Gamification elements
1971 +
1972 +=== FR53: Cross-Organizational Sharing ===
1973 +
1974 +**Fulfills:** Collaboration with other fact-checkers
1975 +
1976 +**Purpose:** Share findings with IFCN/EFCSN members.
1977 +
1978 +**Specification:**
1979 +
1980 +* API for fact-checking organizations
1981 +* Structured data exchange
1982 +* Privacy controls
1983 +* Attribution requirements
1984 +
1985 +== Summary ==
1986 +
1987 +**V1.0 Critical Requirements (Must Have):**
1988 +
1989 +* FR44: ClaimReview Schema ✅
1990 +* FR45: Corrections Notification ✅
1991 +* FR46: Image Verification ✅
1992 +* FR47: Archive.org Integration ✅
1993 +* FR48: Contributor Safety ✅
1994 +* FR49: A/B Testing ✅
1995 +* FR54: Evidence Deduplication ✅
1996 +* NFR11: Quality Assurance Framework ✅
1997 +* NFR12: Security Controls ✅
1998 +* NFR13: Quality Metrics Dashboard ✅
1999 +
2000 +**V1.1+ (Future):**
2001 +
2002 +* FR50: OSINT Integration
2003 +* FR51: Video Verification
2004 +* FR52: Detection Training
2005 +* FR53: Cross-Org Sharing
2006 +
2007 +**Total:** 11 critical requirements for V1.0
2008 +
2009 +=== FR54: Evidence Deduplication ===
2010 +
2011 +**Fulfills:** Accurate evidence counting, quality metrics
2012 +
2013 +**Purpose:** Avoid counting the same source multiple times when it appears in different forms.
2014 +
2015 +**Specification:**
2016 +
2017 +**Deduplication Logic:**
2018 +
2019 +1. **URL Normalization:**
2020 +
2021 +* Remove tracking parameters (?utm_source=...)
2022 +* Normalize http/https
2023 +* Normalize www/non-www
2024 +* Handle redirects
2025 +
2026 +2. **Content Similarity:**
2027 +
2028 +* If two sources have >90% text similarity → Same source
2029 +* If one is subset of other → Same source
2030 +* Use fuzzy matching for minor differences
2031 +
2032 +3. **Cross-Domain Syndication:**
2033 +
2034 +* Detect wire service content (AP, Reuters)
2035 +* Mark as single source if syndicated
2036 +* Count original publication only
2037 +
2038 +**Display:**
2039 +
2040 +{{code}}
2041 +Evidence Sources (3 unique, 5 total):
2042 +
2043 +1. Original Article (NYTimes)
2044 + - Also appeared in: WashPost, Guardian (syndicated)
2045 +
2046 +2. Research Paper (Nature)
2047 +
2048 +3. Official Statement (WHO)
2049 +{{/code}}
2050 +
2051 +**Acceptance Criteria:**
2052 +
2053 +* ✅ URL normalization works
2054 +* ✅ Content similarity detected
2055 +* ✅ Syndicated content identified
2056 +* ✅ Unique vs. total counts accurate
2057 +* ✅ Improves evidence quality metrics
2058 +
2059 +== Additional Requirements (Lower Importance) ==
2060 +
2061 + FR7: Automated Verdicts (Enhanced with Quality Gates) ===
2062 +
2063 +**POC1+ Enhancement:**
2064 +
2065 +After AKEL generates verdict, it passes through quality gates:
2066 +
2067 +{{code}}
2068 +Workflow:
2069 +1. Extract claims
2070 + ↓
2071 +2. [GATE 1] Validate fact-checkable
2072 + ↓
2073 +3. Generate scenarios
2074 + ↓
2075 +4. Generate verdicts
2076 + ↓
2077 +5. [GATE 4] Validate confidence
2078 + ↓
2079 +6. Display to user
2080 +{{/code}}
2081 +
2082 +**Updated Verdict States:**
2083 +
2084 +* PUBLISHED
2085 +* INSUFFICIENT_EVIDENCE
2086 +* NON_FACTUAL_CLAIM
2087 +* PROCESSING
2088 +* ERROR
2089 +
2090 +=== FR4: Analysis Summary (Enhanced with Quality Metadata) ===
2091 +
2092 +**POC1+ Enhancement:**
2093 +
2094 +Display quality indicators:
2095 +
2096 +{{code}}
2097 +Analysis Summary:
2098 + Verifiable Claims: 3/5
2099 + High Confidence Verdicts: 1
2100 + Medium Confidence: 2
2101 + Evidence Sources: 12
2102 + Avg Source Quality: 0.73
2103 + Quality Score: 8.5/10
2104 +{{/code}}