Changes for page Requirements
Last modified by Robert Schaub on 2026/02/08 21:32
Summary
-
Page properties (1 modified, 0 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -1,22 +1,61 @@ 1 1 = Requirements = 2 -This page defines **Roles**, **Content States**, **Rules**, and **System Principles** for FactHarbor. 2 + 3 +{{info}} 4 +**Phase Assignments:** See [[Requirements Roadmap Matrix>>Archive.FactHarbor.Roadmap.Requirements-Roadmap-Matrix.WebHome]] for which requirements are implemented in which phases. 5 +{{/info}} 6 + 7 +**This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.** 8 + 3 3 **Core Philosophy:** Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically. 10 + 11 +== Navigation == 12 + 13 +* **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need from FactHarbor (drives these requirements) 14 +* **This page** - How we fulfill those needs through system design 15 + 16 +(% class="box infomessage" %) 17 +((( 18 +**How to read this page:** 19 + 20 +1. **User Needs drive Requirements**: See [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] for what users need 21 +2. **Requirements define implementation**: This page shows how we fulfill those needs 22 +3. **Functional Requirements (FR)**: Specific features and capabilities 23 +4. **Non-Functional Requirements (NFR)**: Quality attributes (performance, security, etc.) 24 + 25 +Each requirement references which User Needs it fulfills. 26 +))) 27 + 4 4 == 1. Roles == 29 + 30 +**Fulfills**: UN-12 (Submit claims), UN-13 (Cite verdicts), UN-14 (API access) 31 + 5 5 FactHarbor uses three simple roles plus a reputation system. 33 + 6 6 === 1.1 Reader === 35 + 7 7 **Who**: Anyone (no login required) 37 + 8 8 **Can**: 39 + 9 9 * Browse and search claims 10 10 * View scenarios, evidence, verdicts, and confidence scores 11 11 * Flag issues or errors 12 12 * Use filters, search, and visualization tools 13 13 * Submit claims automatically (new claims added if not duplicates) 45 + 14 14 **Cannot**: 47 + 15 15 * Modify content 16 16 * Access edit history details 50 + 51 +**User Needs served**: UN-1 (Trust assessment), UN-2 (Claim verification), UN-3 (Article summary with FactHarbor analysis summary), UN-4 (Social media fact-checking), UN-5 (Source tracing), UN-7 (Evidence transparency), UN-8 (Understanding disagreement), UN-12 (Submit claims), UN-17 (In-article highlighting) 52 + 17 17 === 1.2 Contributor === 54 + 18 18 **Who**: Registered users (earns reputation through contributions) 56 + 19 19 **Can**: 58 + 20 20 * Everything a Reader can do 21 21 * Edit claims, evidence, and scenarios 22 22 * Add sources and citations ... ... @@ -23,18 +23,28 @@ 23 23 * Suggest improvements to AI-generated content 24 24 * Participate in discussions 25 25 * Earn reputation points for quality contributions 65 + 26 26 **Reputation System**: 67 + 27 27 * New contributors: Limited edit privileges 28 28 * Established contributors (established reputation): Full edit access 29 29 * Trusted contributors (substantial reputation): Can approve certain changes 30 30 * Reputation earned through: Accepted edits, helpful flags, quality contributions 31 31 * Reputation lost through: Reverted edits, invalid flags, abuse 73 + 32 32 **Cannot**: 75 + 33 33 * Delete or hide content (only moderators) 34 34 * Override moderation decisions 78 + 79 +**User Needs served**: UN-13 (Cite and contribute) 80 + 35 35 === 1.3 Moderator === 82 + 36 36 **Who**: Trusted community members with proven track record, appointed by governance board 84 + 37 37 **Can**: 86 + 38 38 * Review flagged content 39 39 * Hide harmful or abusive content 40 40 * Resolve disputes between contributors ... ... @@ -41,34 +41,56 @@ 41 41 * Issue warnings or temporary bans 42 42 * Make final decisions on content disputes 43 43 * Access full audit logs 93 + 44 44 **Cannot**: 95 + 45 45 * Change governance rules 46 46 * Permanently ban users without board approval 47 47 * Override technical quality gates 99 + 48 48 **Note**: Small team (3-5 initially), supported by automated moderation tools. 101 + 49 49 === 1.4 Domain Trusted Contributors (Optional, Task-Specific) === 103 + 50 50 **Who**: Subject matter specialists invited for specific high-stakes disputes 105 + 51 51 **Not a permanent role**: Contacted externally when needed for contested claims in their domain 107 + 52 52 **When used**: 109 + 53 53 * Medical claims with life/safety implications 54 54 * Legal interpretations with significant impact 55 55 * Scientific claims with high controversy 56 56 * Technical claims requiring specialized knowledge 114 + 57 57 **Process**: 116 + 58 58 * Moderator identifies need for expert input 59 59 * Contact expert externally (don't require them to be users) 60 60 * Trusted Contributor provides written opinion with sources 61 61 * Opinion added to claim record 62 62 * Trusted Contributor acknowledged in claim 122 + 123 +**User Needs served**: UN-16 (Expert validation status) 124 + 63 63 == 2. Content States == 126 + 127 +**Fulfills**: UN-1 (Trust indicators), UN-16 (Review status transparency) 128 + 64 64 FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping. 130 + 65 65 === 2.1 Published === 132 + 66 66 **Status**: Visible to all users 134 + 67 67 **Includes**: 136 + 68 68 * AI-generated analyses (default state) 69 69 * User-contributed content 70 70 * Edited/improved content 140 + 71 71 **Quality Indicators** (displayed with content): 142 + 72 72 * **Confidence Score**: 0-100% (AI's confidence in analysis) 73 73 * **Source Quality Score**: 0-100% (based on source track record) 74 74 * **Controversy Flag**: If high dispute/edit activity ... ... @@ -75,14 +75,23 @@ 75 75 * **Completeness Score**: % of expected fields filled 76 76 * **Last Updated**: Date of most recent change 77 77 * **Edit Count**: Number of revisions 149 +* **Review Status**: AI-generated / Human-reviewed / Expert-validated 150 + 78 78 **Automatic Warnings**: 152 + 79 79 * Confidence < 60%: "Low confidence - use caution" 80 80 * Source quality < 40%: "Sources may be unreliable" 81 81 * High controversy: "Disputed - multiple interpretations exist" 82 82 * Medical/Legal/Safety domain: "Seek professional advice" 157 + 158 +**User Needs served**: UN-1 (Trust score), UN-9 (Methodology transparency), ~~UN-15 (Evolution timeline - Deferred)~~, UN-16 (Review status) 159 + 83 83 === 2.2 Hidden === 161 + 84 84 **Status**: Not visible to regular users (only to moderators) 163 + 85 85 **Reasons**: 165 + 86 86 * Spam or advertising 87 87 * Personal attacks or harassment 88 88 * Illegal content ... ... @@ -89,22 +89,32 @@ 89 89 * Privacy violations 90 90 * Deliberate misinformation (verified) 91 91 * Abuse or harmful content 172 + 92 92 **Process**: 174 + 93 93 * Automated detection flags for moderator review 94 94 * Moderator confirms and hides 95 95 * Original author notified with reason 96 96 * Can appeal to board if disputes moderator decision 179 + 97 97 **Note**: Content is hidden, not deleted (for audit trail) 181 + 98 98 == 3. Contribution Rules == 183 + 99 99 === 3.1 All Contributors Must === 185 + 100 100 * Provide sources for factual claims 101 101 * Use clear, neutral language in FactHarbor's own summaries 102 102 * Respect others and maintain civil discourse 103 103 * Accept community feedback constructively 104 104 * Focus on improving quality, not protecting ego 191 + 105 105 === 3.2 AKEL (AI System) === 193 + 106 106 **AKEL is the primary system**. Human contributions supplement and train AKEL. 195 + 107 107 **AKEL Must**: 197 + 108 108 * Mark all outputs as AI-generated 109 109 * Display confidence scores prominently 110 110 * Provide source citations ... ... @@ -111,86 +111,139 @@ 111 111 * Flag uncertainty clearly 112 112 * Identify contradictions in evidence 113 113 * Learn from human corrections 204 + 114 114 **When AKEL Makes Errors**: 206 + 115 115 1. Capture the error pattern (what, why, how common) 116 116 2. Improve the system (better prompt, model, validation) 117 117 3. Re-process affected claims automatically 118 118 4. Measure improvement (did quality increase?) 211 + 119 119 **Human Role**: Train AKEL through corrections, not replace AKEL 213 + 120 120 === 3.3 Contributors Should === 215 + 121 121 * Improve clarity and structure 122 122 * Add missing sources 123 123 * Flag errors for system improvement 124 124 * Suggest better ways to present information 125 125 * Participate in quality discussions 221 + 126 126 === 3.4 Moderators Must === 223 + 127 127 * Be impartial 128 128 * Document moderation decisions 129 129 * Respond to appeals promptly 130 130 * Use automated tools to scale efforts 131 131 * Focus on abuse/harm, not routine quality control 229 + 132 132 == 4. Quality Standards == 231 + 232 +**Fulfills**: UN-5 (Source reliability), UN-6 (Publisher track records), UN-7 (Evidence transparency), UN-9 (Methodology transparency) 233 + 133 133 === 4.1 Source Requirements === 235 + 134 134 **Track Record Over Credentials**: 237 + 135 135 * Sources evaluated by historical accuracy 136 136 * Correction policy matters 137 137 * Independence from conflicts of interest 138 138 * Methodology transparency 242 + 139 139 **Source Quality Database**: 244 + 140 140 * Automated tracking of source accuracy 141 141 * Correction frequency 142 142 * Reliability score (updated continuously) 143 143 * Users can see source track record 249 + 144 144 **No automatic trust** for government, academia, or media - all evaluated by track record. 251 + 252 +**User Needs served**: UN-5 (Source provenance), UN-6 (Publisher reliability) 253 + 145 145 === 4.2 Claim Requirements === 255 + 146 146 * Clear subject and assertion 147 147 * Verifiable with available information 148 148 * Sourced (or explicitly marked as needing sources) 149 149 * Neutral language in FactHarbor summaries 150 150 * Appropriate context provided 261 + 262 +**User Needs served**: UN-2 (Claim extraction and verification) 263 + 151 151 === 4.3 Evidence Requirements === 265 + 152 152 * Publicly accessible (or explain why not) 153 153 * Properly cited with attribution 154 154 * Relevant to claim being evaluated 155 155 * Original source preferred over secondary 270 + 271 +**User Needs served**: UN-7 (Evidence transparency) 272 + 156 156 === 4.4 Confidence Scoring === 274 + 157 157 **Automated confidence calculation based on**: 276 + 158 158 * Source quality scores 159 159 * Evidence consistency 160 160 * Contradiction detection 161 161 * Completeness of analysis 162 162 * Historical accuracy of similar claims 282 + 163 163 **Thresholds**: 284 + 164 164 * < 40%: Too low to publish (needs improvement) 165 165 * 40-60%: Published with "Low confidence" warning 166 166 * 60-80%: Published as standard 167 167 * 80-100%: Published as "High confidence" 289 + 290 +**User Needs served**: UN-1 (Trust assessment), UN-9 (Methodology transparency) 291 + 168 168 == 5. Automated Risk Scoring == 293 + 294 +**Fulfills**: UN-10 (Manipulation detection), UN-16 (Appropriate review level) 295 + 169 169 **Replace manual risk tiers with continuous automated scoring**. 297 + 170 170 === 5.1 Risk Score Calculation === 299 + 171 171 **Factors** (weighted algorithm): 301 + 172 172 * **Domain sensitivity**: Medical, legal, safety auto-flagged higher 173 173 * **Potential impact**: Views, citations, spread 174 174 * **Controversy level**: Flags, disputes, edit wars 175 175 * **Uncertainty**: Low confidence, contradictory evidence 176 176 * **Source reliability**: Track record of sources used 307 + 177 177 **Score**: 0-100 (higher = more risk) 309 + 178 178 === 5.2 Automated Actions === 311 + 179 179 * **Score > 80**: Flag for moderator review before publication 180 180 * **Score 60-80**: Publish with prominent warnings 181 181 * **Score 40-60**: Publish with standard warnings 182 182 * **Score < 40**: Publish normally 316 + 183 183 **Continuous monitoring**: Risk score recalculated as new information emerges 318 + 319 +**User Needs served**: UN-10 (Detect manipulation tactics), UN-16 (Review status) 320 + 184 184 == 6. System Improvement Process == 322 + 185 185 **Core principle**: Fix the system, not just the data. 324 + 186 186 === 6.1 Error Capture === 326 + 187 187 **When users flag errors or make corrections**: 328 + 188 188 1. What was wrong? (categorize) 189 189 2. What should it have been? 190 190 3. Why did the system fail? (root cause) 191 191 4. How common is this pattern? 192 192 5. Store in ErrorPattern table (improvement queue) 193 -=== 6.2 Weekly Improvement Cycle === 334 + 335 +=== 6.2 Continuous Improvement Cycle === 336 + 194 194 1. **Review**: Analyze top error patterns 195 195 2. **Develop**: Create fix (prompt, model, validation) 196 196 3. **Test**: Validate fix on sample claims ... ... @@ -197,8 +197,11 @@ 197 197 4. **Deploy**: Roll out if quality improves 198 198 5. **Re-process**: Automatically update affected claims 199 199 6. **Monitor**: Track quality metrics 343 + 200 200 === 6.3 Quality Metrics Dashboard === 345 + 201 201 **Track continuously**: 347 + 202 202 * Error rate by category 203 203 * Source quality distribution 204 204 * Confidence score trends ... ... @@ -206,158 +206,1753 @@ 206 206 * Correction acceptance rate 207 207 * Re-work rate 208 208 * Claims processed per hour 209 -**Goal**: 10% monthly improvement in error rate 355 + 356 +**Goal**: continuous improvement in error rate 357 + 210 210 == 7. Automated Quality Monitoring == 359 + 211 211 **Replace manual audit sampling with automated monitoring**. 361 + 212 212 === 7.1 Continuous Metrics === 363 + 213 213 * **Source quality**: Track record database 214 214 * **Consistency**: Contradiction detection 215 215 * **Clarity**: Readability scores 216 216 * **Completeness**: Field validation 217 217 * **Accuracy**: User corrections tracked 369 + 218 218 === 7.2 Anomaly Detection === 371 + 219 219 **Automated alerts for**: 373 + 220 220 * Sudden quality drops 221 221 * Unusual patterns 222 222 * Contradiction clusters 223 223 * Source reliability changes 224 224 * User behavior anomalies 379 + 225 225 === 7.3 Targeted Review === 381 + 226 226 * Review only flagged items 227 227 * Random sampling for calibration (not quotas) 228 228 * Learn from corrections to improve automation 229 -== 8. Claim Intake & Normalization == 230 -=== 8.1 FR1 – Claim Intake === 385 + 386 +== 8. Functional Requirements == 387 + 388 +This section defines specific features that fulfill user needs. 389 + 390 +=== 8.1 Claim Intake & Normalization === 391 + 392 +==== FR1 — Claim Intake ==== 393 + 394 +**Fulfills**: UN-2 (Claim extraction), UN-4 (Quick fact-checking), UN-12 (Submit claims) 395 + 231 231 * Users submit claims via simple form or API 232 232 * Claims can be text, URL, or image 233 233 * Duplicate detection (semantic similarity) 234 234 * Auto-categorization by domain 235 -=== 8.2 FR2 – Claim Normalization === 400 + 401 +==== FR2 — Claim Normalization ==== 402 + 403 +**Fulfills**: UN-2 (Claim verification) 404 + 236 236 * Standardize to clear assertion format 237 237 * Extract key entities (who, what, when, where) 238 238 * Identify claim type (factual, predictive, evaluative) 239 239 * Link to existing similar claims 240 -=== 8.3 FR3 – Claim Classification === 409 + 410 +==== FR3 — Claim Classification ==== 411 + 412 +**Fulfills**: UN-11 (Filtered research) 413 + 241 241 * Domain: Politics, Science, Health, etc. 242 242 * Type: Historical fact, current stat, prediction, etc. 243 243 * Risk score: Automated calculation 244 244 * Complexity: Simple, moderate, complex 245 -== 9. Scenario System == 246 -=== 9.1 FR4 – Scenario Generation === 418 + 419 +=== 8.2 Scenario System === 420 + 421 +==== FR4 — Scenario Generation ==== 422 + 423 +**Fulfills**: UN-2 (Context-dependent verification), UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement) 424 + 247 247 **Automated scenario creation**: 248 -* AKEL analyzes claim and generates likely scenarios 249 -* Each scenario includes: assumptions, evidence, conclusion 426 + 427 +* AKEL analyzes claim and generates likely scenarios (use-cases and contexts) 428 +* Each scenario includes: assumptions, definitions, boundaries, evidence context 250 250 * Users can flag incorrect scenarios 251 251 * System learns from corrections 252 -=== 9.2 FR5 – Evidence Linking === 431 + 432 +**Key Concept**: Scenarios represent different interpretations or contexts (e.g., "Clinical trials with healthy adults" vs. "Real-world data with diverse populations") 433 + 434 +==== FR5 — Evidence Linking ==== 435 + 436 +**Fulfills**: UN-5 (Source tracing), UN-7 (Evidence transparency) 437 + 253 253 * Automated evidence discovery from sources 254 254 * Relevance scoring 255 255 * Contradiction detection 256 256 * Source quality assessment 257 -=== 9.3 FR6 – Scenario Comparison === 442 + 443 +==== FR6 — Scenario Comparison ==== 444 + 445 +**Fulfills**: UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement) 446 + 258 258 * Side-by-side comparison interface 259 -* Highlight key differences 260 -* Show evidence supporting each 261 -* Display confidence scores 262 -== 10. Verdicts & Analysis == 263 -=== 10.1 FR7 – Automated Verdicts === 264 -* AKEL generates verdict based on evidence 448 +* Highlight key differences between scenarios 449 +* Show evidence supporting each scenario 450 +* Display confidence scores per scenario 451 + 452 +=== 8.3 Verdicts & Analysis === 453 + 454 +==== FR7 — Automated Verdicts ==== 455 + 456 +**Fulfills**: UN-1 (Trust score), UN-2 (Verification verdicts), UN-3 (Article summary with FactHarbor analysis summary), UN-13 (Cite verdicts) 457 + 458 +* AKEL generates verdict based on evidence within each scenario 459 +* **Likelihood range** displayed (e.g., "0.70-0.85 (likely true)") - NOT binary true/false 460 +* **Uncertainty factors** explicitly listed (e.g., "Small sample sizes", "Long-term effects unknown") 265 265 * Confidence score displayed prominently 266 -* Source quality indicators 462 +* Source quality indicators shown 267 267 * Contradictions noted 268 268 * Uncertainty acknowledged 269 -=== 10.2 FR8 – Time Evolution === 270 -* Claims update as new evidence emerges 271 -* Version history maintained 465 + 466 +**Key Innovation**: Detailed probabilistic verdicts with explicit uncertainty, not binary judgments 467 + 468 +==== FR8 — Time Evolution ==== 469 + 470 +{{warning}} 471 +**Status:** Deferred (Not in V1.0) 472 + 473 +This requirement has been **dropped from the current architecture and design**. Versioned entities have been replaced with simple edit history tracking only. Full evolution timeline functionality is deferred to future releases beyond V1.0. 474 +{{/warning}} 475 + 476 +**Fulfills**: UN-15 (Verdict evolution timeline) 477 + 478 +* Claims and verdicts update as new evidence emerges 479 +* Version history maintained for all verdicts 272 272 * Changes highlighted 273 273 * Confidence score trends visible 274 -== 11. Workflow & Moderation == 275 -=== 11.1 FR9 – Publication Workflow === 482 +* Users can see "as of date X, what did we know?" 483 + 484 +=== 8.4 User Interface & Presentation === 485 + 486 +==== FR12 — Two-Panel Summary View (Article Summary with FactHarbor Analysis Summary) ==== 487 + 488 +**Fulfills**: UN-3 (Article Summary with FactHarbor Analysis Summary) 489 + 490 +**Purpose**: Provide side-by-side comparison of what a document claims vs. FactHarbor's complete analysis of its credibility 491 + 492 +**Left Panel: Article Summary**: 493 + 494 +* Document title, source, and claimed credibility 495 +* "The Big Picture" - main thesis or position change 496 +* "Key Findings" - structured summary of document's main claims 497 +* "Reasoning" - document's explanation for positions 498 +* "Conclusion" - document's bottom line 499 + 500 +**Right Panel: FactHarbor Analysis Summary**: 501 + 502 +* FactHarbor's independent source credibility assessment 503 +* Claim-by-claim verdicts with confidence scores 504 +* Methodology assessment (strengths, limitations) 505 +* Overall verdict on document quality 506 +* Analysis ID for reference 507 + 508 +**Design Principles**: 509 + 510 +* No scrolling required - both panels visible simultaneously 511 +* Visual distinction between "what they say" and "FactHarbor's analysis" 512 +* Color coding for verdicts (supported, uncertain, refuted) 513 +* Confidence percentages clearly visible 514 +* Mobile responsive (panels stack vertically on small screens) 515 + 516 +**Implementation Notes**: 517 + 518 +* Generated automatically by AKEL for every analyzed document 519 +* Updates when verdict evolves (maintains version history) 520 +* Exportable as standalone summary report 521 +* Shareable via permanent URL 522 + 523 +==== FR13 — In-Article Claim Highlighting ==== 524 + 525 +**Fulfills**: UN-17 (In-article claim highlighting) 526 + 527 +**Purpose**: Enable readers to quickly assess claim credibility while reading by visually highlighting factual claims with color-coded indicators 528 + 529 +==== Visual Example: Article with Highlighted Claims ==== 530 + 531 +(% class="box" %) 532 +((( 533 +**Article: "New Study Shows Benefits of Mediterranean Diet"** 534 + 535 +A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet. 536 + 537 +(% class="box successmessage" style="margin:10px 0;" %) 538 +((( 539 +🟢 **Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups** 540 + 541 +(% style="font-size:0.9em; color:#666;" %) 542 +↑ WELL SUPPORTED • 87% confidence 543 +[[Click for evidence details →]] 544 + 545 + 546 +))) 547 + 548 +The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers. 549 + 550 +(% class="box warningmessage" style="margin:10px 0;" %) 551 +((( 552 +🟡 **Some experts believe this diet can completely prevent heart attacks** 553 + 554 +(% style="font-size:0.9em; color:#666;" %) 555 +↑ UNCERTAIN • 45% confidence 556 +Overstated - evidence shows risk reduction, not prevention 557 +[[Click for details →]] 558 + 559 + 560 +))) 561 + 562 +Dr. Maria Rodriguez, lead researcher, recommends incorporating more olive oil, fish, and vegetables into daily meals. 563 + 564 +(% class="box errormessage" style="margin:10px 0;" %) 565 +((( 566 +🔴 **The study proves that saturated fats cause heart disease** 567 + 568 +(% style="font-size:0.9em; color:#666;" %) 569 +↑ REFUTED • 15% confidence 570 +Claim not supported by study design; correlation ≠ causation 571 +[[Click for counter-evidence →]] 572 + 573 + 574 +))) 575 + 576 +Participants also reported feeling more energetic and experiencing better sleep quality, though these were secondary measures. 577 +))) 578 + 579 +**Legend:** 580 + 581 +* 🟢 = Well-supported claim (confidence ≥75%) 582 +* 🟡 = Uncertain claim (confidence 40-74%) 583 +* 🔴 = Refuted/unsupported claim (confidence <40%) 584 +* Plain text = Non-factual content (context, opinions, recommendations) 585 + 586 +==== Tooltip on Hover/Click ==== 587 + 588 +(% class="box infomessage" %) 589 +((( 590 +**FactHarbor Analysis** 591 + 592 +**Claim:** 593 +"Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease" 594 + 595 +**Verdict:** WELL SUPPORTED 596 +**Confidence:** 87% 597 + 598 +**Evidence Summary:** 599 + 600 +* Meta-analysis of 12 RCTs confirms 23-28% risk reduction 601 +* Consistent findings across multiple populations 602 +* Published in peer-reviewed journal (high credibility) 603 + 604 +**Uncertainty Factors:** 605 + 606 +* Exact percentage varies by study (20-30% range) 607 + 608 +[[View Full Analysis →]] 609 +))) 610 + 611 +**Color-Coding System**: 612 + 613 +* **Green**: Well-supported claims (confidence ≥75%, strong evidence) 614 +* **Yellow/Orange**: Uncertain claims (confidence 40-74%, conflicting or limited evidence) 615 +* **Red**: Refuted or unsupported claims (confidence <40%, contradicted by evidence) 616 +* **Gray/Neutral**: Non-factual content (opinions, questions, procedural text) 617 + 618 +==== Interactive Highlighting Example (Detailed View) ==== 619 + 620 +(% style="width:100%; border-collapse:collapse;" %) 621 +|=**Article Text**|=**Status**|=**Analysis** 622 +|((( 623 +A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet. 624 +)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Context - no highlighting 625 +|((( 626 +//Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups// 627 +)))|(% style="background-color:#D4EDDA; text-align:center; padding:8px;" %)🟢 **WELL SUPPORTED**|((( 628 +**87% confidence** 629 + 630 +Meta-analysis of 12 RCTs confirms 23-28% risk reduction 631 + 632 +[[View Full Analysis]] 633 +))) 634 +|((( 635 +The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers. 636 +)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Methodology - no highlighting 637 +|((( 638 +//Some experts believe this diet can completely prevent heart attacks// 639 +)))|(% style="background-color:#FFF3CD; text-align:center; padding:8px;" %)🟡 **UNCERTAIN**|((( 640 +**45% confidence** 641 + 642 +Overstated - evidence shows risk reduction, not prevention 643 + 644 +[[View Details]] 645 +))) 646 +|((( 647 +Dr. Rodriguez recommends incorporating more olive oil, fish, and vegetables into daily meals. 648 +)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Recommendation - no highlighting 649 +|((( 650 +//The study proves that saturated fats cause heart disease// 651 +)))|(% style="background-color:#F8D7DA; text-align:center; padding:8px;" %)🔴 **REFUTED**|((( 652 +**15% confidence** 653 + 654 +Claim not supported by study; correlation ≠ causation 655 + 656 +[[View Counter-Evidence]] 657 +))) 658 + 659 +**Design Notes:** 660 + 661 +* Highlighted claims use italics to distinguish from plain text 662 +* Color backgrounds match XWiki message box colors (success/warning/error) 663 +* Status column shows verdict prominently 664 +* Analysis column provides quick summary with link to details 665 + 666 +**User Actions**: 667 + 668 +* **Hover** over highlighted claim → Tooltip appears 669 +* **Click** highlighted claim → Detailed analysis modal/panel 670 +* **Toggle** button to turn highlighting on/off 671 +* **Keyboard**: Tab through highlighted claims 672 + 673 +**Interaction Design**: 674 + 675 +* Hover/click on highlighted claim → Show tooltip with: 676 +* Claim text 677 +* Verdict (e.g., "WELL SUPPORTED") 678 +* Confidence score (e.g., "85%") 679 +* Brief evidence summary 680 +* Link to detailed analysis 681 +* Toggle highlighting on/off (user preference) 682 +* Adjustable color intensity for accessibility 683 + 684 +**Technical Requirements**: 685 + 686 +* Real-time highlighting as page loads (non-blocking) 687 +* Claim boundary detection (start/end of assertion) 688 +* Handle nested or overlapping claims 689 +* Preserve original article formatting 690 +* Work with various content formats (HTML, plain text, PDFs) 691 + 692 +**Performance Requirements**: 693 + 694 +* Highlighting renders within 500ms of page load 695 +* No perceptible delay in reading experience 696 +* Efficient DOM manipulation (avoid reflows) 697 + 698 +**Accessibility**: 699 + 700 +* Color-blind friendly palette (use patterns/icons in addition to color) 701 +* Screen reader compatible (ARIA labels for claim credibility) 702 +* Keyboard navigation to highlighted claims 703 + 704 +**Implementation Notes**: 705 + 706 +* Claims extracted and analyzed by AKEL during initial processing 707 +* Highlighting data stored as annotations with byte offsets 708 +* Client-side rendering of highlights based on verdict data 709 +* Mobile responsive (tap instead of hover) 710 + 711 +=== 8.5 Workflow & Moderation === 712 + 713 +==== FR9 — Publication Workflow ==== 714 + 715 +**Fulfills**: UN-1 (Fast access to verified content), UN-16 (Clear review status) 716 + 276 276 **Simple flow**: 718 + 277 277 1. Claim submitted 278 278 2. AKEL processes (automated) 279 -3. If confidence > threshold: Publish 721 +3. If confidence > threshold: Publish (labeled as AI-generated) 280 280 4. If confidence < threshold: Flag for improvement 281 281 5. If risk score > threshold: Flag for moderator 724 + 282 282 **No multi-stage approval process** 283 -=== 11.2 FR10 – Moderation === 726 + 727 +==== FR10 — Moderation ==== 728 + 284 284 **Focus on abuse, not routine quality**: 730 + 285 285 * Automated abuse detection 286 286 * Moderators handle flags 287 287 * Quick response to harmful content 288 288 * Minimal involvement in routine content 289 -=== 11.3 FR11 – Audit Trail === 735 + 736 +==== FR11 — Audit Trail ==== 737 + 738 +**Fulfills**: UN-14 (API access to histories), UN-15 (Evolution tracking) 739 + 290 290 * All edits logged 291 291 * Version history public 292 292 * Moderation decisions documented 293 293 * System improvements tracked 294 -== 12. Technical Requirements == 295 -=== 12.1 NFR1 – Performance === 744 + 745 +== 9. Non-Functional Requirements == 746 + 747 +=== 9.1 NFR1 — Performance === 748 + 749 +**Fulfills**: UN-4 (Fast fact-checking), UN-11 (Responsive filtering) 750 + 296 296 * Claim processing: < 30 seconds 297 297 * Search response: < 2 seconds 298 298 * Page load: < 3 seconds 299 299 * 99% uptime 300 -=== 12.2 NFR2 – Scalability === 755 + 756 +=== 9.2 NFR2 — Scalability === 757 + 758 +**Fulfills**: UN-14 (API access at scale) 759 + 301 301 * Handle 10,000 claims initially 302 302 * Scale to 1M+ claims 303 303 * Support 100K+ concurrent users 304 304 * Automated processing scales linearly 305 -=== 12.3 NFR3 – Transparency === 764 + 765 +=== 9.3 NFR3 — Transparency === 766 + 767 +**Fulfills**: UN-7 (Evidence transparency), UN-9 (Methodology transparency), UN-13 (Citable verdicts), UN-15 (Evolution visibility) 768 + 306 306 * All algorithms open source 307 307 * All data exportable 308 308 * All decisions documented 309 309 * Quality metrics public 310 -=== 12.4 NFR4 – Security & Privacy === 773 + 774 +=== 9.4 NFR4 — Security & Privacy === 775 + 311 311 * Follow [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]] 312 312 * Secure authentication 313 313 * Data encryption 314 314 * Regular security audits 315 -=== 12.5 NFR5 – Maintainability === 780 + 781 +=== 9.5 NFR5 — Maintainability === 782 + 316 316 * Modular architecture 317 317 * Automated testing 318 318 * Continuous integration 319 319 * Comprehensive documentation 320 -== 13. MVP Scope == 321 -**Phase 1 (Months 1-3): Read-Only MVP** 322 -Build: 323 -* Automated claim analysis 324 -* Confidence scoring 325 -* Source evaluation 326 -* Browse/search interface 327 -* User flagging system 328 -**Goal**: Prove AI quality before adding user editing 329 -**Phase 2 (Months 4-6): User Contributions** 330 -Add only if needed: 331 -* Simple editing (Wikipedia-style) 332 -* Reputation system 333 -* Basic moderation 334 -**Phase 3 (Months 7-12): Refinement** 335 -* Continuous quality improvement 336 -* Feature additions based on real usage 337 -* Scale infrastructure 338 -**Deferred**: 339 -* Federation (until multiple successful instances exist) 340 -* Complex contribution workflows (focus on automation) 341 -* Extensive role hierarchy (keep simple) 342 -== 14. Success Metrics == 343 -**System Quality** (track weekly): 344 -* Error rate by category (target: -10%/month) 345 -* Average confidence score (target: increase) 346 -* Source quality distribution (target: more high-quality) 347 -* Contradiction detection rate (target: increase) 348 -**Efficiency** (track monthly): 349 -* Claims processed per hour (target: increase) 350 -* Human hours per claim (target: decrease) 351 -* Automation coverage (target: >90%) 352 -* Re-work rate (target: <5%) 353 -**User Satisfaction** (track quarterly): 354 -* User flag rate (issues found) 355 -* Correction acceptance rate (flags valid) 356 -* Return user rate 357 -* Trust indicators (surveys) 358 -== 15. Related Pages == 359 -* [[Architecture>>FactHarbor.Specification.Architecture.WebHome]] 360 -* [[Data Model>>FactHarbor.Specification.Data Model.WebHome]] 361 -* [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] 362 -* [[Global Rules>>FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]] 787 + 788 +=== NFR11: AKEL Quality Assurance Framework === 789 + 790 +**Fulfills:** AI safety, IFCN methodology transparency 791 + 792 +**Specification:** 793 + 794 +Multi-layer AI quality gates to detect hallucinations, low-confidence results, and logical inconsistencies. 795 + 796 +==== Quality Gate 1: Claim Extraction Validation ==== 797 + 798 +**Purpose:** Ensure extracted claims are factual assertions (not opinions/predictions) 799 + 800 +**Checks:** 801 + 802 +1. **Factual Statement Test:** Is this verifiable? (Yes/No) 803 +2. **Opinion Detection:** Contains hedging language? ("I think", "probably", "best") 804 +3. **Future Prediction Test:** Makes claims about future events? 805 +4. **Specificity Score:** Contains specific entities, numbers, dates? 806 + 807 +**Thresholds:** 808 + 809 +* Factual: Must be "Yes" 810 +* Opinion markers: <2 hedging phrases 811 +* Specificity: ≥3 specific elements 812 + 813 +**Action if Failed:** Flag as "Non-verifiable", do NOT generate verdict 814 + 815 +==== Quality Gate 2: Evidence Relevance Validation ==== 816 + 817 +**Purpose:** Ensure AI-linked evidence actually relates to claim 818 + 819 +**Checks:** 820 + 821 +1. **Semantic Similarity Score:** Evidence vs. claim (embeddings) 822 +2. **Entity Overlap:** Shared people/places/things? 823 +3. **Topic Relevance:** Discusses claim subject? 824 + 825 +**Thresholds:** 826 + 827 +* Similarity: ≥0.6 (cosine similarity) 828 +* Entity overlap: ≥1 shared entity 829 +* Topic relevance: ≥0.5 830 + 831 +**Action if Failed:** Discard irrelevant evidence 832 + 833 +==== Quality Gate 3: Scenario Coherence Check ==== 834 + 835 +**Purpose:** Validate scenario assumptions are logical and complete 836 + 837 +**Checks:** 838 + 839 +1. **Completeness:** All required fields populated 840 +2. **Internal Consistency:** Assumptions don't contradict 841 +3. **Distinguishability:** Scenarios meaningfully different 842 + 843 +**Thresholds:** 844 + 845 +* Required fields: 100% 846 +* Contradiction score: <0.3 847 +* Scenario similarity: <0.8 848 + 849 +**Action if Failed:** Merge duplicates, reduce confidence -20% 850 + 851 +==== Quality Gate 4: Verdict Confidence Assessment ==== 852 + 853 +**Purpose:** Only publish high-confidence verdicts 854 + 855 +**Checks:** 856 + 857 +1. **Evidence Count:** Minimum 2 sources 858 +2. **Source Quality:** Average reliability ≥0.6 859 +3. **Evidence Agreement:** Supporting vs. contradicting ≥0.6 860 +4. **Uncertainty Factors:** Hedging in reasoning 861 + 862 +**Confidence Tiers:** 863 + 864 +* **HIGH (80-100%):** ≥3 sources, ≥0.7 quality, ≥80% agreement 865 +* **MEDIUM (50-79%):** ≥2 sources, ≥0.6 quality, ≥60% agreement 866 +* **LOW (0-49%):** <2 sources OR low quality/agreement 867 +* **INSUFFICIENT:** <2 sources → DO NOT PUBLISH 868 + 869 +**Implementation Phases:** 870 + 871 +* **POC1:** Gates 1 & 4 only (basic validation) 872 +* **POC2:** All 4 gates (complete framework) 873 +* **V1.0:** Hardened with <5% hallucination rate 874 + 875 +**Acceptance Criteria:** 876 + 877 +* ✅ All gates operational 878 +* ✅ Hallucination rate <5% 879 +* ✅ Quality metrics public 880 + 881 +=== NFR12: Security Controls === 882 + 883 +**Fulfills:** Data protection, system integrity, user privacy, production readiness 884 + 885 +**Purpose:** Protect FactHarbor systems, user data, and operations from security threats, ensuring production-grade security posture. 886 + 887 +**Specification:** 888 + 889 +==== API Security ==== 890 + 891 +**Rate Limiting:** 892 + 893 +* **Analysis endpoints:** 100 requests/hour per IP 894 +* **Read endpoints:** 1,000 requests/hour per IP 895 +* **Search:** 500 requests/hour per IP 896 +* **Authenticated users:** 5x higher limits 897 +* **Burst protection:** Max 10 requests/second 898 + 899 +**Authentication & Authorization:** 900 + 901 +* **API Keys:** Required for programmatic access 902 +* **JWT tokens:** For user sessions (1-hour expiry) 903 +* **OAuth2:** For third-party integrations 904 +* **Role-Based Access Control (RBAC):** 905 +* Public: Read-only access to published claims 906 +* Contributor: Submit claims, provide evidence 907 +* Moderator: Review contributions, manage quality 908 +* Admin: System configuration, user management 909 + 910 +**CORS Policies:** 911 + 912 +* Whitelist approved domains only 913 +* No wildcard origins in production 914 +* Credentials required for sensitive endpoints 915 + 916 +**Input Sanitization:** 917 + 918 +* Validate all user input against schemas 919 +* Sanitize HTML/JavaScript in text submissions 920 +* Prevent SQL injection (use parameterized queries) 921 +* Prevent command injection (no shell execution of user input) 922 +* Max request size: 10MB 923 +* File upload restrictions: Whitelist file types, scan for malware 924 + 925 +---- 926 + 927 +==== Data Security ==== 928 + 929 +**Encryption at Rest:** 930 + 931 +* Database encryption using AES-256 932 +* Encrypted backups 933 +* Key management via cloud provider KMS (AWS KMS, Google Cloud KMS) 934 +* Regular key rotation (90-day cycle) 935 + 936 +**Encryption in Transit:** 937 + 938 +* HTTPS/TLS 1.3 only (no TLS 1.0/1.1) 939 +* Strong cipher suites only 940 +* HSTS (HTTP Strict Transport Security) enabled 941 +* Certificate pinning for mobile apps 942 + 943 +**Secure Credential Storage:** 944 + 945 +* Passwords hashed with bcrypt (cost factor 12+) 946 +* API keys encrypted in database 947 +* Secrets stored in environment variables (never in code) 948 +* Use secrets manager (AWS Secrets Manager, HashiCorp Vault) 949 + 950 +**Data Privacy:** 951 + 952 +* Minimal data collection (privacy by design) 953 +* User data deletion on request (GDPR compliance) 954 +* PII encryption in database 955 +* Anonymize logs (no PII in log files) 956 + 957 +---- 958 + 959 +==== Application Security ==== 960 + 961 +**OWASP Top 10 Compliance:** 962 + 963 +1. **Broken Access Control:** RBAC implementation, path traversal prevention 964 +2. **Cryptographic Failures:** Strong encryption, secure key management 965 +3. **Injection:** Parameterized queries, input validation 966 +4. **Insecure Design:** Security review of all features 967 +5. **Security Misconfiguration:** Hardened defaults, security headers 968 +6. **Vulnerable Components:** Dependency scanning (see below) 969 +7. **Authentication Failures:** Strong password policy, MFA support 970 +8. **Data Integrity Failures:** Signature verification, checksums 971 +9. **Security Logging Failures:** Comprehensive audit logs 972 +10. **Server-Side Request Forgery:** URL validation, whitelist domains 973 + 974 +**Security Headers:** 975 + 976 +* `Content-Security-Policy`: Strict CSP to prevent XSS 977 +* `X-Frame-Options`: DENY (prevent clickjacking) 978 +* `X-Content-Type-Options`: nosniff 979 +* `Referrer-Policy`: strict-origin-when-cross-origin 980 +* `Permissions-Policy`: Restrict browser features 981 + 982 +**Dependency Vulnerability Scanning:** 983 + 984 +* **Tools:** Snyk, Dependabot, npm audit, pip-audit 985 +* **Frequency:** Daily automated scans 986 +* **Action:** Patch critical vulnerabilities within 24 hours 987 +* **Policy:** No known high/critical CVEs in production 988 + 989 +**Security Audits:** 990 + 991 +* **Internal:** Quarterly security reviews 992 +* **External:** Annual penetration testing by certified firm 993 +* **Bug Bounty:** Public bug bounty program (V1.1+) 994 +* **Compliance:** SOC 2 Type II certification target (V1.5) 995 + 996 +---- 997 + 998 +==== Operational Security ==== 999 + 1000 +**DDoS Protection:** 1001 + 1002 +* CloudFlare or AWS Shield 1003 +* Rate limiting at CDN layer 1004 +* Automatic IP blocking for abuse patterns 1005 + 1006 +**Monitoring & Alerting:** 1007 + 1008 +* Real-time security event monitoring 1009 +* Alerts for: 1010 +* Failed login attempts (>5 in 10 minutes) 1011 +* API abuse patterns 1012 +* Unusual data access patterns 1013 +* Security scan detections 1014 +* Integration with SIEM (Security Information and Event Management) 1015 + 1016 +**Incident Response:** 1017 + 1018 +* Documented incident response plan 1019 +* Security incident classification (P1-P4) 1020 +* On-call rotation for security issues 1021 +* Post-mortem for all security incidents 1022 +* Public disclosure policy (coordinated disclosure) 1023 + 1024 +**Backup & Recovery:** 1025 + 1026 +* Daily encrypted backups 1027 +* 30-day retention period 1028 +* Tested recovery procedures (quarterly) 1029 +* Disaster recovery plan (RTO: 4 hours, RPO: 1 hour) 1030 + 1031 +---- 1032 + 1033 +==== Compliance & Standards ==== 1034 + 1035 +**GDPR Compliance:** 1036 + 1037 +* User consent management 1038 +* Right to access data 1039 +* Right to deletion 1040 +* Data portability 1041 +* Privacy policy published 1042 + 1043 +**Accessibility:** 1044 + 1045 +* WCAG 2.1 AA compliance 1046 +* Screen reader compatibility 1047 +* Keyboard navigation 1048 +* Alt text for images 1049 + 1050 +**Browser Support:** 1051 + 1052 +* Modern browsers only (Chrome/Edge/Firefox/Safari latest 2 versions) 1053 +* No IE11 support 1054 + 1055 +**Acceptance Criteria:** 1056 + 1057 +* ✅ Passes OWASP ZAP security scan (no high/critical findings) 1058 +* ✅ All dependencies with known vulnerabilities patched 1059 +* ✅ Penetration test completed with no critical findings 1060 +* ✅ Rate limiting blocks abuse attempts 1061 +* ✅ Encryption at rest and in transit verified 1062 +* ✅ Security headers scored A+ on securityheaders.com 1063 +* ✅ Incident response plan documented and tested 1064 +* ✅ 95% uptime over 30-day period 1065 + 1066 +=== NFR13: Quality Metrics Transparency === 1067 + 1068 +**Fulfills:** User trust, transparency, continuous improvement, IFCN methodology transparency 1069 + 1070 +**Purpose:** Provide transparent, measurable quality metrics that demonstrate AKEL's performance and build user trust in automated fact-checking. 1071 + 1072 +**Specification:** 1073 + 1074 +==== Component: Public Quality Dashboard ==== 1075 + 1076 +**Core Metrics to Display:** 1077 + 1078 +* \\ 1079 +** \\ 1080 +**1. Verdict Quality Metrics 1081 + 1082 +**TIGERScore (Fact-Checking Quality):** 1083 + 1084 +* **Definition:** Measures how well generated verdicts match expert fact-checker judgments 1085 +* **Scale:** 0-100 (higher is better) 1086 +* **Calculation:** Using TIGERScore framework (Truth-conditional accuracy, Informativeness, Generality, Evaluativeness, Relevance) 1087 +* **Target:** Average ≥80 for production release 1088 +* **Display:** 1089 +{{code}}Verdict Quality (TIGERScore): 1090 +Overall: 84.2 ▲ (+2.1 from last month) 1091 + 1092 +Distribution: 1093 + Excellent (>80): 67% 1094 + Good (60-80): 28% 1095 + Needs Improvement (<60): 5% 1096 + 1097 +Trend: [Graph showing improvement over time]{{/code}} 1098 + 1099 +**2. Hallucination & Faithfulness Metrics** 1100 + 1101 +**AlignScore (Faithfulness to Evidence):** 1102 + 1103 +* **Definition:** Measures how well verdicts align with actual evidence content 1104 +* **Scale:** 0-1 (higher is better) 1105 +* **Purpose:** Detect AI hallucinations (making claims not supported by evidence) 1106 +* **Target:** Average ≥0.85, hallucination rate <5% 1107 +* **Display:** 1108 +{{code}}Evidence Faithfulness (AlignScore): 1109 +Average: 0.87 ▼ (-0.02 from last month) 1110 + 1111 +Hallucination Rate: 4.2% 1112 + - Claims without evidence support: 3.1% 1113 + - Misrepresented evidence: 1.1% 1114 + 1115 +Action: Prompt engineering review scheduled{{/code}} 1116 + 1117 +**3. Evidence Quality Metrics** 1118 + 1119 +**Source Reliability:** 1120 + 1121 +* Average source quality score (0-1 scale) 1122 +* Distribution of high/medium/low quality sources 1123 +* Publisher track record trends 1124 + 1125 +**Evidence Coverage:** 1126 + 1127 +* Average number of sources per claim 1128 +* Percentage of claims with ≥2 sources (EFCSN minimum) 1129 +* Geographic diversity of sources 1130 + 1131 +**Display:** 1132 +{{code}}Evidence Quality: 1133 + 1134 +Average Sources per Claim: 4.2 1135 +Claims with ≥2 sources: 94% (EFCSN compliant) 1136 + 1137 +Source Quality Distribution: 1138 + High quality (>0.8): 48% 1139 + Medium quality (0.5-0.8): 43% 1140 + Low quality (<0.5): 9% 1141 + 1142 +Geographic Diversity: 23 countries represented{{/code}} 1143 + 1144 +**4. Contributor Consensus Metrics** (when human reviewers involved) 1145 + 1146 +**Inter-Rater Reliability (IRR):** 1147 + 1148 +* **Calculation:** Cohen's Kappa or Fleiss' Kappa for multiple raters 1149 +* **Scale:** 0-1 (higher is better) 1150 +* **Interpretation:** 1151 +* >0.8: Almost perfect agreement 1152 +* 0.6-0.8: Substantial agreement 1153 +* 0.4-0.6: Moderate agreement 1154 +* <0.4: Poor agreement 1155 +* **Target:** Maintain ≥0.7 (substantial agreement) 1156 + 1157 +**Display:** 1158 +{{code}}Contributor Consensus: 1159 + 1160 +Inter-Rater Reliability (IRR): 0.73 (Substantial agreement) 1161 + - Verdict agreement: 78% 1162 + - Evidence quality agreement: 71% 1163 + - Scenario structure agreement: 69% 1164 + 1165 +Cases requiring moderator review: 12 1166 +Moderator override rate: 8%{{/code}} 1167 + 1168 +---- 1169 + 1170 +==== Quality Dashboard Implementation ==== 1171 + 1172 +**Dashboard Location:** `/quality-metrics` 1173 + 1174 +**Update Frequency:** 1175 + 1176 +* **POC2:** Weekly manual updates 1177 +* **Beta 0:** Daily automated updates 1178 +* **V1.0:** Real-time metrics (updated hourly) 1179 + 1180 +**Dashboard Sections:** 1181 + 1182 +1. **Overview:** Key metrics at a glance 1183 +2. **Verdict Quality:** TIGERScore trends and distributions 1184 +3. **Evidence Analysis:** Source quality and coverage 1185 +4. **AI Performance:** Hallucination rates, AlignScore 1186 +5. **Human Oversight:** Contributor consensus, review rates 1187 +6. **System Health:** Processing times, error rates, uptime 1188 + 1189 +**Example Dashboard Layout:** 1190 + 1191 +{{code}} 1192 +┌─────────────────────────────────────────────────────────────┐ 1193 +│ FactHarbor Quality Metrics Last updated: │ 1194 +│ Public Dashboard 2 hours ago │ 1195 +└─────────────────────────────────────────────────────────────┘ 1196 + 1197 +📊 KEY METRICS 1198 +───────────────────────────────────────────────────────────── 1199 +TIGERScore (Verdict Quality): 84.2 ▲ (+2.1) 1200 +AlignScore (Faithfulness): 0.87 ▼ (-0.02) 1201 +Hallucination Rate: 4.2% ✓ (Target: <5%) 1202 +Average Sources per Claim: 4.2 ▲ (+0.3) 1203 + 1204 +📈 TRENDS (30 days) 1205 +───────────────────────────────────────────────────────────── 1206 +[Graph: TIGERScore trending upward] 1207 +[Graph: Hallucination rate declining] 1208 +[Graph: Evidence quality stable] 1209 + 1210 +⚠️ IMPROVEMENT TARGETS 1211 +───────────────────────────────────────────────────────────── 1212 +1. Reduce hallucination rate to <3% (Current: 4.2%) 1213 +2. Increase TIGERScore average to >85 (Current: 84.2) 1214 +3. Maintain IRR >0.75 (Current: 0.73) 1215 + 1216 +📄 DETAILED REPORTS 1217 +───────────────────────────────────────────────────────────── 1218 +• Monthly Quality Report (PDF) 1219 +• Methodology Documentation 1220 +• AKEL Performance Analysis 1221 +• Contributor Agreement Analysis 1222 + 1223 +{{/code}} 1224 + 1225 +---- 1226 + 1227 +==== Continuous Improvement Feedback Loop ==== 1228 + 1229 +**How Metrics Inform AKEL Improvements:** 1230 + 1231 +1. **Identify Weak Areas:** 1232 + 1233 +* Low TIGERScore → Review prompt engineering 1234 +* High hallucination → Strengthen evidence grounding 1235 +* Low IRR → Clarify evaluation criteria 1236 + 1237 +2. **A/B Testing Integration:** 1238 + 1239 +* Test prompt variations 1240 +* Measure impact on quality metrics 1241 +* Deploy winners automatically 1242 + 1243 +3. **Alert Thresholds:** 1244 + 1245 +* TIGERScore drops below 75 → Alert team 1246 +* Hallucination rate exceeds 7% → Pause auto-publishing 1247 +* IRR below 0.6 → Moderator training needed 1248 + 1249 +4. **Monthly Quality Reviews:** 1250 + 1251 +* Analyze trends 1252 +* Identify systematic issues 1253 +* Plan prompt improvements 1254 +* Update AKEL models 1255 + 1256 +---- 1257 + 1258 +==== Metric Calculation Details ==== 1259 + 1260 +**TIGERScore Implementation:** 1261 + 1262 +* Reference: https://github.com/TIGER-AI-Lab/TIGERScore 1263 +* Input: Generated verdict + reference verdict (from expert) 1264 +* Output: 0-100 score across 5 dimensions 1265 +* Requires: Test set of expert-reviewed claims (minimum 100) 1266 + 1267 +**AlignScore Implementation:** 1268 + 1269 +* Reference: https://github.com/yuh-zha/AlignScore 1270 +* Input: Generated verdict + source evidence text 1271 +* Output: 0-1 faithfulness score 1272 +* Calculation: Semantic alignment between claim and evidence 1273 + 1274 +**Source Quality Scoring:** 1275 + 1276 +* Use existing source reliability database (e.g., NewsGuard, MBFC) 1277 +* Factor in: Publication history, corrections record, transparency 1278 +* Scale: 0-1 (weighted average across sources) 1279 + 1280 +---- 1281 + 1282 +==== Integration Points ==== 1283 + 1284 +* **NFR11: AKEL Quality Assurance** - Metrics validate quality gate effectiveness 1285 +* **FR49: A/B Testing** - Metrics measure test success 1286 +* **FR11: Audit Trail** - Source of quality data 1287 +* **NFR3: Transparency** - Public metrics build trust 1288 + 1289 +**Acceptance Criteria:** 1290 + 1291 +* ✅ All core metrics implemented and calculating correctly 1292 +* ✅ Dashboard updates daily (Beta 0) or hourly (V1.0) 1293 +* ✅ Alerts trigger when metrics degrade beyond thresholds 1294 +* ✅ Monthly quality report auto-generates 1295 +* ✅ Dashboard is publicly accessible (no login required) 1296 +* ✅ Mobile-responsive dashboard design 1297 +* ✅ Metrics inform quarterly AKEL improvement planning 1298 + 1299 +== 13. Requirements Traceability == 1300 + 1301 +For full traceability matrix showing which requirements fulfill which user needs, see: 1302 + 1303 +* [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] - Section 8 includes comprehensive mapping tables 1304 + 1305 +== 14. Related Pages == 1306 + 1307 +**Non-Functional Requirements (see Section 9):** 1308 + 1309 +* [[NFR11 — AKEL Quality Assurance Framework>>#NFR11]] 1310 +* [[NFR12 — Security Controls>>#NFR12]] 1311 +* [[NFR13 — Quality Metrics Transparency>>#NFR13]] 1312 + 1313 +**Other Requirements:** 1314 + 1315 +* [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]] 1316 +* [[V1.0 Requirements>>FactHarbor.Specification.Requirements.V10.]] 1317 +* [[Gap Analysis>>FactHarbor.Specification.Requirements.GapAnalysis]] 1318 + 1319 +* **[[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]** - What users need (drives these requirements) 1320 +* [[Architecture>>FactHarbor.Specification.Architecture.WebHome]] - How requirements are implemented 1321 +* [[Data Model>>FactHarbor.Specification.Data Model.WebHome]] - Data structures supporting requirements 1322 +* [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] - User interaction workflows 1323 +* [[AKEL>>Archive.FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements 1324 +* [[Global Rules>>Archive.FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]] 363 363 * [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]] 1326 + 1327 += V0.9.70 Additional Requirements = 1328 + 1329 +== Functional Requirements (Additional) == 1330 + 1331 +=== FR44: ClaimReview Schema Implementation === 1332 + 1333 +**Fulfills:** UN-13 (Cite FactHarbor Verdicts), UN-14 (API Access for Integration), UN-26 (Search Engine Visibility) 1334 + 1335 +**Purpose:** Generate valid ClaimReview structured data for every published analysis to enable Google/Bing search visibility and fact-check discovery. 1336 + 1337 +**Specification:** 1338 + 1339 +==== Component: Schema.org Markup Generator ==== 1340 + 1341 +FactHarbor must generate valid ClaimReview structured data following Schema.org specifications for every published claim analysis. 1342 + 1343 +**Required JSON-LD Schema:** 1344 + 1345 +{{code language="json"}} 1346 +{ 1347 + "@context": "https://schema.org", 1348 + "@type": "ClaimReview", 1349 + "datePublished": "YYYY-MM-DD", 1350 + "url": "https://factharbor.org/claims/{claim_id}", 1351 + "claimReviewed": "The exact claim text", 1352 + "author": { 1353 + "@type": "Organization", 1354 + "name": "FactHarbor", 1355 + "url": "https://factharbor.org" 1356 + }, 1357 + "reviewRating": { 1358 + "@type": "Rating", 1359 + "ratingValue": "1-5", 1360 + "bestRating": "5", 1361 + "worstRating": "1", 1362 + "alternateName": "FactHarbor likelihood score" 1363 + }, 1364 + "itemReviewed": { 1365 + "@type": "Claim", 1366 + "author": { 1367 + "@type": "Person", 1368 + "name": "Claim author if known" 1369 + }, 1370 + "datePublished": "YYYY-MM-DD if known", 1371 + "appearance": { 1372 + "@type": "CreativeWork", 1373 + "url": "Original claim URL if from article" 1374 + } 1375 + } 1376 +} 1377 +{{/code}} 1378 + 1379 +**FactHarbor-Specific Mapping:** 1380 + 1381 +**Likelihood Score to Rating Scale:** 1382 + 1383 +* 80-100% likelihood → 5 (Highly Supported) 1384 +* 60-79% likelihood → 4 (Supported) 1385 +* 40-59% likelihood → 3 (Mixed/Uncertain) 1386 +* 20-39% likelihood → 2 (Questionable) 1387 +* 0-19% likelihood → 1 (Refuted) 1388 + 1389 +**Multiple Scenarios Handling:** 1390 + 1391 +* If claim has multiple scenarios with different verdicts, generate **separate ClaimReview** for each scenario 1392 +* Add `disambiguatingDescription` field explaining scenario context 1393 +* Example: "Scenario: If interpreted as referring to 2023 data..." 1394 + 1395 +==== Implementation Requirements ==== 1396 + 1397 +1. **Auto-generate** on claim publication 1398 +2. **Embed** in HTML `<head>` section as JSON-LD script 1399 +3. **Validate** against Schema.org validator before publishing 1400 +4. **Submit** to Google Search Console for indexing 1401 +5. **Update** automatically when verdict changes (integrate with FR8: Time Evolution) 1402 + 1403 +==== Integration Points ==== 1404 + 1405 +* **FR7: Automated Verdicts** - Source of rating data and claim text 1406 +* **FR8: Time Evolution** - Triggers schema updates when verdicts change 1407 +* **FR11: Audit Trail** - Logs all schema generation and update events 1408 + 1409 +==== Resources ==== 1410 + 1411 +* ClaimReview Project: https://www.claimreviewproject.com 1412 +* Schema.org ClaimReview: https://schema.org/ClaimReview 1413 +* Google Fact Check Guidelines: https://developers.google.com/search/docs/appearance/fact-check 1414 + 1415 +**Acceptance Criteria:** 1416 + 1417 +* ✅ Passes Google Structured Data Testing Tool 1418 +* ✅ Appears in Google Fact Check Explorer within 48 hours of publication 1419 +* ✅ Valid JSON-LD syntax (no errors) 1420 +* ✅ All required fields populated with correct data types 1421 +* ✅ Handles multi-scenario claims correctly (separate ClaimReview per scenario) 1422 + 1423 +=== FR45: User Corrections Notification System === 1424 + 1425 +**Fulfills:** IFCN Principle 5 (Open & Honest Corrections), EFCSN compliance 1426 + 1427 +**Purpose:** When any claim analysis is corrected, notify users who previously viewed the claim to maintain transparency and build trust. 1428 + 1429 +**Specification:** 1430 + 1431 +==== Component: Corrections Visibility Framework ==== 1432 + 1433 +**Correction Types:** 1434 + 1435 +1. **Major Correction:** Verdict changes category (e.g., "Supported" → "Refuted") 1436 +2. **Significant Correction:** Likelihood score changes >20% 1437 +3. **Minor Correction:** Evidence additions, source quality updates 1438 +4. **Scenario Addition:** New scenario added to existing claim 1439 + 1440 +==== Notification Mechanisms ==== 1441 + 1442 +* \\ 1443 +** \\ 1444 +**1. In-Page Banner: 1445 + 1446 +Display prominent banner on claim page: 1447 + 1448 +{{code}} 1449 +[!] CORRECTION NOTICE 1450 +This analysis was updated on [DATE]. [View what changed] [Dismiss] 1451 + 1452 +Major changes: 1453 +• Verdict changed from "Likely True (75%)" to "Uncertain (45%)" 1454 +• New contradicting evidence added from [Source] 1455 +• Scenario 2 updated with additional context 1456 + 1457 +[See full correction log] 1458 +{{/code}} 1459 + 1460 +**2. Correction Log Page:** 1461 + 1462 +* Public changelog at `/claims/{id}/corrections` 1463 +* Displays for each correction: 1464 +* Date/time of correction 1465 +* What changed (before/after comparison) 1466 +* Why changed (reason if provided) 1467 +* Who made change (AKEL auto-update vs. contributor override) 1468 + 1469 +**3. Email Notifications (opt-in):** 1470 + 1471 +* Send to users who bookmarked or shared the claim 1472 +* Subject: "FactHarbor Correction: [Claim title]" 1473 +* Include summary of changes 1474 +* Link to updated analysis 1475 + 1476 +**4. RSS/API Feed:** 1477 + 1478 +* Corrections feed at `/corrections.rss` 1479 +* API endpoint: `GET /api/corrections?since={timestamp}` 1480 +* Enables external monitoring by journalists and researchers 1481 + 1482 +==== Display Rules ==== 1483 + 1484 +* Show banner on **ALL pages** displaying the claim (search results, related claims, embeddings) 1485 +* Banner persists for **30 days** after correction 1486 +* **"Corrections" count badge** on claim card 1487 +* **Timestamp** on every verdict: "Last updated: [datetime]" 1488 + 1489 +==== IFCN Compliance Requirements ==== 1490 + 1491 +* Corrections policy published at `/corrections-policy` 1492 +* User can report suspected errors via `/report-error/{claim_id}` 1493 +* Link to IFCN complaint process (if FactHarbor becomes signatory) 1494 +* **Scrupulous transparency:** Never silently edit analyses 1495 + 1496 +==== Integration Points ==== 1497 + 1498 +* **FR8: Time Evolution** - Triggers corrections when verdicts change 1499 +* **FR11: Audit Trail** - Source of correction data and change history 1500 +* **NFR3: Transparency** - Public correction log demonstrates commitment 1501 + 1502 +**Acceptance Criteria:** 1503 + 1504 +* ✅ Banner appears within 60 seconds of correction 1505 +* ✅ Correction log is permanent and publicly accessible 1506 +* ✅ Email notifications deliver within 5 minutes 1507 +* ✅ RSS feed updates in real-time 1508 +* ✅ Mobile-responsive banner design 1509 +* ✅ Accessible (screen reader compatible) 1510 + 1511 +=== FR46: Image Verification System === 1512 + 1513 +**Fulfills:** UN-27 (Visual Claim Verification) 1514 + 1515 +**Purpose:** Verify authenticity and context of images shared with claims to detect manipulation, misattribution, and out-of-context usage. 1516 + 1517 +**Specification:** 1518 + 1519 +==== Component: Multi-Method Image Verification ==== 1520 + 1521 +**Method 1: Reverse Image Search** 1522 + 1523 +**Purpose:** Find earlier uses of the image to verify context 1524 + 1525 +**Implementation:** 1526 + 1527 +* Integrate APIs: 1528 +* **Google Vision AI** (reverse search) 1529 +* **TinEye** (oldest known uses) 1530 +* **Bing Visual Search** (broad coverage) 1531 + 1532 +**Process:** 1533 + 1534 +1. Extract image from claim or user upload 1535 +2. Query multiple reverse search services 1536 +3. Analyze results for: 1537 + 1538 +* Earliest known publication 1539 +* Original context (what was it really showing?) 1540 +* Publication timeline 1541 +* Geographic spread 1542 + 1543 +**Output:** 1544 +{{code}}Reverse Image Search Results: 1545 + 1546 +Earliest known use: 2019-03-15 (5 years before claim) 1547 +Original context: "Photo from 2019 flooding in Mumbai" 1548 +This claim uses it for: "2024 hurricane damage in Florida" 1549 + 1550 +⚠️ Image is OUT OF CONTEXT 1551 + 1552 +Found in 47 other articles: 1553 +• 2019-03-15: Mumbai floods (original) 1554 +• 2020-07-22: Bangladesh monsoon 1555 +• 2024-10-15: Current claim (misattributed) 1556 + 1557 +[View full timeline]{{/code}} 1558 + 1559 +---- 1560 + 1561 +**Method 2: AI Manipulation Detection** 1562 + 1563 +**Purpose:** Detect deepfakes, face swaps, and digital alterations 1564 + 1565 +**Implementation:** 1566 + 1567 +* Integrate detection services: 1568 +* **Sensity AI** (deepfake detection) 1569 +* **Reality Defender** (multimodal analysis) 1570 +* **AWS Rekognition** (face detection inconsistencies) 1571 + 1572 +**Detection Categories:** 1573 + 1574 +1. **Face Manipulation:** 1575 + 1576 +* Deepfake face swaps 1577 +* Expression manipulation 1578 +* Identity replacement 1579 + 1580 +2. **Image Manipulation:** 1581 + 1582 +* Copy-paste artifacts 1583 +* Clone stamp detection 1584 +* Content-aware fill detection 1585 +* JPEG compression inconsistencies 1586 + 1587 +3. **AI Generation:** 1588 + 1589 +* Detect fully AI-generated images 1590 +* Identify generation artifacts 1591 +* Check for model signatures 1592 + 1593 +**Confidence Scoring:** 1594 + 1595 +* **HIGH (80-100%):** Strong evidence of manipulation 1596 +* **MEDIUM (50-79%):** Suspicious artifacts detected 1597 +* **LOW (0-49%):** Minor inconsistencies or inconclusive 1598 + 1599 +**Output:** 1600 +{{code}}Manipulation Analysis: 1601 + 1602 +Face Manipulation: LOW RISK (12%) 1603 +Image Editing: MEDIUM RISK (64%) 1604 + • Clone stamp artifacts detected in sky region 1605 + • JPEG compression inconsistent between objects 1606 + 1607 +AI Generation: LOW RISK (8%) 1608 + 1609 +⚠️ Possible manipulation detected. Manual review recommended.{{/code}} 1610 + 1611 +---- 1612 + 1613 +**Method 3: Metadata Analysis (EXIF)** 1614 + 1615 +**Purpose:** Extract technical details that may reveal manipulation or misattribution 1616 + 1617 +**Extracted Data:** 1618 + 1619 +* **Camera/Device:** Make, model, software 1620 +* **Timestamps:** Original date, modification dates 1621 +* **Location:** GPS coordinates (if present) 1622 +* **Editing History:** Software used, edit count 1623 +* **File Properties:** Resolution, compression, format conversions 1624 + 1625 +**Red Flags:** 1626 + 1627 +* Metadata completely stripped (suspicious) 1628 +* Timestamp conflicts with claimed date 1629 +* GPS location conflicts with claimed location 1630 +* Multiple edit rounds (hiding something?) 1631 +* Creation date after modification date (impossible) 1632 + 1633 +**Output:** 1634 +{{code}}Image Metadata: 1635 + 1636 +Camera: iPhone 14 Pro 1637 +Original date: 2023-08-12 14:32:15 1638 +Location: 40.7128°N, 74.0060°W (New York City) 1639 +Modified: 2024-10-15 08:45:22 1640 +Software: Adobe Photoshop 2024 1641 + 1642 +⚠️ Location conflicts with claim 1643 +Claim says: "Taken in Los Angeles" 1644 +EXIF says: New York City 1645 + 1646 +⚠️ Edited 14 months after capture{{/code}} 1647 + 1648 +---- 1649 + 1650 +==== Verification Workflow ==== 1651 + 1652 +**Automatic Triggers:** 1653 + 1654 +1. User submits claim with image 1655 +2. Article being analyzed contains images 1656 +3. Social media post includes photos 1657 + 1658 +**Process:** 1659 + 1660 +1. Extract images from content 1661 +2. Run all 3 verification methods in parallel 1662 +3. Aggregate results into confidence score 1663 +4. Generate human-readable summary 1664 +5. Display prominently in analysis 1665 + 1666 +**Display Integration:** 1667 + 1668 +Show image verification panel in claim analysis: 1669 + 1670 +{{code}} 1671 +📷 IMAGE VERIFICATION 1672 + 1673 +[Image thumbnail] 1674 + 1675 +✅ Reverse Search: Original context verified 1676 +⚠️ Manipulation: Possible editing detected (64% confidence) 1677 +✅ Metadata: Consistent with claim details 1678 + 1679 +Overall Assessment: CAUTION ADVISED 1680 +This image may have been edited. Original context appears accurate. 1681 + 1682 +[View detailed analysis] 1683 +{{/code}} 1684 + 1685 +==== Integration Points ==== 1686 + 1687 +* **FR7: Automated Verdicts** - Image verification affects claim credibility 1688 +* **FR4: Analysis Summary** - Image findings included in summary 1689 +* **UN-27: Visual Claim Verification** - Direct fulfillment 1690 + 1691 +==== Cost Considerations ==== 1692 + 1693 +**API Costs (estimated per image):** 1694 + 1695 +* Google Vision AI: $0.001-0.003 1696 +* TinEye: $0.02 (commercial API) 1697 +* Sensity AI: $0.05-0.10 1698 +* AWS Rekognition: $0.001-0.002 1699 + 1700 +**Total per image:** $0.07-0.15** 1701 + 1702 +**Mitigation Strategies:** 1703 + 1704 +* Cache results for duplicate images 1705 +* Use free tier quotas where available 1706 +* Prioritize higher-value claims for deep analysis 1707 +* Offer premium verification as paid tier 1708 + 1709 +**Acceptance Criteria:** 1710 + 1711 +* ✅ Reverse image search finds original sources 1712 +* ✅ Manipulation detection accuracy >80% on test dataset 1713 +* ✅ EXIF extraction works for major image formats (JPEG, PNG, HEIC) 1714 +* ✅ Results display within 10 seconds 1715 +* ✅ Mobile-friendly image comparison interface 1716 +* ✅ False positive rate <15% 1717 + 1718 +=== FR47: Archive.org Integration === 1719 + 1720 +**Importance:** CRITICAL 1721 +**Fulfills:** Evidence persistence, FR5 (Evidence linking) 1722 + 1723 +**Purpose:** Ensure evidence remains accessible even if original sources are deleted. 1724 + 1725 +**Specification:** 1726 + 1727 +**Automatic Archiving:** 1728 + 1729 +When AKEL links evidence: 1730 + 1731 +1. Check if URL already archived (Wayback Machine API) 1732 +2. If not, submit for archiving (Save Page Now API) 1733 +3. Store both original URL and archive URL 1734 +4. Display both to users 1735 + 1736 +**Archive Display:** 1737 + 1738 +{{code}} 1739 +Evidence Source: [Original URL] 1740 +Archived: [Archive.org URL] (Captured: [date]) 1741 + 1742 +[View Original] [View Archive] 1743 +{{/code}} 1744 + 1745 +**Fallback Logic:** 1746 + 1747 +* If original URL unavailable → Auto-redirect to archive 1748 +* If archive unavailable → Display warning 1749 +* If both unavailable → Flag for manual review 1750 + 1751 +**API Integration:** 1752 + 1753 +* Use Wayback Machine Availability API 1754 +* Use Save Page Now API (SPNv2) 1755 +* Rate limiting: 15 requests/minute (Wayback limit) 1756 + 1757 +**Acceptance Criteria:** 1758 + 1759 +* ✅ All evidence URLs auto-archived 1760 +* ✅ Archive links displayed to users 1761 +* ✅ Fallback to archive if original unavailable 1762 +* ✅ API rate limits respected 1763 +* ✅ Archive status visible in evidence display 1764 + 1765 +== Category 4: Community Safety == 1766 + 1767 + FR48: Contributor Safety Framework === 1768 + 1769 +**Importance:** CRITICAL 1770 +**Fulfills:** UN-28 (Safe contribution environment) 1771 + 1772 +**Purpose:** Protect contributors from harassment, doxxing, and coordinated attacks. 1773 + 1774 +**Specification:** 1775 + 1776 +* \\ 1777 +** \\ 1778 +**1. Privacy Protection: 1779 + 1780 +* **Optional Pseudonymity:** Contributors can use pseudonyms 1781 +* **Email Privacy:** Emails never displayed publicly 1782 +* **Profile Privacy:** Contributors control what's public 1783 +* **IP Logging:** Only for abuse prevention, not public 1784 + 1785 +**2. Harassment Prevention:** 1786 + 1787 +* **Automated Toxicity Detection:** Flag abusive comments 1788 +* **Personal Information Detection:** Auto-block doxxing attempts 1789 +* **Coordinated Attack Detection:** Identify brigading patterns 1790 +* **Rapid Response:** Moderator alerts for harassment 1791 + 1792 +**3. Safety Features:** 1793 + 1794 +* **Block Users:** Contributors can block harassers 1795 +* **Private Contributions:** Option to contribute anonymously 1796 +* **Report Harassment:** One-click harassment reporting 1797 +* **Safety Resources:** Links to support resources 1798 + 1799 +**4. Moderator Tools:** 1800 + 1801 +* **Quick Ban:** Immediately block abusers 1802 +* **Pattern Detection:** Identify coordinated attacks 1803 +* **Appeal Process:** Fair review of moderation actions 1804 +* **Escalation:** Serious threats escalated to authorities 1805 + 1806 +**5. Trusted Contributor Protection:** 1807 + 1808 +* **Enhanced Privacy:** Additional protection for high-profile contributors 1809 +* **Verification:** Optional identity verification (not public) 1810 +* **Legal Support:** Resources for contributors facing legal threats 1811 + 1812 +**Acceptance Criteria:** 1813 + 1814 +* ✅ Pseudonyms supported 1815 +* ✅ Toxicity detection active 1816 +* ✅ Doxxing auto-blocked 1817 +* ✅ Harassment reporting functional 1818 +* ✅ Moderator tools implemented 1819 +* ✅ Safety policy published 1820 + 1821 +== Category 5: Continuous Improvement == 1822 + 1823 + FR49: A/B Testing Framework === 1824 + 1825 +**Importance:** CRITICAL 1826 +**Fulfills:** Continuous system improvement 1827 + 1828 +**Purpose:** Test and measure improvements to AKEL prompts, algorithms, and workflows. 1829 + 1830 +**Specification:** 1831 + 1832 +**Test Capabilities:** 1833 + 1834 +1. **Prompt Variations:** 1835 + 1836 +* Test different claim extraction prompts 1837 +* Test different verdict generation prompts 1838 +* Measure: Accuracy, clarity, completeness 1839 + 1840 +2. **Algorithm Variations:** 1841 + 1842 +* Test different source scoring algorithms 1843 +* Test different confidence calculations 1844 +* Measure: Audit accuracy, user satisfaction 1845 + 1846 +3. **Workflow Variations:** 1847 + 1848 +* Test different quality gate thresholds 1849 +* Test different risk tier assignments 1850 +* Measure: Publication rate, quality scores 1851 + 1852 +**Implementation:** 1853 + 1854 +* **Traffic Split:** 50/50 or 90/10 splits 1855 +* **Randomization:** Consistent per claim (not per user) 1856 +* **Metrics Collection:** Automatic for all variants 1857 +* **Statistical Significance:** Minimum sample size calculation 1858 +* **Rollout:** Winner promoted to 100% traffic 1859 + 1860 +**A/B Test Workflow:** 1861 + 1862 +{{code}} 1863 +1. Hypothesis: "New prompt improves claim extraction" 1864 +2. Design test: Control vs. Variant 1865 +3. Define metrics: Extraction accuracy, completeness 1866 +4. Run test: 7-14 days, minimum 100 claims each 1867 +5. Analyze results: Statistical significance? 1868 +6. Decision: Deploy winner or iterate 1869 +{{/code}} 1870 + 1871 +**Acceptance Criteria:** 1872 + 1873 +* ✅ A/B testing framework implemented 1874 +* ✅ Can test prompt variations 1875 +* ✅ Can test algorithm variations 1876 +* ✅ Metrics automatically collected 1877 +* ✅ Statistical significance calculated 1878 +* ✅ Results inform system improvements 1879 + 1880 +=== FR54: Evidence Deduplication === 1881 + 1882 +**Importance:** CRITICAL (POC2/Beta) 1883 +**Fulfills:** Accurate evidence counting, quality metrics 1884 + 1885 +**Purpose:** Avoid counting the same source multiple times when it appears in different forms. 1886 + 1887 +**Specification:** 1888 + 1889 +**Deduplication Logic:** 1890 + 1891 +1. **URL Normalization:** 1892 + 1893 +* Remove tracking parameters (?utm_source=...) 1894 +* Normalize http/https 1895 +* Normalize www/non-www 1896 +* Handle redirects 1897 + 1898 +2. **Content Similarity:** 1899 + 1900 +* If two sources have >90% text similarity → Same source 1901 +* If one is subset of other → Same source 1902 +* Use fuzzy matching for minor differences 1903 + 1904 +3. **Cross-Domain Syndication:** 1905 + 1906 +* Detect wire service content (AP, Reuters) 1907 +* Mark as single source if syndicated 1908 +* Count original publication only 1909 + 1910 +**Display:** 1911 + 1912 +{{code}} 1913 +Evidence Sources (3 unique, 5 total): 1914 + 1915 +1. Original Article (NYTimes) 1916 + - Also appeared in: WashPost, Guardian (syndicated) 1917 + 1918 +2. Research Paper (Nature) 1919 + 1920 +3. Official Statement (WHO) 1921 +{{/code}} 1922 + 1923 +**Acceptance Criteria:** 1924 + 1925 +* ✅ URL normalization works 1926 +* ✅ Content similarity detected 1927 +* ✅ Syndicated content identified 1928 +* ✅ Unique vs. total counts accurate 1929 +* ✅ Improves evidence quality metrics 1930 + 1931 +== Additional Requirements (Lower Importance) == 1932 + 1933 + FR50: OSINT Toolkit Integration === 1934 + 1935 +**Fulfills:** Advanced media verification 1936 + 1937 +**Purpose:** Integrate open-source intelligence tools for advanced verification. 1938 + 1939 +**Tools to Integrate:** 1940 + 1941 +* InVID/WeVerify (video verification) 1942 +* Bellingcat toolkit 1943 +* Additional TBD based on V1.0 learnings 1944 + 1945 +=== FR51: Video Verification System === 1946 + 1947 +**Fulfills:** UN-27 (Visual claims), advanced media verification 1948 + 1949 +**Purpose:** Verify video-based claims. 1950 + 1951 +**Specification:** 1952 + 1953 +* Keyframe extraction 1954 +* Reverse video search 1955 +* Deepfake detection (AI-powered) 1956 +* Metadata analysis 1957 +* Acoustic signature analysis 1958 + 1959 +=== FR52: Interactive Detection Training === 1960 + 1961 +**Fulfills:** Media literacy education 1962 + 1963 +**Purpose:** Teach users to identify misinformation. 1964 + 1965 +**Specification:** 1966 + 1967 +* Interactive tutorials 1968 +* Practice exercises 1969 +* Detection quizzes 1970 +* Gamification elements 1971 + 1972 +=== FR53: Cross-Organizational Sharing === 1973 + 1974 +**Fulfills:** Collaboration with other fact-checkers 1975 + 1976 +**Purpose:** Share findings with IFCN/EFCSN members. 1977 + 1978 +**Specification:** 1979 + 1980 +* API for fact-checking organizations 1981 +* Structured data exchange 1982 +* Privacy controls 1983 +* Attribution requirements 1984 + 1985 +== Summary == 1986 + 1987 +**V1.0 Critical Requirements (Must Have):** 1988 + 1989 +* FR44: ClaimReview Schema ✅ 1990 +* FR45: Corrections Notification ✅ 1991 +* FR46: Image Verification ✅ 1992 +* FR47: Archive.org Integration ✅ 1993 +* FR48: Contributor Safety ✅ 1994 +* FR49: A/B Testing ✅ 1995 +* FR54: Evidence Deduplication ✅ 1996 +* NFR11: Quality Assurance Framework ✅ 1997 +* NFR12: Security Controls ✅ 1998 +* NFR13: Quality Metrics Dashboard ✅ 1999 + 2000 +**V1.1+ (Future):** 2001 + 2002 +* FR50: OSINT Integration 2003 +* FR51: Video Verification 2004 +* FR52: Detection Training 2005 +* FR53: Cross-Org Sharing 2006 + 2007 +**Total:** 11 critical requirements for V1.0 2008 + 2009 +=== FR54: Evidence Deduplication === 2010 + 2011 +**Fulfills:** Accurate evidence counting, quality metrics 2012 + 2013 +**Purpose:** Avoid counting the same source multiple times when it appears in different forms. 2014 + 2015 +**Specification:** 2016 + 2017 +**Deduplication Logic:** 2018 + 2019 +1. **URL Normalization:** 2020 + 2021 +* Remove tracking parameters (?utm_source=...) 2022 +* Normalize http/https 2023 +* Normalize www/non-www 2024 +* Handle redirects 2025 + 2026 +2. **Content Similarity:** 2027 + 2028 +* If two sources have >90% text similarity → Same source 2029 +* If one is subset of other → Same source 2030 +* Use fuzzy matching for minor differences 2031 + 2032 +3. **Cross-Domain Syndication:** 2033 + 2034 +* Detect wire service content (AP, Reuters) 2035 +* Mark as single source if syndicated 2036 +* Count original publication only 2037 + 2038 +**Display:** 2039 + 2040 +{{code}} 2041 +Evidence Sources (3 unique, 5 total): 2042 + 2043 +1. Original Article (NYTimes) 2044 + - Also appeared in: WashPost, Guardian (syndicated) 2045 + 2046 +2. Research Paper (Nature) 2047 + 2048 +3. Official Statement (WHO) 2049 +{{/code}} 2050 + 2051 +**Acceptance Criteria:** 2052 + 2053 +* ✅ URL normalization works 2054 +* ✅ Content similarity detected 2055 +* ✅ Syndicated content identified 2056 +* ✅ Unique vs. total counts accurate 2057 +* ✅ Improves evidence quality metrics 2058 + 2059 +== Additional Requirements (Lower Importance) == 2060 + 2061 + FR7: Automated Verdicts (Enhanced with Quality Gates) === 2062 + 2063 +**POC1+ Enhancement:** 2064 + 2065 +After AKEL generates verdict, it passes through quality gates: 2066 + 2067 +{{code}} 2068 +Workflow: 2069 +1. Extract claims 2070 + ↓ 2071 +2. [GATE 1] Validate fact-checkable 2072 + ↓ 2073 +3. Generate scenarios 2074 + ↓ 2075 +4. Generate verdicts 2076 + ↓ 2077 +5. [GATE 4] Validate confidence 2078 + ↓ 2079 +6. Display to user 2080 +{{/code}} 2081 + 2082 +**Updated Verdict States:** 2083 + 2084 +* PUBLISHED 2085 +* INSUFFICIENT_EVIDENCE 2086 +* NON_FACTUAL_CLAIM 2087 +* PROCESSING 2088 +* ERROR 2089 + 2090 +=== FR4: Analysis Summary (Enhanced with Quality Metadata) === 2091 + 2092 +**POC1+ Enhancement:** 2093 + 2094 +Display quality indicators: 2095 + 2096 +{{code}} 2097 +Analysis Summary: 2098 + Verifiable Claims: 3/5 2099 + High Confidence Verdicts: 1 2100 + Medium Confidence: 2 2101 + Evidence Sources: 12 2102 + Avg Source Quality: 0.73 2103 + Quality Score: 8.5/10 2104 +{{/code}}