Wiki source code of Requirements
Last modified by Robert Schaub on 2026/02/08 21:32
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | = Requirements = | ||
| 2 | |||
| 3 | {{info}} | ||
| 4 | **Phase Assignments:** See [[Requirements Roadmap Matrix>>Archive.FactHarbor 2026\.01\.20.Roadmap.Requirements-Roadmap-Matrix.WebHome]] for which requirements are implemented in which phases. | ||
| 5 | {{/info}} | ||
| 6 | |||
| 7 | **This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.** | ||
| 8 | |||
| 9 | **Core Philosophy:** Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically. | ||
| 10 | |||
| 11 | == Navigation == | ||
| 12 | |||
| 13 | * **[[User Needs>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.User Needs.WebHome]]** - What users need from FactHarbor (drives these requirements) | ||
| 14 | * **This page** - How we fulfill those needs through system design | ||
| 15 | |||
| 16 | (% class="box infomessage" %) | ||
| 17 | ((( | ||
| 18 | **How to read this page:** | ||
| 19 | |||
| 20 | 1. **User Needs drive Requirements**: See [[User Needs>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.User Needs.WebHome]] for what users need | ||
| 21 | 2. **Requirements define implementation**: This page shows how we fulfill those needs | ||
| 22 | 3. **Functional Requirements (FR)**: Specific features and capabilities | ||
| 23 | 4. **Non-Functional Requirements (NFR)**: Quality attributes (performance, security, etc.) | ||
| 24 | |||
| 25 | Each requirement references which User Needs it fulfills. | ||
| 26 | ))) | ||
| 27 | |||
| 28 | == 1. Roles == | ||
| 29 | |||
| 30 | **Fulfills**: UN-12 (Submit claims), UN-13 (Cite verdicts), UN-14 (API access) | ||
| 31 | |||
| 32 | FactHarbor uses three simple roles plus a reputation system. | ||
| 33 | |||
| 34 | === 1.1 Reader === | ||
| 35 | |||
| 36 | **Who**: Anyone (no login required) | ||
| 37 | |||
| 38 | **Can**: | ||
| 39 | |||
| 40 | * Browse and search claims | ||
| 41 | * View scenarios, evidence, verdicts, and confidence scores | ||
| 42 | * Flag issues or errors | ||
| 43 | * Use filters, search, and visualization tools | ||
| 44 | * Submit claims automatically (new claims added if not duplicates) | ||
| 45 | |||
| 46 | **Cannot**: | ||
| 47 | |||
| 48 | * Modify content | ||
| 49 | * Access edit history details | ||
| 50 | |||
| 51 | **User Needs served**: UN-1 (Trust assessment), UN-2 (Claim verification), UN-3 (Article summary with FactHarbor analysis summary), UN-4 (Social media fact-checking), UN-5 (Source tracing), UN-7 (Evidence transparency), UN-8 (Understanding disagreement), UN-12 (Submit claims), UN-17 (In-article highlighting) | ||
| 52 | |||
| 53 | === 1.2 Contributor === | ||
| 54 | |||
| 55 | **Who**: Registered users (earns reputation through contributions) | ||
| 56 | |||
| 57 | **Can**: | ||
| 58 | |||
| 59 | * Everything a Reader can do | ||
| 60 | * Edit claims, evidence, and scenarios | ||
| 61 | * Add sources and citations | ||
| 62 | * Suggest improvements to AI-generated content | ||
| 63 | * Participate in discussions | ||
| 64 | * Earn reputation points for quality contributions | ||
| 65 | |||
| 66 | **Reputation System**: | ||
| 67 | |||
| 68 | * New contributors: Limited edit privileges | ||
| 69 | * Established contributors (established reputation): Full edit access | ||
| 70 | * Trusted contributors (substantial reputation): Can approve certain changes | ||
| 71 | * Reputation earned through: Accepted edits, helpful flags, quality contributions | ||
| 72 | * Reputation lost through: Reverted edits, invalid flags, abuse | ||
| 73 | |||
| 74 | **Cannot**: | ||
| 75 | |||
| 76 | * Delete or hide content (only moderators) | ||
| 77 | * Override moderation decisions | ||
| 78 | |||
| 79 | **User Needs served**: UN-13 (Cite and contribute) | ||
| 80 | |||
| 81 | === 1.3 Moderator === | ||
| 82 | |||
| 83 | **Who**: Trusted community members with proven track record, appointed by governance board | ||
| 84 | |||
| 85 | **Can**: | ||
| 86 | |||
| 87 | * Review flagged content | ||
| 88 | * Hide harmful or abusive content | ||
| 89 | * Resolve disputes between contributors | ||
| 90 | * Issue warnings or temporary bans | ||
| 91 | * Make final decisions on content disputes | ||
| 92 | * Access full audit logs | ||
| 93 | |||
| 94 | **Cannot**: | ||
| 95 | |||
| 96 | * Change governance rules | ||
| 97 | * Permanently ban users without board approval | ||
| 98 | * Override technical quality gates | ||
| 99 | |||
| 100 | **Note**: Small team (3-5 initially), supported by automated moderation tools. | ||
| 101 | |||
| 102 | === 1.4 Domain Trusted Contributors (Optional, Task-Specific) === | ||
| 103 | |||
| 104 | **Who**: Subject matter specialists invited for specific high-stakes disputes | ||
| 105 | |||
| 106 | **Not a permanent role**: Contacted externally when needed for contested claims in their domain | ||
| 107 | |||
| 108 | **When used**: | ||
| 109 | |||
| 110 | * Medical claims with life/safety implications | ||
| 111 | * Legal interpretations with significant impact | ||
| 112 | * Scientific claims with high controversy | ||
| 113 | * Technical claims requiring specialized knowledge | ||
| 114 | |||
| 115 | **Process**: | ||
| 116 | |||
| 117 | * Moderator identifies need for expert input | ||
| 118 | * Contact expert externally (don't require them to be users) | ||
| 119 | * Trusted Contributor provides written opinion with sources | ||
| 120 | * Opinion added to claim record | ||
| 121 | * Trusted Contributor acknowledged in claim | ||
| 122 | |||
| 123 | **User Needs served**: UN-16 (Expert validation status) | ||
| 124 | |||
| 125 | == 2. Content States == | ||
| 126 | |||
| 127 | **Fulfills**: UN-1 (Trust indicators), UN-16 (Review status transparency) | ||
| 128 | |||
| 129 | FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping. | ||
| 130 | |||
| 131 | === 2.1 Published === | ||
| 132 | |||
| 133 | **Status**: Visible to all users | ||
| 134 | |||
| 135 | **Includes**: | ||
| 136 | |||
| 137 | * AI-generated analyses (default state) | ||
| 138 | * User-contributed content | ||
| 139 | * Edited/improved content | ||
| 140 | |||
| 141 | **Quality Indicators** (displayed with content): | ||
| 142 | |||
| 143 | * **Confidence Score**: 0-100% (AI's confidence in analysis) | ||
| 144 | * **Source Quality Score**: 0-100% (based on source track record) | ||
| 145 | * **Controversy Flag**: If high dispute/edit activity | ||
| 146 | * **Completeness Score**: % of expected fields filled | ||
| 147 | * **Last Updated**: Date of most recent change | ||
| 148 | * **Edit Count**: Number of revisions | ||
| 149 | * **Review Status**: AI-generated / Human-reviewed / Expert-validated | ||
| 150 | |||
| 151 | **Automatic Warnings**: | ||
| 152 | |||
| 153 | * Confidence < 60%: "Low confidence - use caution" | ||
| 154 | * Source quality < 40%: "Sources may be unreliable" | ||
| 155 | * High controversy: "Disputed - multiple interpretations exist" | ||
| 156 | * Medical/Legal/Safety domain: "Seek professional advice" | ||
| 157 | |||
| 158 | **User Needs served**: UN-1 (Trust score), UN-9 (Methodology transparency), ~~UN-15 (Evolution timeline - Deferred)~~, UN-16 (Review status) | ||
| 159 | |||
| 160 | === 2.2 Hidden === | ||
| 161 | |||
| 162 | **Status**: Not visible to regular users (only to moderators) | ||
| 163 | |||
| 164 | **Reasons**: | ||
| 165 | |||
| 166 | * Spam or advertising | ||
| 167 | * Personal attacks or harassment | ||
| 168 | * Illegal content | ||
| 169 | * Privacy violations | ||
| 170 | * Deliberate misinformation (verified) | ||
| 171 | * Abuse or harmful content | ||
| 172 | |||
| 173 | **Process**: | ||
| 174 | |||
| 175 | * Automated detection flags for moderator review | ||
| 176 | * Moderator confirms and hides | ||
| 177 | * Original author notified with reason | ||
| 178 | * Can appeal to board if disputes moderator decision | ||
| 179 | |||
| 180 | **Note**: Content is hidden, not deleted (for audit trail) | ||
| 181 | |||
| 182 | == 3. Contribution Rules == | ||
| 183 | |||
| 184 | === 3.1 All Contributors Must === | ||
| 185 | |||
| 186 | * Provide sources for factual claims | ||
| 187 | * Use clear, neutral language in FactHarbor's own summaries | ||
| 188 | * Respect others and maintain civil discourse | ||
| 189 | * Accept community feedback constructively | ||
| 190 | * Focus on improving quality, not protecting ego | ||
| 191 | |||
| 192 | === 3.2 AKEL (AI System) === | ||
| 193 | |||
| 194 | **AKEL is the primary system**. Human contributions supplement and train AKEL. | ||
| 195 | |||
| 196 | **AKEL Must**: | ||
| 197 | |||
| 198 | * Mark all outputs as AI-generated | ||
| 199 | * Display confidence scores prominently | ||
| 200 | * Provide source citations | ||
| 201 | * Flag uncertainty clearly | ||
| 202 | * Identify contradictions in evidence | ||
| 203 | * Learn from human corrections | ||
| 204 | |||
| 205 | **When AKEL Makes Errors**: | ||
| 206 | |||
| 207 | 1. Capture the error pattern (what, why, how common) | ||
| 208 | 2. Improve the system (better prompt, model, validation) | ||
| 209 | 3. Re-process affected claims automatically | ||
| 210 | 4. Measure improvement (did quality increase?) | ||
| 211 | |||
| 212 | **Human Role**: Train AKEL through corrections, not replace AKEL | ||
| 213 | |||
| 214 | === 3.3 Contributors Should === | ||
| 215 | |||
| 216 | * Improve clarity and structure | ||
| 217 | * Add missing sources | ||
| 218 | * Flag errors for system improvement | ||
| 219 | * Suggest better ways to present information | ||
| 220 | * Participate in quality discussions | ||
| 221 | |||
| 222 | === 3.4 Moderators Must === | ||
| 223 | |||
| 224 | * Be impartial | ||
| 225 | * Document moderation decisions | ||
| 226 | * Respond to appeals promptly | ||
| 227 | * Use automated tools to scale efforts | ||
| 228 | * Focus on abuse/harm, not routine quality control | ||
| 229 | |||
| 230 | == 4. Quality Standards == | ||
| 231 | |||
| 232 | **Fulfills**: UN-5 (Source reliability), UN-6 (Publisher track records), UN-7 (Evidence transparency), UN-9 (Methodology transparency) | ||
| 233 | |||
| 234 | === 4.1 Source Requirements === | ||
| 235 | |||
| 236 | **Track Record Over Credentials**: | ||
| 237 | |||
| 238 | * Sources evaluated by historical accuracy | ||
| 239 | * Correction policy matters | ||
| 240 | * Independence from conflicts of interest | ||
| 241 | * Methodology transparency | ||
| 242 | |||
| 243 | **Source Quality Database**: | ||
| 244 | |||
| 245 | * Automated tracking of source accuracy | ||
| 246 | * Correction frequency | ||
| 247 | * Reliability score (updated continuously) | ||
| 248 | * Users can see source track record | ||
| 249 | |||
| 250 | **No automatic trust** for government, academia, or media - all evaluated by track record. | ||
| 251 | |||
| 252 | **User Needs served**: UN-5 (Source provenance), UN-6 (Publisher reliability) | ||
| 253 | |||
| 254 | === 4.2 Claim Requirements === | ||
| 255 | |||
| 256 | * Clear subject and assertion | ||
| 257 | * Verifiable with available information | ||
| 258 | * Sourced (or explicitly marked as needing sources) | ||
| 259 | * Neutral language in FactHarbor summaries | ||
| 260 | * Appropriate context provided | ||
| 261 | |||
| 262 | **User Needs served**: UN-2 (Claim extraction and verification) | ||
| 263 | |||
| 264 | === 4.3 Evidence Requirements === | ||
| 265 | |||
| 266 | * Publicly accessible (or explain why not) | ||
| 267 | * Properly cited with attribution | ||
| 268 | * Relevant to claim being evaluated | ||
| 269 | * Original source preferred over secondary | ||
| 270 | |||
| 271 | **User Needs served**: UN-7 (Evidence transparency) | ||
| 272 | |||
| 273 | === 4.4 Confidence Scoring === | ||
| 274 | |||
| 275 | **Automated confidence calculation based on**: | ||
| 276 | |||
| 277 | * Source quality scores | ||
| 278 | * Evidence consistency | ||
| 279 | * Contradiction detection | ||
| 280 | * Completeness of analysis | ||
| 281 | * Historical accuracy of similar claims | ||
| 282 | |||
| 283 | **Thresholds**: | ||
| 284 | |||
| 285 | * < 40%: Too low to publish (needs improvement) | ||
| 286 | * 40-60%: Published with "Low confidence" warning | ||
| 287 | * 60-80%: Published as standard | ||
| 288 | * 80-100%: Published as "High confidence" | ||
| 289 | |||
| 290 | **User Needs served**: UN-1 (Trust assessment), UN-9 (Methodology transparency) | ||
| 291 | |||
| 292 | == 5. Automated Risk Scoring == | ||
| 293 | |||
| 294 | **Fulfills**: UN-10 (Manipulation detection), UN-16 (Appropriate review level) | ||
| 295 | |||
| 296 | **Replace manual risk tiers with continuous automated scoring**. | ||
| 297 | |||
| 298 | === 5.1 Risk Score Calculation === | ||
| 299 | |||
| 300 | **Factors** (weighted algorithm): | ||
| 301 | |||
| 302 | * **Domain sensitivity**: Medical, legal, safety auto-flagged higher | ||
| 303 | * **Potential impact**: Views, citations, spread | ||
| 304 | * **Controversy level**: Flags, disputes, edit wars | ||
| 305 | * **Uncertainty**: Low confidence, contradictory evidence | ||
| 306 | * **Source reliability**: Track record of sources used | ||
| 307 | |||
| 308 | **Score**: 0-100 (higher = more risk) | ||
| 309 | |||
| 310 | === 5.2 Automated Actions === | ||
| 311 | |||
| 312 | * **Score > 80**: Flag for moderator review before publication | ||
| 313 | * **Score 60-80**: Publish with prominent warnings | ||
| 314 | * **Score 40-60**: Publish with standard warnings | ||
| 315 | * **Score < 40**: Publish normally | ||
| 316 | |||
| 317 | **Continuous monitoring**: Risk score recalculated as new information emerges | ||
| 318 | |||
| 319 | **User Needs served**: UN-10 (Detect manipulation tactics), UN-16 (Review status) | ||
| 320 | |||
| 321 | == 6. System Improvement Process == | ||
| 322 | |||
| 323 | **Core principle**: Fix the system, not just the data. | ||
| 324 | |||
| 325 | === 6.1 Error Capture === | ||
| 326 | |||
| 327 | **When users flag errors or make corrections**: | ||
| 328 | |||
| 329 | 1. What was wrong? (categorize) | ||
| 330 | 2. What should it have been? | ||
| 331 | 3. Why did the system fail? (root cause) | ||
| 332 | 4. How common is this pattern? | ||
| 333 | 5. Store in ErrorPattern table (improvement queue) | ||
| 334 | |||
| 335 | === 6.2 Continuous Improvement Cycle === | ||
| 336 | |||
| 337 | 1. **Review**: Analyze top error patterns | ||
| 338 | 2. **Develop**: Create fix (prompt, model, validation) | ||
| 339 | 3. **Test**: Validate fix on sample claims | ||
| 340 | 4. **Deploy**: Roll out if quality improves | ||
| 341 | 5. **Re-process**: Automatically update affected claims | ||
| 342 | 6. **Monitor**: Track quality metrics | ||
| 343 | |||
| 344 | === 6.3 Quality Metrics Dashboard === | ||
| 345 | |||
| 346 | **Track continuously**: | ||
| 347 | |||
| 348 | * Error rate by category | ||
| 349 | * Source quality distribution | ||
| 350 | * Confidence score trends | ||
| 351 | * User flag rate (issues found) | ||
| 352 | * Correction acceptance rate | ||
| 353 | * Re-work rate | ||
| 354 | * Claims processed per hour | ||
| 355 | |||
| 356 | **Goal**: continuous improvement in error rate | ||
| 357 | |||
| 358 | == 7. Automated Quality Monitoring == | ||
| 359 | |||
| 360 | **Replace manual audit sampling with automated monitoring**. | ||
| 361 | |||
| 362 | === 7.1 Continuous Metrics === | ||
| 363 | |||
| 364 | * **Source quality**: Track record database | ||
| 365 | * **Consistency**: Contradiction detection | ||
| 366 | * **Clarity**: Readability scores | ||
| 367 | * **Completeness**: Field validation | ||
| 368 | * **Accuracy**: User corrections tracked | ||
| 369 | |||
| 370 | === 7.2 Anomaly Detection === | ||
| 371 | |||
| 372 | **Automated alerts for**: | ||
| 373 | |||
| 374 | * Sudden quality drops | ||
| 375 | * Unusual patterns | ||
| 376 | * Contradiction clusters | ||
| 377 | * Source reliability changes | ||
| 378 | * User behavior anomalies | ||
| 379 | |||
| 380 | === 7.3 Targeted Review === | ||
| 381 | |||
| 382 | * Review only flagged items | ||
| 383 | * Random sampling for calibration (not quotas) | ||
| 384 | * Learn from corrections to improve automation | ||
| 385 | |||
| 386 | == 8. Functional Requirements == | ||
| 387 | |||
| 388 | This section defines specific features that fulfill user needs. | ||
| 389 | |||
| 390 | === 8.1 Claim Intake & Normalization === | ||
| 391 | |||
| 392 | ==== FR1 — Claim Intake ==== | ||
| 393 | |||
| 394 | **Fulfills**: UN-2 (Claim extraction), UN-4 (Quick fact-checking), UN-12 (Submit claims) | ||
| 395 | |||
| 396 | * Users submit claims via simple form or API | ||
| 397 | * Claims can be text, URL, or image | ||
| 398 | * Duplicate detection (semantic similarity) | ||
| 399 | * Auto-categorization by domain | ||
| 400 | |||
| 401 | ==== FR2 — Claim Normalization ==== | ||
| 402 | |||
| 403 | **Fulfills**: UN-2 (Claim verification) | ||
| 404 | |||
| 405 | * Standardize to clear assertion format | ||
| 406 | * Extract key entities (who, what, when, where) | ||
| 407 | * Identify claim type (factual, predictive, evaluative) | ||
| 408 | * Link to existing similar claims | ||
| 409 | |||
| 410 | ==== FR3 — Claim Classification ==== | ||
| 411 | |||
| 412 | **Fulfills**: UN-11 (Filtered research) | ||
| 413 | |||
| 414 | * Domain: Politics, Science, Health, etc. | ||
| 415 | * Type: Historical fact, current stat, prediction, etc. | ||
| 416 | * Risk score: Automated calculation | ||
| 417 | * Complexity: Simple, moderate, complex | ||
| 418 | |||
| 419 | === 8.2 Scenario System === | ||
| 420 | |||
| 421 | ==== FR4 — Scenario Generation ==== | ||
| 422 | |||
| 423 | **Fulfills**: UN-2 (Context-dependent verification), UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement) | ||
| 424 | |||
| 425 | **Automated scenario creation**: | ||
| 426 | |||
| 427 | * AKEL analyzes claim and generates likely scenarios (use-cases and contexts) | ||
| 428 | * Each scenario includes: assumptions, definitions, boundaries, evidence context | ||
| 429 | * Users can flag incorrect scenarios | ||
| 430 | * System learns from corrections | ||
| 431 | |||
| 432 | **Key Concept**: Scenarios represent different interpretations or contexts (e.g., "Clinical trials with healthy adults" vs. "Real-world data with diverse populations") | ||
| 433 | |||
| 434 | ==== FR5 — Evidence Linking ==== | ||
| 435 | |||
| 436 | **Fulfills**: UN-5 (Source tracing), UN-7 (Evidence transparency) | ||
| 437 | |||
| 438 | * Automated evidence discovery from sources | ||
| 439 | * Relevance scoring | ||
| 440 | * Contradiction detection | ||
| 441 | * Source quality assessment | ||
| 442 | |||
| 443 | ==== FR6 — Scenario Comparison ==== | ||
| 444 | |||
| 445 | **Fulfills**: UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement) | ||
| 446 | |||
| 447 | * Side-by-side comparison interface | ||
| 448 | * Highlight key differences between scenarios | ||
| 449 | * Show evidence supporting each scenario | ||
| 450 | * Display confidence scores per scenario | ||
| 451 | |||
| 452 | === 8.3 Verdicts & Analysis === | ||
| 453 | |||
| 454 | ==== FR7 — Automated Verdicts ==== | ||
| 455 | |||
| 456 | **Fulfills**: UN-1 (Trust score), UN-2 (Verification verdicts), UN-3 (Article summary with FactHarbor analysis summary), UN-13 (Cite verdicts) | ||
| 457 | |||
| 458 | * AKEL generates verdict based on evidence within each scenario | ||
| 459 | * **Likelihood range** displayed (e.g., "0.70-0.85 (likely true)") - NOT binary true/false | ||
| 460 | * **Uncertainty factors** explicitly listed (e.g., "Small sample sizes", "Long-term effects unknown") | ||
| 461 | * Confidence score displayed prominently | ||
| 462 | * Source quality indicators shown | ||
| 463 | * Contradictions noted | ||
| 464 | * Uncertainty acknowledged | ||
| 465 | |||
| 466 | **Key Innovation**: Detailed probabilistic verdicts with explicit uncertainty, not binary judgments | ||
| 467 | |||
| 468 | ==== FR8 — Time Evolution ==== | ||
| 469 | |||
| 470 | {{warning}} | ||
| 471 | **Status:** Deferred (Not in V1.0) | ||
| 472 | |||
| 473 | This requirement has been **dropped from the current architecture and design**. Versioned entities have been replaced with simple edit history tracking only. Full evolution timeline functionality is deferred to future releases beyond V1.0. | ||
| 474 | {{/warning}} | ||
| 475 | |||
| 476 | **Fulfills**: UN-15 (Verdict evolution timeline) | ||
| 477 | |||
| 478 | * Claims and verdicts update as new evidence emerges | ||
| 479 | * Version history maintained for all verdicts | ||
| 480 | * Changes highlighted | ||
| 481 | * Confidence score trends visible | ||
| 482 | * Users can see "as of date X, what did we know?" | ||
| 483 | |||
| 484 | === 8.4 User Interface & Presentation === | ||
| 485 | |||
| 486 | ==== FR12 — Two-Panel Summary View (Article Summary with FactHarbor Analysis Summary) ==== | ||
| 487 | |||
| 488 | **Fulfills**: UN-3 (Article Summary with FactHarbor Analysis Summary) | ||
| 489 | |||
| 490 | **Purpose**: Provide side-by-side comparison of what a document claims vs. FactHarbor's complete analysis of its credibility | ||
| 491 | |||
| 492 | **Left Panel: Article Summary**: | ||
| 493 | |||
| 494 | * Document title, source, and claimed credibility | ||
| 495 | * "The Big Picture" - main thesis or position change | ||
| 496 | * "Key Findings" - structured summary of document's main claims | ||
| 497 | * "Reasoning" - document's explanation for positions | ||
| 498 | * "Conclusion" - document's bottom line | ||
| 499 | |||
| 500 | **Right Panel: FactHarbor Analysis Summary**: | ||
| 501 | |||
| 502 | * FactHarbor's independent source credibility assessment | ||
| 503 | * Claim-by-claim verdicts with confidence scores | ||
| 504 | * Methodology assessment (strengths, limitations) | ||
| 505 | * Overall verdict on document quality | ||
| 506 | * Analysis ID for reference | ||
| 507 | |||
| 508 | **Design Principles**: | ||
| 509 | |||
| 510 | * No scrolling required - both panels visible simultaneously | ||
| 511 | * Visual distinction between "what they say" and "FactHarbor's analysis" | ||
| 512 | * Color coding for verdicts (supported, uncertain, refuted) | ||
| 513 | * Confidence percentages clearly visible | ||
| 514 | * Mobile responsive (panels stack vertically on small screens) | ||
| 515 | |||
| 516 | **Implementation Notes**: | ||
| 517 | |||
| 518 | * Generated automatically by AKEL for every analyzed document | ||
| 519 | * Updates when verdict evolves (maintains version history) | ||
| 520 | * Exportable as standalone summary report | ||
| 521 | * Shareable via permanent URL | ||
| 522 | |||
| 523 | ==== FR13 — In-Article Claim Highlighting ==== | ||
| 524 | |||
| 525 | **Fulfills**: UN-17 (In-article claim highlighting) | ||
| 526 | |||
| 527 | **Purpose**: Enable readers to quickly assess claim credibility while reading by visually highlighting factual claims with color-coded indicators | ||
| 528 | |||
| 529 | ==== Visual Example: Article with Highlighted Claims ==== | ||
| 530 | |||
| 531 | (% class="box" %) | ||
| 532 | ((( | ||
| 533 | **Article: "New Study Shows Benefits of Mediterranean Diet"** | ||
| 534 | |||
| 535 | A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet. | ||
| 536 | |||
| 537 | (% class="box successmessage" style="margin:10px 0;" %) | ||
| 538 | ((( | ||
| 539 | 🟢 **Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups** | ||
| 540 | |||
| 541 | (% style="font-size:0.9em; color:#666;" %) | ||
| 542 | ↑ WELL SUPPORTED • 87% confidence | ||
| 543 | [[Click for evidence details →]] | ||
| 544 | |||
| 545 | |||
| 546 | ))) | ||
| 547 | |||
| 548 | The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers. | ||
| 549 | |||
| 550 | (% class="box warningmessage" style="margin:10px 0;" %) | ||
| 551 | ((( | ||
| 552 | 🟡 **Some experts believe this diet can completely prevent heart attacks** | ||
| 553 | |||
| 554 | (% style="font-size:0.9em; color:#666;" %) | ||
| 555 | ↑ UNCERTAIN • 45% confidence | ||
| 556 | Overstated - evidence shows risk reduction, not prevention | ||
| 557 | [[Click for details →]] | ||
| 558 | |||
| 559 | |||
| 560 | ))) | ||
| 561 | |||
| 562 | Dr. Maria Rodriguez, lead researcher, recommends incorporating more olive oil, fish, and vegetables into daily meals. | ||
| 563 | |||
| 564 | (% class="box errormessage" style="margin:10px 0;" %) | ||
| 565 | ((( | ||
| 566 | 🔴 **The study proves that saturated fats cause heart disease** | ||
| 567 | |||
| 568 | (% style="font-size:0.9em; color:#666;" %) | ||
| 569 | ↑ REFUTED • 15% confidence | ||
| 570 | Claim not supported by study design; correlation ≠ causation | ||
| 571 | [[Click for counter-evidence →]] | ||
| 572 | |||
| 573 | |||
| 574 | ))) | ||
| 575 | |||
| 576 | Participants also reported feeling more energetic and experiencing better sleep quality, though these were secondary measures. | ||
| 577 | ))) | ||
| 578 | |||
| 579 | **Legend:** | ||
| 580 | |||
| 581 | * 🟢 = Well-supported claim (confidence ≥75%) | ||
| 582 | * 🟡 = Uncertain claim (confidence 40-74%) | ||
| 583 | * 🔴 = Refuted/unsupported claim (confidence <40%) | ||
| 584 | * Plain text = Non-factual content (context, opinions, recommendations) | ||
| 585 | |||
| 586 | ==== Tooltip on Hover/Click ==== | ||
| 587 | |||
| 588 | (% class="box infomessage" %) | ||
| 589 | ((( | ||
| 590 | **FactHarbor Analysis** | ||
| 591 | |||
| 592 | **Claim:** | ||
| 593 | "Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease" | ||
| 594 | |||
| 595 | **Verdict:** WELL SUPPORTED | ||
| 596 | **Confidence:** 87% | ||
| 597 | |||
| 598 | **Evidence Summary:** | ||
| 599 | |||
| 600 | * Meta-analysis of 12 RCTs confirms 23-28% risk reduction | ||
| 601 | * Consistent findings across multiple populations | ||
| 602 | * Published in peer-reviewed journal (high credibility) | ||
| 603 | |||
| 604 | **Uncertainty Factors:** | ||
| 605 | |||
| 606 | * Exact percentage varies by study (20-30% range) | ||
| 607 | |||
| 608 | [[View Full Analysis →]] | ||
| 609 | ))) | ||
| 610 | |||
| 611 | **Color-Coding System**: | ||
| 612 | |||
| 613 | * **Green**: Well-supported claims (confidence ≥75%, strong evidence) | ||
| 614 | * **Yellow/Orange**: Uncertain claims (confidence 40-74%, conflicting or limited evidence) | ||
| 615 | * **Red**: Refuted or unsupported claims (confidence <40%, contradicted by evidence) | ||
| 616 | * **Gray/Neutral**: Non-factual content (opinions, questions, procedural text) | ||
| 617 | |||
| 618 | ==== Interactive Highlighting Example (Detailed View) ==== | ||
| 619 | |||
| 620 | (% style="width:100%; border-collapse:collapse;" %) | ||
| 621 | |=**Article Text**|=**Status**|=**Analysis** | ||
| 622 | |((( | ||
| 623 | A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet. | ||
| 624 | )))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Context - no highlighting | ||
| 625 | |((( | ||
| 626 | //Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups// | ||
| 627 | )))|(% style="background-color:#D4EDDA; text-align:center; padding:8px;" %)🟢 **WELL SUPPORTED**|((( | ||
| 628 | **87% confidence** | ||
| 629 | |||
| 630 | Meta-analysis of 12 RCTs confirms 23-28% risk reduction | ||
| 631 | |||
| 632 | [[View Full Analysis]] | ||
| 633 | ))) | ||
| 634 | |((( | ||
| 635 | The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers. | ||
| 636 | )))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Methodology - no highlighting | ||
| 637 | |((( | ||
| 638 | //Some experts believe this diet can completely prevent heart attacks// | ||
| 639 | )))|(% style="background-color:#FFF3CD; text-align:center; padding:8px;" %)🟡 **UNCERTAIN**|((( | ||
| 640 | **45% confidence** | ||
| 641 | |||
| 642 | Overstated - evidence shows risk reduction, not prevention | ||
| 643 | |||
| 644 | [[View Details]] | ||
| 645 | ))) | ||
| 646 | |((( | ||
| 647 | Dr. Rodriguez recommends incorporating more olive oil, fish, and vegetables into daily meals. | ||
| 648 | )))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Recommendation - no highlighting | ||
| 649 | |((( | ||
| 650 | //The study proves that saturated fats cause heart disease// | ||
| 651 | )))|(% style="background-color:#F8D7DA; text-align:center; padding:8px;" %)🔴 **REFUTED**|((( | ||
| 652 | **15% confidence** | ||
| 653 | |||
| 654 | Claim not supported by study; correlation ≠ causation | ||
| 655 | |||
| 656 | [[View Counter-Evidence]] | ||
| 657 | ))) | ||
| 658 | |||
| 659 | **Design Notes:** | ||
| 660 | |||
| 661 | * Highlighted claims use italics to distinguish from plain text | ||
| 662 | * Color backgrounds match XWiki message box colors (success/warning/error) | ||
| 663 | * Status column shows verdict prominently | ||
| 664 | * Analysis column provides quick summary with link to details | ||
| 665 | |||
| 666 | **User Actions**: | ||
| 667 | |||
| 668 | * **Hover** over highlighted claim → Tooltip appears | ||
| 669 | * **Click** highlighted claim → Detailed analysis modal/panel | ||
| 670 | * **Toggle** button to turn highlighting on/off | ||
| 671 | * **Keyboard**: Tab through highlighted claims | ||
| 672 | |||
| 673 | **Interaction Design**: | ||
| 674 | |||
| 675 | * Hover/click on highlighted claim → Show tooltip with: | ||
| 676 | * Claim text | ||
| 677 | * Verdict (e.g., "WELL SUPPORTED") | ||
| 678 | * Confidence score (e.g., "85%") | ||
| 679 | * Brief evidence summary | ||
| 680 | * Link to detailed analysis | ||
| 681 | * Toggle highlighting on/off (user preference) | ||
| 682 | * Adjustable color intensity for accessibility | ||
| 683 | |||
| 684 | **Technical Requirements**: | ||
| 685 | |||
| 686 | * Real-time highlighting as page loads (non-blocking) | ||
| 687 | * Claim boundary detection (start/end of assertion) | ||
| 688 | * Handle nested or overlapping claims | ||
| 689 | * Preserve original article formatting | ||
| 690 | * Work with various content formats (HTML, plain text, PDFs) | ||
| 691 | |||
| 692 | **Performance Requirements**: | ||
| 693 | |||
| 694 | * Highlighting renders within 500ms of page load | ||
| 695 | * No perceptible delay in reading experience | ||
| 696 | * Efficient DOM manipulation (avoid reflows) | ||
| 697 | |||
| 698 | **Accessibility**: | ||
| 699 | |||
| 700 | * Color-blind friendly palette (use patterns/icons in addition to color) | ||
| 701 | * Screen reader compatible (ARIA labels for claim credibility) | ||
| 702 | * Keyboard navigation to highlighted claims | ||
| 703 | |||
| 704 | **Implementation Notes**: | ||
| 705 | |||
| 706 | * Claims extracted and analyzed by AKEL during initial processing | ||
| 707 | * Highlighting data stored as annotations with byte offsets | ||
| 708 | * Client-side rendering of highlights based on verdict data | ||
| 709 | * Mobile responsive (tap instead of hover) | ||
| 710 | |||
| 711 | === 8.5 Workflow & Moderation === | ||
| 712 | |||
| 713 | ==== FR9 — Publication Workflow ==== | ||
| 714 | |||
| 715 | **Fulfills**: UN-1 (Fast access to verified content), UN-16 (Clear review status) | ||
| 716 | |||
| 717 | **Simple flow**: | ||
| 718 | |||
| 719 | 1. Claim submitted | ||
| 720 | 2. AKEL processes (automated) | ||
| 721 | 3. If confidence > threshold: Publish (labeled as AI-generated) | ||
| 722 | 4. If confidence < threshold: Flag for improvement | ||
| 723 | 5. If risk score > threshold: Flag for moderator | ||
| 724 | |||
| 725 | **No multi-stage approval process** | ||
| 726 | |||
| 727 | ==== FR10 — Moderation ==== | ||
| 728 | |||
| 729 | **Focus on abuse, not routine quality**: | ||
| 730 | |||
| 731 | * Automated abuse detection | ||
| 732 | * Moderators handle flags | ||
| 733 | * Quick response to harmful content | ||
| 734 | * Minimal involvement in routine content | ||
| 735 | |||
| 736 | ==== FR11 — Audit Trail ==== | ||
| 737 | |||
| 738 | **Fulfills**: UN-14 (API access to histories), UN-15 (Evolution tracking) | ||
| 739 | |||
| 740 | * All edits logged | ||
| 741 | * Version history public | ||
| 742 | * Moderation decisions documented | ||
| 743 | * System improvements tracked | ||
| 744 | |||
| 745 | == 9. Non-Functional Requirements == | ||
| 746 | |||
| 747 | === 9.1 NFR1 — Performance === | ||
| 748 | |||
| 749 | **Fulfills**: UN-4 (Fast fact-checking), UN-11 (Responsive filtering) | ||
| 750 | |||
| 751 | * Claim processing: < 30 seconds | ||
| 752 | * Search response: < 2 seconds | ||
| 753 | * Page load: < 3 seconds | ||
| 754 | * 99% uptime | ||
| 755 | |||
| 756 | === 9.2 NFR2 — Scalability === | ||
| 757 | |||
| 758 | **Fulfills**: UN-14 (API access at scale) | ||
| 759 | |||
| 760 | * Handle 10,000 claims initially | ||
| 761 | * Scale to 1M+ claims | ||
| 762 | * Support 100K+ concurrent users | ||
| 763 | * Automated processing scales linearly | ||
| 764 | |||
| 765 | === 9.3 NFR3 — Transparency === | ||
| 766 | |||
| 767 | **Fulfills**: UN-7 (Evidence transparency), UN-9 (Methodology transparency), UN-13 (Citable verdicts), UN-15 (Evolution visibility) | ||
| 768 | |||
| 769 | * All algorithms open source | ||
| 770 | * All data exportable | ||
| 771 | * All decisions documented | ||
| 772 | * Quality metrics public | ||
| 773 | |||
| 774 | === 9.4 NFR4 — Security & Privacy === | ||
| 775 | |||
| 776 | * Follow [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]] | ||
| 777 | * Secure authentication | ||
| 778 | * Data encryption | ||
| 779 | * Regular security audits | ||
| 780 | |||
| 781 | === 9.5 NFR5 — Maintainability === | ||
| 782 | |||
| 783 | * Modular architecture | ||
| 784 | * Automated testing | ||
| 785 | * Continuous integration | ||
| 786 | * Comprehensive documentation | ||
| 787 | |||
| 788 | === NFR11: AKEL Quality Assurance Framework === | ||
| 789 | |||
| 790 | **Fulfills:** AI safety, IFCN methodology transparency | ||
| 791 | |||
| 792 | **Specification:** | ||
| 793 | |||
| 794 | Multi-layer AI quality gates to detect hallucinations, low-confidence results, and logical inconsistencies. | ||
| 795 | |||
| 796 | ==== Quality Gate 1: Claim Extraction Validation ==== | ||
| 797 | |||
| 798 | **Purpose:** Ensure extracted claims are factual assertions (not opinions/predictions) | ||
| 799 | |||
| 800 | **Checks:** | ||
| 801 | |||
| 802 | 1. **Factual Statement Test:** Is this verifiable? (Yes/No) | ||
| 803 | 2. **Opinion Detection:** Contains hedging language? ("I think", "probably", "best") | ||
| 804 | 3. **Future Prediction Test:** Makes claims about future events? | ||
| 805 | 4. **Specificity Score:** Contains specific entities, numbers, dates? | ||
| 806 | |||
| 807 | **Thresholds:** | ||
| 808 | |||
| 809 | * Factual: Must be "Yes" | ||
| 810 | * Opinion markers: <2 hedging phrases | ||
| 811 | * Specificity: ≥3 specific elements | ||
| 812 | |||
| 813 | **Action if Failed:** Flag as "Non-verifiable", do NOT generate verdict | ||
| 814 | |||
| 815 | ==== Quality Gate 2: Evidence Relevance Validation ==== | ||
| 816 | |||
| 817 | **Purpose:** Ensure AI-linked evidence actually relates to claim | ||
| 818 | |||
| 819 | **Checks:** | ||
| 820 | |||
| 821 | 1. **Semantic Similarity Score:** Evidence vs. claim (embeddings) | ||
| 822 | 2. **Entity Overlap:** Shared people/places/things? | ||
| 823 | 3. **Topic Relevance:** Discusses claim subject? | ||
| 824 | |||
| 825 | **Thresholds:** | ||
| 826 | |||
| 827 | * Similarity: ≥0.6 (cosine similarity) | ||
| 828 | * Entity overlap: ≥1 shared entity | ||
| 829 | * Topic relevance: ≥0.5 | ||
| 830 | |||
| 831 | **Action if Failed:** Discard irrelevant evidence | ||
| 832 | |||
| 833 | ==== Quality Gate 3: Scenario Coherence Check ==== | ||
| 834 | |||
| 835 | **Purpose:** Validate scenario assumptions are logical and complete | ||
| 836 | |||
| 837 | **Checks:** | ||
| 838 | |||
| 839 | 1. **Completeness:** All required fields populated | ||
| 840 | 2. **Internal Consistency:** Assumptions don't contradict | ||
| 841 | 3. **Distinguishability:** Scenarios meaningfully different | ||
| 842 | |||
| 843 | **Thresholds:** | ||
| 844 | |||
| 845 | * Required fields: 100% | ||
| 846 | * Contradiction score: <0.3 | ||
| 847 | * Scenario similarity: <0.8 | ||
| 848 | |||
| 849 | **Action if Failed:** Merge duplicates, reduce confidence -20% | ||
| 850 | |||
| 851 | ==== Quality Gate 4: Verdict Confidence Assessment ==== | ||
| 852 | |||
| 853 | **Purpose:** Only publish high-confidence verdicts | ||
| 854 | |||
| 855 | **Checks:** | ||
| 856 | |||
| 857 | 1. **Evidence Count:** Minimum 2 sources | ||
| 858 | 2. **Source Quality:** Average reliability ≥0.6 | ||
| 859 | 3. **Evidence Agreement:** Supporting vs. contradicting ≥0.6 | ||
| 860 | 4. **Uncertainty Factors:** Hedging in reasoning | ||
| 861 | |||
| 862 | **Confidence Tiers:** | ||
| 863 | |||
| 864 | * **HIGH (80-100%):** ≥3 sources, ≥0.7 quality, ≥80% agreement | ||
| 865 | * **MEDIUM (50-79%):** ≥2 sources, ≥0.6 quality, ≥60% agreement | ||
| 866 | * **LOW (0-49%):** <2 sources OR low quality/agreement | ||
| 867 | * **INSUFFICIENT:** <2 sources → DO NOT PUBLISH | ||
| 868 | |||
| 869 | **Implementation Phases:** | ||
| 870 | |||
| 871 | * **POC1:** Gates 1 & 4 only (basic validation) | ||
| 872 | * **POC2:** All 4 gates (complete framework) | ||
| 873 | * **V1.0:** Hardened with <5% hallucination rate | ||
| 874 | |||
| 875 | **Acceptance Criteria:** | ||
| 876 | |||
| 877 | * ✅ All gates operational | ||
| 878 | * ✅ Hallucination rate <5% | ||
| 879 | * ✅ Quality metrics public | ||
| 880 | |||
| 881 | === NFR12: Security Controls === | ||
| 882 | |||
| 883 | **Fulfills:** Data protection, system integrity, user privacy, production readiness | ||
| 884 | |||
| 885 | **Purpose:** Protect FactHarbor systems, user data, and operations from security threats, ensuring production-grade security posture. | ||
| 886 | |||
| 887 | **Specification:** | ||
| 888 | |||
| 889 | ==== API Security ==== | ||
| 890 | |||
| 891 | **Rate Limiting:** | ||
| 892 | |||
| 893 | * **Analysis endpoints:** 100 requests/hour per IP | ||
| 894 | * **Read endpoints:** 1,000 requests/hour per IP | ||
| 895 | * **Search:** 500 requests/hour per IP | ||
| 896 | * **Authenticated users:** 5x higher limits | ||
| 897 | * **Burst protection:** Max 10 requests/second | ||
| 898 | |||
| 899 | **Authentication & Authorization:** | ||
| 900 | |||
| 901 | * **API Keys:** Required for programmatic access | ||
| 902 | * **JWT tokens:** For user sessions (1-hour expiry) | ||
| 903 | * **OAuth2:** For third-party integrations | ||
| 904 | * **Role-Based Access Control (RBAC):** | ||
| 905 | * Public: Read-only access to published claims | ||
| 906 | * Contributor: Submit claims, provide evidence | ||
| 907 | * Moderator: Review contributions, manage quality | ||
| 908 | * Admin: System configuration, user management | ||
| 909 | |||
| 910 | **CORS Policies:** | ||
| 911 | |||
| 912 | * Whitelist approved domains only | ||
| 913 | * No wildcard origins in production | ||
| 914 | * Credentials required for sensitive endpoints | ||
| 915 | |||
| 916 | **Input Sanitization:** | ||
| 917 | |||
| 918 | * Validate all user input against schemas | ||
| 919 | * Sanitize HTML/JavaScript in text submissions | ||
| 920 | * Prevent SQL injection (use parameterized queries) | ||
| 921 | * Prevent command injection (no shell execution of user input) | ||
| 922 | * Max request size: 10MB | ||
| 923 | * File upload restrictions: Whitelist file types, scan for malware | ||
| 924 | |||
| 925 | ---- | ||
| 926 | |||
| 927 | ==== Data Security ==== | ||
| 928 | |||
| 929 | **Encryption at Rest:** | ||
| 930 | |||
| 931 | * Database encryption using AES-256 | ||
| 932 | * Encrypted backups | ||
| 933 | * Key management via cloud provider KMS (AWS KMS, Google Cloud KMS) | ||
| 934 | * Regular key rotation (90-day cycle) | ||
| 935 | |||
| 936 | **Encryption in Transit:** | ||
| 937 | |||
| 938 | * HTTPS/TLS 1.3 only (no TLS 1.0/1.1) | ||
| 939 | * Strong cipher suites only | ||
| 940 | * HSTS (HTTP Strict Transport Security) enabled | ||
| 941 | * Certificate pinning for mobile apps | ||
| 942 | |||
| 943 | **Secure Credential Storage:** | ||
| 944 | |||
| 945 | * Passwords hashed with bcrypt (cost factor 12+) | ||
| 946 | * API keys encrypted in database | ||
| 947 | * Secrets stored in environment variables (never in code) | ||
| 948 | * Use secrets manager (AWS Secrets Manager, HashiCorp Vault) | ||
| 949 | |||
| 950 | **Data Privacy:** | ||
| 951 | |||
| 952 | * Minimal data collection (privacy by design) | ||
| 953 | * User data deletion on request (GDPR compliance) | ||
| 954 | * PII encryption in database | ||
| 955 | * Anonymize logs (no PII in log files) | ||
| 956 | |||
| 957 | ---- | ||
| 958 | |||
| 959 | ==== Application Security ==== | ||
| 960 | |||
| 961 | **OWASP Top 10 Compliance:** | ||
| 962 | |||
| 963 | 1. **Broken Access Control:** RBAC implementation, path traversal prevention | ||
| 964 | 2. **Cryptographic Failures:** Strong encryption, secure key management | ||
| 965 | 3. **Injection:** Parameterized queries, input validation | ||
| 966 | 4. **Insecure Design:** Security review of all features | ||
| 967 | 5. **Security Misconfiguration:** Hardened defaults, security headers | ||
| 968 | 6. **Vulnerable Components:** Dependency scanning (see below) | ||
| 969 | 7. **Authentication Failures:** Strong password policy, MFA support | ||
| 970 | 8. **Data Integrity Failures:** Signature verification, checksums | ||
| 971 | 9. **Security Logging Failures:** Comprehensive audit logs | ||
| 972 | 10. **Server-Side Request Forgery:** URL validation, whitelist domains | ||
| 973 | |||
| 974 | **Security Headers:** | ||
| 975 | |||
| 976 | * `Content-Security-Policy`: Strict CSP to prevent XSS | ||
| 977 | * `X-Frame-Options`: DENY (prevent clickjacking) | ||
| 978 | * `X-Content-Type-Options`: nosniff | ||
| 979 | * `Referrer-Policy`: strict-origin-when-cross-origin | ||
| 980 | * `Permissions-Policy`: Restrict browser features | ||
| 981 | |||
| 982 | **Dependency Vulnerability Scanning:** | ||
| 983 | |||
| 984 | * **Tools:** Snyk, Dependabot, npm audit, pip-audit | ||
| 985 | * **Frequency:** Daily automated scans | ||
| 986 | * **Action:** Patch critical vulnerabilities within 24 hours | ||
| 987 | * **Policy:** No known high/critical CVEs in production | ||
| 988 | |||
| 989 | **Security Audits:** | ||
| 990 | |||
| 991 | * **Internal:** Quarterly security reviews | ||
| 992 | * **External:** Annual penetration testing by certified firm | ||
| 993 | * **Bug Bounty:** Public bug bounty program (V1.1+) | ||
| 994 | * **Compliance:** SOC 2 Type II certification target (V1.5) | ||
| 995 | |||
| 996 | ---- | ||
| 997 | |||
| 998 | ==== Operational Security ==== | ||
| 999 | |||
| 1000 | **DDoS Protection:** | ||
| 1001 | |||
| 1002 | * CloudFlare or AWS Shield | ||
| 1003 | * Rate limiting at CDN layer | ||
| 1004 | * Automatic IP blocking for abuse patterns | ||
| 1005 | |||
| 1006 | **Monitoring & Alerting:** | ||
| 1007 | |||
| 1008 | * Real-time security event monitoring | ||
| 1009 | * Alerts for: | ||
| 1010 | * Failed login attempts (>5 in 10 minutes) | ||
| 1011 | * API abuse patterns | ||
| 1012 | * Unusual data access patterns | ||
| 1013 | * Security scan detections | ||
| 1014 | * Integration with SIEM (Security Information and Event Management) | ||
| 1015 | |||
| 1016 | **Incident Response:** | ||
| 1017 | |||
| 1018 | * Documented incident response plan | ||
| 1019 | * Security incident classification (P1-P4) | ||
| 1020 | * On-call rotation for security issues | ||
| 1021 | * Post-mortem for all security incidents | ||
| 1022 | * Public disclosure policy (coordinated disclosure) | ||
| 1023 | |||
| 1024 | **Backup & Recovery:** | ||
| 1025 | |||
| 1026 | * Daily encrypted backups | ||
| 1027 | * 30-day retention period | ||
| 1028 | * Tested recovery procedures (quarterly) | ||
| 1029 | * Disaster recovery plan (RTO: 4 hours, RPO: 1 hour) | ||
| 1030 | |||
| 1031 | ---- | ||
| 1032 | |||
| 1033 | ==== Compliance & Standards ==== | ||
| 1034 | |||
| 1035 | **GDPR Compliance:** | ||
| 1036 | |||
| 1037 | * User consent management | ||
| 1038 | * Right to access data | ||
| 1039 | * Right to deletion | ||
| 1040 | * Data portability | ||
| 1041 | * Privacy policy published | ||
| 1042 | |||
| 1043 | **Accessibility:** | ||
| 1044 | |||
| 1045 | * WCAG 2.1 AA compliance | ||
| 1046 | * Screen reader compatibility | ||
| 1047 | * Keyboard navigation | ||
| 1048 | * Alt text for images | ||
| 1049 | |||
| 1050 | **Browser Support:** | ||
| 1051 | |||
| 1052 | * Modern browsers only (Chrome/Edge/Firefox/Safari latest 2 versions) | ||
| 1053 | * No IE11 support | ||
| 1054 | |||
| 1055 | **Acceptance Criteria:** | ||
| 1056 | |||
| 1057 | * ✅ Passes OWASP ZAP security scan (no high/critical findings) | ||
| 1058 | * ✅ All dependencies with known vulnerabilities patched | ||
| 1059 | * ✅ Penetration test completed with no critical findings | ||
| 1060 | * ✅ Rate limiting blocks abuse attempts | ||
| 1061 | * ✅ Encryption at rest and in transit verified | ||
| 1062 | * ✅ Security headers scored A+ on securityheaders.com | ||
| 1063 | * ✅ Incident response plan documented and tested | ||
| 1064 | * ✅ 95% uptime over 30-day period | ||
| 1065 | |||
| 1066 | === NFR13: Quality Metrics Transparency === | ||
| 1067 | |||
| 1068 | **Fulfills:** User trust, transparency, continuous improvement, IFCN methodology transparency | ||
| 1069 | |||
| 1070 | **Purpose:** Provide transparent, measurable quality metrics that demonstrate AKEL's performance and build user trust in automated fact-checking. | ||
| 1071 | |||
| 1072 | **Specification:** | ||
| 1073 | |||
| 1074 | ==== Component: Public Quality Dashboard ==== | ||
| 1075 | |||
| 1076 | **Core Metrics to Display:** | ||
| 1077 | |||
| 1078 | * \\ | ||
| 1079 | ** \\ | ||
| 1080 | **1. Verdict Quality Metrics | ||
| 1081 | |||
| 1082 | **TIGERScore (Fact-Checking Quality):** | ||
| 1083 | |||
| 1084 | * **Definition:** Measures how well generated verdicts match expert fact-checker judgments | ||
| 1085 | * **Scale:** 0-100 (higher is better) | ||
| 1086 | * **Calculation:** Using TIGERScore framework (Truth-conditional accuracy, Informativeness, Generality, Evaluativeness, Relevance) | ||
| 1087 | * **Target:** Average ≥80 for production release | ||
| 1088 | * **Display:** | ||
| 1089 | {{code}}Verdict Quality (TIGERScore): | ||
| 1090 | Overall: 84.2 ▲ (+2.1 from last month) | ||
| 1091 | |||
| 1092 | Distribution: | ||
| 1093 | Excellent (>80): 67% | ||
| 1094 | Good (60-80): 28% | ||
| 1095 | Needs Improvement (<60): 5% | ||
| 1096 | |||
| 1097 | Trend: [Graph showing improvement over time]{{/code}} | ||
| 1098 | |||
| 1099 | **2. Hallucination & Faithfulness Metrics** | ||
| 1100 | |||
| 1101 | **AlignScore (Faithfulness to Evidence):** | ||
| 1102 | |||
| 1103 | * **Definition:** Measures how well verdicts align with actual evidence content | ||
| 1104 | * **Scale:** 0-1 (higher is better) | ||
| 1105 | * **Purpose:** Detect AI hallucinations (making claims not supported by evidence) | ||
| 1106 | * **Target:** Average ≥0.85, hallucination rate <5% | ||
| 1107 | * **Display:** | ||
| 1108 | {{code}}Evidence Faithfulness (AlignScore): | ||
| 1109 | Average: 0.87 ▼ (-0.02 from last month) | ||
| 1110 | |||
| 1111 | Hallucination Rate: 4.2% | ||
| 1112 | - Claims without evidence support: 3.1% | ||
| 1113 | - Misrepresented evidence: 1.1% | ||
| 1114 | |||
| 1115 | Action: Prompt engineering review scheduled{{/code}} | ||
| 1116 | |||
| 1117 | **3. Evidence Quality Metrics** | ||
| 1118 | |||
| 1119 | **Source Reliability:** | ||
| 1120 | |||
| 1121 | * Average source quality score (0-1 scale) | ||
| 1122 | * Distribution of high/medium/low quality sources | ||
| 1123 | * Publisher track record trends | ||
| 1124 | |||
| 1125 | **Evidence Coverage:** | ||
| 1126 | |||
| 1127 | * Average number of sources per claim | ||
| 1128 | * Percentage of claims with ≥2 sources (EFCSN minimum) | ||
| 1129 | * Geographic diversity of sources | ||
| 1130 | |||
| 1131 | **Display:** | ||
| 1132 | {{code}}Evidence Quality: | ||
| 1133 | |||
| 1134 | Average Sources per Claim: 4.2 | ||
| 1135 | Claims with ≥2 sources: 94% (EFCSN compliant) | ||
| 1136 | |||
| 1137 | Source Quality Distribution: | ||
| 1138 | High quality (>0.8): 48% | ||
| 1139 | Medium quality (0.5-0.8): 43% | ||
| 1140 | Low quality (<0.5): 9% | ||
| 1141 | |||
| 1142 | Geographic Diversity: 23 countries represented{{/code}} | ||
| 1143 | |||
| 1144 | **4. Contributor Consensus Metrics** (when human reviewers involved) | ||
| 1145 | |||
| 1146 | **Inter-Rater Reliability (IRR):** | ||
| 1147 | |||
| 1148 | * **Calculation:** Cohen's Kappa or Fleiss' Kappa for multiple raters | ||
| 1149 | * **Scale:** 0-1 (higher is better) | ||
| 1150 | * **Interpretation:** | ||
| 1151 | * >0.8: Almost perfect agreement | ||
| 1152 | * 0.6-0.8: Substantial agreement | ||
| 1153 | * 0.4-0.6: Moderate agreement | ||
| 1154 | * <0.4: Poor agreement | ||
| 1155 | * **Target:** Maintain ≥0.7 (substantial agreement) | ||
| 1156 | |||
| 1157 | **Display:** | ||
| 1158 | {{code}}Contributor Consensus: | ||
| 1159 | |||
| 1160 | Inter-Rater Reliability (IRR): 0.73 (Substantial agreement) | ||
| 1161 | - Verdict agreement: 78% | ||
| 1162 | - Evidence quality agreement: 71% | ||
| 1163 | - Scenario structure agreement: 69% | ||
| 1164 | |||
| 1165 | Cases requiring moderator review: 12 | ||
| 1166 | Moderator override rate: 8%{{/code}} | ||
| 1167 | |||
| 1168 | ---- | ||
| 1169 | |||
| 1170 | ==== Quality Dashboard Implementation ==== | ||
| 1171 | |||
| 1172 | **Dashboard Location:** `/quality-metrics` | ||
| 1173 | |||
| 1174 | **Update Frequency:** | ||
| 1175 | |||
| 1176 | * **POC2:** Weekly manual updates | ||
| 1177 | * **Beta 0:** Daily automated updates | ||
| 1178 | * **V1.0:** Real-time metrics (updated hourly) | ||
| 1179 | |||
| 1180 | **Dashboard Sections:** | ||
| 1181 | |||
| 1182 | 1. **Overview:** Key metrics at a glance | ||
| 1183 | 2. **Verdict Quality:** TIGERScore trends and distributions | ||
| 1184 | 3. **Evidence Analysis:** Source quality and coverage | ||
| 1185 | 4. **AI Performance:** Hallucination rates, AlignScore | ||
| 1186 | 5. **Human Oversight:** Contributor consensus, review rates | ||
| 1187 | 6. **System Health:** Processing times, error rates, uptime | ||
| 1188 | |||
| 1189 | **Example Dashboard Layout:** | ||
| 1190 | |||
| 1191 | {{code}} | ||
| 1192 | ┌─────────────────────────────────────────────────────────────┐ | ||
| 1193 | │ FactHarbor Quality Metrics Last updated: │ | ||
| 1194 | │ Public Dashboard 2 hours ago │ | ||
| 1195 | └─────────────────────────────────────────────────────────────┘ | ||
| 1196 | |||
| 1197 | 📊 KEY METRICS | ||
| 1198 | ───────────────────────────────────────────────────────────── | ||
| 1199 | TIGERScore (Verdict Quality): 84.2 ▲ (+2.1) | ||
| 1200 | AlignScore (Faithfulness): 0.87 ▼ (-0.02) | ||
| 1201 | Hallucination Rate: 4.2% ✓ (Target: <5%) | ||
| 1202 | Average Sources per Claim: 4.2 ▲ (+0.3) | ||
| 1203 | |||
| 1204 | 📈 TRENDS (30 days) | ||
| 1205 | ───────────────────────────────────────────────────────────── | ||
| 1206 | [Graph: TIGERScore trending upward] | ||
| 1207 | [Graph: Hallucination rate declining] | ||
| 1208 | [Graph: Evidence quality stable] | ||
| 1209 | |||
| 1210 | ⚠️ IMPROVEMENT TARGETS | ||
| 1211 | ───────────────────────────────────────────────────────────── | ||
| 1212 | 1. Reduce hallucination rate to <3% (Current: 4.2%) | ||
| 1213 | 2. Increase TIGERScore average to >85 (Current: 84.2) | ||
| 1214 | 3. Maintain IRR >0.75 (Current: 0.73) | ||
| 1215 | |||
| 1216 | 📄 DETAILED REPORTS | ||
| 1217 | ───────────────────────────────────────────────────────────── | ||
| 1218 | • Monthly Quality Report (PDF) | ||
| 1219 | • Methodology Documentation | ||
| 1220 | • AKEL Performance Analysis | ||
| 1221 | • Contributor Agreement Analysis | ||
| 1222 | |||
| 1223 | {{/code}} | ||
| 1224 | |||
| 1225 | ---- | ||
| 1226 | |||
| 1227 | ==== Continuous Improvement Feedback Loop ==== | ||
| 1228 | |||
| 1229 | **How Metrics Inform AKEL Improvements:** | ||
| 1230 | |||
| 1231 | 1. **Identify Weak Areas:** | ||
| 1232 | |||
| 1233 | * Low TIGERScore → Review prompt engineering | ||
| 1234 | * High hallucination → Strengthen evidence grounding | ||
| 1235 | * Low IRR → Clarify evaluation criteria | ||
| 1236 | |||
| 1237 | 2. **A/B Testing Integration:** | ||
| 1238 | |||
| 1239 | * Test prompt variations | ||
| 1240 | * Measure impact on quality metrics | ||
| 1241 | * Deploy winners automatically | ||
| 1242 | |||
| 1243 | 3. **Alert Thresholds:** | ||
| 1244 | |||
| 1245 | * TIGERScore drops below 75 → Alert team | ||
| 1246 | * Hallucination rate exceeds 7% → Pause auto-publishing | ||
| 1247 | * IRR below 0.6 → Moderator training needed | ||
| 1248 | |||
| 1249 | 4. **Monthly Quality Reviews:** | ||
| 1250 | |||
| 1251 | * Analyze trends | ||
| 1252 | * Identify systematic issues | ||
| 1253 | * Plan prompt improvements | ||
| 1254 | * Update AKEL models | ||
| 1255 | |||
| 1256 | ---- | ||
| 1257 | |||
| 1258 | ==== Metric Calculation Details ==== | ||
| 1259 | |||
| 1260 | **TIGERScore Implementation:** | ||
| 1261 | |||
| 1262 | * Reference: https://github.com/TIGER-AI-Lab/TIGERScore | ||
| 1263 | * Input: Generated verdict + reference verdict (from expert) | ||
| 1264 | * Output: 0-100 score across 5 dimensions | ||
| 1265 | * Requires: Test set of expert-reviewed claims (minimum 100) | ||
| 1266 | |||
| 1267 | **AlignScore Implementation:** | ||
| 1268 | |||
| 1269 | * Reference: https://github.com/yuh-zha/AlignScore | ||
| 1270 | * Input: Generated verdict + source evidence text | ||
| 1271 | * Output: 0-1 faithfulness score | ||
| 1272 | * Calculation: Semantic alignment between claim and evidence | ||
| 1273 | |||
| 1274 | **Source Quality Scoring:** | ||
| 1275 | |||
| 1276 | * Use existing source reliability database (e.g., NewsGuard, MBFC) | ||
| 1277 | * Factor in: Publication history, corrections record, transparency | ||
| 1278 | * Scale: 0-1 (weighted average across sources) | ||
| 1279 | |||
| 1280 | ---- | ||
| 1281 | |||
| 1282 | ==== Integration Points ==== | ||
| 1283 | |||
| 1284 | * **NFR11: AKEL Quality Assurance** - Metrics validate quality gate effectiveness | ||
| 1285 | * **FR49: A/B Testing** - Metrics measure test success | ||
| 1286 | * **FR11: Audit Trail** - Source of quality data | ||
| 1287 | * **NFR3: Transparency** - Public metrics build trust | ||
| 1288 | |||
| 1289 | **Acceptance Criteria:** | ||
| 1290 | |||
| 1291 | * ✅ All core metrics implemented and calculating correctly | ||
| 1292 | * ✅ Dashboard updates daily (Beta 0) or hourly (V1.0) | ||
| 1293 | * ✅ Alerts trigger when metrics degrade beyond thresholds | ||
| 1294 | * ✅ Monthly quality report auto-generates | ||
| 1295 | * ✅ Dashboard is publicly accessible (no login required) | ||
| 1296 | * ✅ Mobile-responsive dashboard design | ||
| 1297 | * ✅ Metrics inform quarterly AKEL improvement planning | ||
| 1298 | |||
| 1299 | == 13. Requirements Traceability == | ||
| 1300 | |||
| 1301 | For full traceability matrix showing which requirements fulfill which user needs, see: | ||
| 1302 | |||
| 1303 | * [[User Needs>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.User Needs.WebHome]] - Section 8 includes comprehensive mapping tables | ||
| 1304 | |||
| 1305 | == 14. Related Pages == | ||
| 1306 | |||
| 1307 | **Non-Functional Requirements (see Section 9):** | ||
| 1308 | |||
| 1309 | * [[NFR11 — AKEL Quality Assurance Framework>>#NFR11]] | ||
| 1310 | * [[NFR12 — Security Controls>>#NFR12]] | ||
| 1311 | * [[NFR13 — Quality Metrics Transparency>>#NFR13]] | ||
| 1312 | |||
| 1313 | **Other Requirements:** | ||
| 1314 | |||
| 1315 | * [[User Needs>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.User Needs.WebHome]] | ||
| 1316 | * [[V1.0 Requirements>>FactHarbor.Specification.Requirements.V10.]] | ||
| 1317 | * [[Gap Analysis>>FactHarbor.Specification.Requirements.GapAnalysis]] | ||
| 1318 | |||
| 1319 | * **[[User Needs>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.User Needs.WebHome]]** - What users need (drives these requirements) | ||
| 1320 | * [[Architecture>>Archive.FactHarbor 2026\.01\.20.Specification.Architecture.WebHome]] - How requirements are implemented | ||
| 1321 | * [[Data Model>>Archive.FactHarbor 2026\.01\.20.Specification.Data Model.WebHome]] - Data structures supporting requirements | ||
| 1322 | * [[Workflows>>Archive.FactHarbor 2026\.01\.20.Specification.Workflows.WebHome]] - User interaction workflows | ||
| 1323 | * [[AKEL>>Archive.FactHarbor 2026\.01\.20.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements | ||
| 1324 | * [[Global Rules>>Archive.FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]] | ||
| 1325 | * [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]] | ||
| 1326 | |||
| 1327 | = V0.9.70 Additional Requirements = | ||
| 1328 | |||
| 1329 | == Functional Requirements (Additional) == | ||
| 1330 | |||
| 1331 | === FR44: ClaimReview Schema Implementation === | ||
| 1332 | |||
| 1333 | **Fulfills:** UN-13 (Cite FactHarbor Verdicts), UN-14 (API Access for Integration), UN-26 (Search Engine Visibility) | ||
| 1334 | |||
| 1335 | **Purpose:** Generate valid ClaimReview structured data for every published analysis to enable Google/Bing search visibility and fact-check discovery. | ||
| 1336 | |||
| 1337 | **Specification:** | ||
| 1338 | |||
| 1339 | ==== Component: Schema.org Markup Generator ==== | ||
| 1340 | |||
| 1341 | FactHarbor must generate valid ClaimReview structured data following Schema.org specifications for every published claim analysis. | ||
| 1342 | |||
| 1343 | **Required JSON-LD Schema:** | ||
| 1344 | |||
| 1345 | {{code language="json"}} | ||
| 1346 | { | ||
| 1347 | "@context": "https://schema.org", | ||
| 1348 | "@type": "ClaimReview", | ||
| 1349 | "datePublished": "YYYY-MM-DD", | ||
| 1350 | "url": "https://factharbor.org/claims/{claim_id}", | ||
| 1351 | "claimReviewed": "The exact claim text", | ||
| 1352 | "author": { | ||
| 1353 | "@type": "Organization", | ||
| 1354 | "name": "FactHarbor", | ||
| 1355 | "url": "https://factharbor.org" | ||
| 1356 | }, | ||
| 1357 | "reviewRating": { | ||
| 1358 | "@type": "Rating", | ||
| 1359 | "ratingValue": "1-5", | ||
| 1360 | "bestRating": "5", | ||
| 1361 | "worstRating": "1", | ||
| 1362 | "alternateName": "FactHarbor likelihood score" | ||
| 1363 | }, | ||
| 1364 | "itemReviewed": { | ||
| 1365 | "@type": "Claim", | ||
| 1366 | "author": { | ||
| 1367 | "@type": "Person", | ||
| 1368 | "name": "Claim author if known" | ||
| 1369 | }, | ||
| 1370 | "datePublished": "YYYY-MM-DD if known", | ||
| 1371 | "appearance": { | ||
| 1372 | "@type": "CreativeWork", | ||
| 1373 | "url": "Original claim URL if from article" | ||
| 1374 | } | ||
| 1375 | } | ||
| 1376 | } | ||
| 1377 | {{/code}} | ||
| 1378 | |||
| 1379 | **FactHarbor-Specific Mapping:** | ||
| 1380 | |||
| 1381 | **Likelihood Score to Rating Scale:** | ||
| 1382 | |||
| 1383 | * 80-100% likelihood → 5 (Highly Supported) | ||
| 1384 | * 60-79% likelihood → 4 (Supported) | ||
| 1385 | * 40-59% likelihood → 3 (Mixed/Uncertain) | ||
| 1386 | * 20-39% likelihood → 2 (Questionable) | ||
| 1387 | * 0-19% likelihood → 1 (Refuted) | ||
| 1388 | |||
| 1389 | **Multiple Scenarios Handling:** | ||
| 1390 | |||
| 1391 | * If claim has multiple scenarios with different verdicts, generate **separate ClaimReview** for each scenario | ||
| 1392 | * Add `disambiguatingDescription` field explaining scenario context | ||
| 1393 | * Example: "Scenario: If interpreted as referring to 2023 data..." | ||
| 1394 | |||
| 1395 | ==== Implementation Requirements ==== | ||
| 1396 | |||
| 1397 | 1. **Auto-generate** on claim publication | ||
| 1398 | 2. **Embed** in HTML `<head>` section as JSON-LD script | ||
| 1399 | 3. **Validate** against Schema.org validator before publishing | ||
| 1400 | 4. **Submit** to Google Search Console for indexing | ||
| 1401 | 5. **Update** automatically when verdict changes (integrate with FR8: Time Evolution) | ||
| 1402 | |||
| 1403 | ==== Integration Points ==== | ||
| 1404 | |||
| 1405 | * **FR7: Automated Verdicts** - Source of rating data and claim text | ||
| 1406 | * **FR8: Time Evolution** - Triggers schema updates when verdicts change | ||
| 1407 | * **FR11: Audit Trail** - Logs all schema generation and update events | ||
| 1408 | |||
| 1409 | ==== Resources ==== | ||
| 1410 | |||
| 1411 | * ClaimReview Project: https://www.claimreviewproject.com | ||
| 1412 | * Schema.org ClaimReview: https://schema.org/ClaimReview | ||
| 1413 | * Google Fact Check Guidelines: https://developers.google.com/search/docs/appearance/fact-check | ||
| 1414 | |||
| 1415 | **Acceptance Criteria:** | ||
| 1416 | |||
| 1417 | * ✅ Passes Google Structured Data Testing Tool | ||
| 1418 | * ✅ Appears in Google Fact Check Explorer within 48 hours of publication | ||
| 1419 | * ✅ Valid JSON-LD syntax (no errors) | ||
| 1420 | * ✅ All required fields populated with correct data types | ||
| 1421 | * ✅ Handles multi-scenario claims correctly (separate ClaimReview per scenario) | ||
| 1422 | |||
| 1423 | === FR45: User Corrections Notification System === | ||
| 1424 | |||
| 1425 | **Fulfills:** IFCN Principle 5 (Open & Honest Corrections), EFCSN compliance | ||
| 1426 | |||
| 1427 | **Purpose:** When any claim analysis is corrected, notify users who previously viewed the claim to maintain transparency and build trust. | ||
| 1428 | |||
| 1429 | **Specification:** | ||
| 1430 | |||
| 1431 | ==== Component: Corrections Visibility Framework ==== | ||
| 1432 | |||
| 1433 | **Correction Types:** | ||
| 1434 | |||
| 1435 | 1. **Major Correction:** Verdict changes category (e.g., "Supported" → "Refuted") | ||
| 1436 | 2. **Significant Correction:** Likelihood score changes >20% | ||
| 1437 | 3. **Minor Correction:** Evidence additions, source quality updates | ||
| 1438 | 4. **Scenario Addition:** New scenario added to existing claim | ||
| 1439 | |||
| 1440 | ==== Notification Mechanisms ==== | ||
| 1441 | |||
| 1442 | * \\ | ||
| 1443 | ** \\ | ||
| 1444 | **1. In-Page Banner: | ||
| 1445 | |||
| 1446 | Display prominent banner on claim page: | ||
| 1447 | |||
| 1448 | {{code}} | ||
| 1449 | [!] CORRECTION NOTICE | ||
| 1450 | This analysis was updated on [DATE]. [View what changed] [Dismiss] | ||
| 1451 | |||
| 1452 | Major changes: | ||
| 1453 | • Verdict changed from "Likely True (75%)" to "Uncertain (45%)" | ||
| 1454 | • New contradicting evidence added from [Source] | ||
| 1455 | • Scenario 2 updated with additional context | ||
| 1456 | |||
| 1457 | [See full correction log] | ||
| 1458 | {{/code}} | ||
| 1459 | |||
| 1460 | **2. Correction Log Page:** | ||
| 1461 | |||
| 1462 | * Public changelog at `/claims/{id}/corrections` | ||
| 1463 | * Displays for each correction: | ||
| 1464 | * Date/time of correction | ||
| 1465 | * What changed (before/after comparison) | ||
| 1466 | * Why changed (reason if provided) | ||
| 1467 | * Who made change (AKEL auto-update vs. contributor override) | ||
| 1468 | |||
| 1469 | **3. Email Notifications (opt-in):** | ||
| 1470 | |||
| 1471 | * Send to users who bookmarked or shared the claim | ||
| 1472 | * Subject: "FactHarbor Correction: [Claim title]" | ||
| 1473 | * Include summary of changes | ||
| 1474 | * Link to updated analysis | ||
| 1475 | |||
| 1476 | **4. RSS/API Feed:** | ||
| 1477 | |||
| 1478 | * Corrections feed at `/corrections.rss` | ||
| 1479 | * API endpoint: `GET /api/corrections?since={timestamp}` | ||
| 1480 | * Enables external monitoring by journalists and researchers | ||
| 1481 | |||
| 1482 | ==== Display Rules ==== | ||
| 1483 | |||
| 1484 | * Show banner on **ALL pages** displaying the claim (search results, related claims, embeddings) | ||
| 1485 | * Banner persists for **30 days** after correction | ||
| 1486 | * **"Corrections" count badge** on claim card | ||
| 1487 | * **Timestamp** on every verdict: "Last updated: [datetime]" | ||
| 1488 | |||
| 1489 | ==== IFCN Compliance Requirements ==== | ||
| 1490 | |||
| 1491 | * Corrections policy published at `/corrections-policy` | ||
| 1492 | * User can report suspected errors via `/report-error/{claim_id}` | ||
| 1493 | * Link to IFCN complaint process (if FactHarbor becomes signatory) | ||
| 1494 | * **Scrupulous transparency:** Never silently edit analyses | ||
| 1495 | |||
| 1496 | ==== Integration Points ==== | ||
| 1497 | |||
| 1498 | * **FR8: Time Evolution** - Triggers corrections when verdicts change | ||
| 1499 | * **FR11: Audit Trail** - Source of correction data and change history | ||
| 1500 | * **NFR3: Transparency** - Public correction log demonstrates commitment | ||
| 1501 | |||
| 1502 | **Acceptance Criteria:** | ||
| 1503 | |||
| 1504 | * ✅ Banner appears within 60 seconds of correction | ||
| 1505 | * ✅ Correction log is permanent and publicly accessible | ||
| 1506 | * ✅ Email notifications deliver within 5 minutes | ||
| 1507 | * ✅ RSS feed updates in real-time | ||
| 1508 | * ✅ Mobile-responsive banner design | ||
| 1509 | * ✅ Accessible (screen reader compatible) | ||
| 1510 | |||
| 1511 | === FR46: Image Verification System === | ||
| 1512 | |||
| 1513 | **Fulfills:** UN-27 (Visual Claim Verification) | ||
| 1514 | |||
| 1515 | **Purpose:** Verify authenticity and context of images shared with claims to detect manipulation, misattribution, and out-of-context usage. | ||
| 1516 | |||
| 1517 | **Specification:** | ||
| 1518 | |||
| 1519 | ==== Component: Multi-Method Image Verification ==== | ||
| 1520 | |||
| 1521 | **Method 1: Reverse Image Search** | ||
| 1522 | |||
| 1523 | **Purpose:** Find earlier uses of the image to verify context | ||
| 1524 | |||
| 1525 | **Implementation:** | ||
| 1526 | |||
| 1527 | * Integrate APIs: | ||
| 1528 | * **Google Vision AI** (reverse search) | ||
| 1529 | * **TinEye** (oldest known uses) | ||
| 1530 | * **Bing Visual Search** (broad coverage) | ||
| 1531 | |||
| 1532 | **Process:** | ||
| 1533 | |||
| 1534 | 1. Extract image from claim or user upload | ||
| 1535 | 2. Query multiple reverse search services | ||
| 1536 | 3. Analyze results for: | ||
| 1537 | |||
| 1538 | * Earliest known publication | ||
| 1539 | * Original context (what was it really showing?) | ||
| 1540 | * Publication timeline | ||
| 1541 | * Geographic spread | ||
| 1542 | |||
| 1543 | **Output:** | ||
| 1544 | {{code}}Reverse Image Search Results: | ||
| 1545 | |||
| 1546 | Earliest known use: 2019-03-15 (5 years before claim) | ||
| 1547 | Original context: "Photo from 2019 flooding in Mumbai" | ||
| 1548 | This claim uses it for: "2024 hurricane damage in Florida" | ||
| 1549 | |||
| 1550 | ⚠️ Image is OUT OF CONTEXT | ||
| 1551 | |||
| 1552 | Found in 47 other articles: | ||
| 1553 | • 2019-03-15: Mumbai floods (original) | ||
| 1554 | • 2020-07-22: Bangladesh monsoon | ||
| 1555 | • 2024-10-15: Current claim (misattributed) | ||
| 1556 | |||
| 1557 | [View full timeline]{{/code}} | ||
| 1558 | |||
| 1559 | ---- | ||
| 1560 | |||
| 1561 | **Method 2: AI Manipulation Detection** | ||
| 1562 | |||
| 1563 | **Purpose:** Detect deepfakes, face swaps, and digital alterations | ||
| 1564 | |||
| 1565 | **Implementation:** | ||
| 1566 | |||
| 1567 | * Integrate detection services: | ||
| 1568 | * **Sensity AI** (deepfake detection) | ||
| 1569 | * **Reality Defender** (multimodal analysis) | ||
| 1570 | * **AWS Rekognition** (face detection inconsistencies) | ||
| 1571 | |||
| 1572 | **Detection Categories:** | ||
| 1573 | |||
| 1574 | 1. **Face Manipulation:** | ||
| 1575 | |||
| 1576 | * Deepfake face swaps | ||
| 1577 | * Expression manipulation | ||
| 1578 | * Identity replacement | ||
| 1579 | |||
| 1580 | 2. **Image Manipulation:** | ||
| 1581 | |||
| 1582 | * Copy-paste artifacts | ||
| 1583 | * Clone stamp detection | ||
| 1584 | * Content-aware fill detection | ||
| 1585 | * JPEG compression inconsistencies | ||
| 1586 | |||
| 1587 | 3. **AI Generation:** | ||
| 1588 | |||
| 1589 | * Detect fully AI-generated images | ||
| 1590 | * Identify generation artifacts | ||
| 1591 | * Check for model signatures | ||
| 1592 | |||
| 1593 | **Confidence Scoring:** | ||
| 1594 | |||
| 1595 | * **HIGH (80-100%):** Strong evidence of manipulation | ||
| 1596 | * **MEDIUM (50-79%):** Suspicious artifacts detected | ||
| 1597 | * **LOW (0-49%):** Minor inconsistencies or inconclusive | ||
| 1598 | |||
| 1599 | **Output:** | ||
| 1600 | {{code}}Manipulation Analysis: | ||
| 1601 | |||
| 1602 | Face Manipulation: LOW RISK (12%) | ||
| 1603 | Image Editing: MEDIUM RISK (64%) | ||
| 1604 | • Clone stamp artifacts detected in sky region | ||
| 1605 | • JPEG compression inconsistent between objects | ||
| 1606 | |||
| 1607 | AI Generation: LOW RISK (8%) | ||
| 1608 | |||
| 1609 | ⚠️ Possible manipulation detected. Manual review recommended.{{/code}} | ||
| 1610 | |||
| 1611 | ---- | ||
| 1612 | |||
| 1613 | **Method 3: Metadata Analysis (EXIF)** | ||
| 1614 | |||
| 1615 | **Purpose:** Extract technical details that may reveal manipulation or misattribution | ||
| 1616 | |||
| 1617 | **Extracted Data:** | ||
| 1618 | |||
| 1619 | * **Camera/Device:** Make, model, software | ||
| 1620 | * **Timestamps:** Original date, modification dates | ||
| 1621 | * **Location:** GPS coordinates (if present) | ||
| 1622 | * **Editing History:** Software used, edit count | ||
| 1623 | * **File Properties:** Resolution, compression, format conversions | ||
| 1624 | |||
| 1625 | **Red Flags:** | ||
| 1626 | |||
| 1627 | * Metadata completely stripped (suspicious) | ||
| 1628 | * Timestamp conflicts with claimed date | ||
| 1629 | * GPS location conflicts with claimed location | ||
| 1630 | * Multiple edit rounds (hiding something?) | ||
| 1631 | * Creation date after modification date (impossible) | ||
| 1632 | |||
| 1633 | **Output:** | ||
| 1634 | {{code}}Image Metadata: | ||
| 1635 | |||
| 1636 | Camera: iPhone 14 Pro | ||
| 1637 | Original date: 2023-08-12 14:32:15 | ||
| 1638 | Location: 40.7128°N, 74.0060°W (New York City) | ||
| 1639 | Modified: 2024-10-15 08:45:22 | ||
| 1640 | Software: Adobe Photoshop 2024 | ||
| 1641 | |||
| 1642 | ⚠️ Location conflicts with claim | ||
| 1643 | Claim says: "Taken in Los Angeles" | ||
| 1644 | EXIF says: New York City | ||
| 1645 | |||
| 1646 | ⚠️ Edited 14 months after capture{{/code}} | ||
| 1647 | |||
| 1648 | ---- | ||
| 1649 | |||
| 1650 | ==== Verification Workflow ==== | ||
| 1651 | |||
| 1652 | **Automatic Triggers:** | ||
| 1653 | |||
| 1654 | 1. User submits claim with image | ||
| 1655 | 2. Article being analyzed contains images | ||
| 1656 | 3. Social media post includes photos | ||
| 1657 | |||
| 1658 | **Process:** | ||
| 1659 | |||
| 1660 | 1. Extract images from content | ||
| 1661 | 2. Run all 3 verification methods in parallel | ||
| 1662 | 3. Aggregate results into confidence score | ||
| 1663 | 4. Generate human-readable summary | ||
| 1664 | 5. Display prominently in analysis | ||
| 1665 | |||
| 1666 | **Display Integration:** | ||
| 1667 | |||
| 1668 | Show image verification panel in claim analysis: | ||
| 1669 | |||
| 1670 | {{code}} | ||
| 1671 | 📷 IMAGE VERIFICATION | ||
| 1672 | |||
| 1673 | [Image thumbnail] | ||
| 1674 | |||
| 1675 | ✅ Reverse Search: Original context verified | ||
| 1676 | ⚠️ Manipulation: Possible editing detected (64% confidence) | ||
| 1677 | ✅ Metadata: Consistent with claim details | ||
| 1678 | |||
| 1679 | Overall Assessment: CAUTION ADVISED | ||
| 1680 | This image may have been edited. Original context appears accurate. | ||
| 1681 | |||
| 1682 | [View detailed analysis] | ||
| 1683 | {{/code}} | ||
| 1684 | |||
| 1685 | ==== Integration Points ==== | ||
| 1686 | |||
| 1687 | * **FR7: Automated Verdicts** - Image verification affects claim credibility | ||
| 1688 | * **FR4: Analysis Summary** - Image findings included in summary | ||
| 1689 | * **UN-27: Visual Claim Verification** - Direct fulfillment | ||
| 1690 | |||
| 1691 | ==== Cost Considerations ==== | ||
| 1692 | |||
| 1693 | **API Costs (estimated per image):** | ||
| 1694 | |||
| 1695 | * Google Vision AI: $0.001-0.003 | ||
| 1696 | * TinEye: $0.02 (commercial API) | ||
| 1697 | * Sensity AI: $0.05-0.10 | ||
| 1698 | * AWS Rekognition: $0.001-0.002 | ||
| 1699 | |||
| 1700 | **Total per image:** $0.07-0.15** | ||
| 1701 | |||
| 1702 | **Mitigation Strategies:** | ||
| 1703 | |||
| 1704 | * Cache results for duplicate images | ||
| 1705 | * Use free tier quotas where available | ||
| 1706 | * Prioritize higher-value claims for deep analysis | ||
| 1707 | * Offer premium verification as paid tier | ||
| 1708 | |||
| 1709 | **Acceptance Criteria:** | ||
| 1710 | |||
| 1711 | * ✅ Reverse image search finds original sources | ||
| 1712 | * ✅ Manipulation detection accuracy >80% on test dataset | ||
| 1713 | * ✅ EXIF extraction works for major image formats (JPEG, PNG, HEIC) | ||
| 1714 | * ✅ Results display within 10 seconds | ||
| 1715 | * ✅ Mobile-friendly image comparison interface | ||
| 1716 | * ✅ False positive rate <15% | ||
| 1717 | |||
| 1718 | === FR47: Archive.org Integration === | ||
| 1719 | |||
| 1720 | **Importance:** CRITICAL | ||
| 1721 | **Fulfills:** Evidence persistence, FR5 (Evidence linking) | ||
| 1722 | |||
| 1723 | **Purpose:** Ensure evidence remains accessible even if original sources are deleted. | ||
| 1724 | |||
| 1725 | **Specification:** | ||
| 1726 | |||
| 1727 | **Automatic Archiving:** | ||
| 1728 | |||
| 1729 | When AKEL links evidence: | ||
| 1730 | |||
| 1731 | 1. Check if URL already archived (Wayback Machine API) | ||
| 1732 | 2. If not, submit for archiving (Save Page Now API) | ||
| 1733 | 3. Store both original URL and archive URL | ||
| 1734 | 4. Display both to users | ||
| 1735 | |||
| 1736 | **Archive Display:** | ||
| 1737 | |||
| 1738 | {{code}} | ||
| 1739 | Evidence Source: [Original URL] | ||
| 1740 | Archived: [Archive.org URL] (Captured: [date]) | ||
| 1741 | |||
| 1742 | [View Original] [View Archive] | ||
| 1743 | {{/code}} | ||
| 1744 | |||
| 1745 | **Fallback Logic:** | ||
| 1746 | |||
| 1747 | * If original URL unavailable → Auto-redirect to archive | ||
| 1748 | * If archive unavailable → Display warning | ||
| 1749 | * If both unavailable → Flag for manual review | ||
| 1750 | |||
| 1751 | **API Integration:** | ||
| 1752 | |||
| 1753 | * Use Wayback Machine Availability API | ||
| 1754 | * Use Save Page Now API (SPNv2) | ||
| 1755 | * Rate limiting: 15 requests/minute (Wayback limit) | ||
| 1756 | |||
| 1757 | **Acceptance Criteria:** | ||
| 1758 | |||
| 1759 | * ✅ All evidence URLs auto-archived | ||
| 1760 | * ✅ Archive links displayed to users | ||
| 1761 | * ✅ Fallback to archive if original unavailable | ||
| 1762 | * ✅ API rate limits respected | ||
| 1763 | * ✅ Archive status visible in evidence display | ||
| 1764 | |||
| 1765 | == Category 4: Community Safety == | ||
| 1766 | |||
| 1767 | FR48: Contributor Safety Framework === | ||
| 1768 | |||
| 1769 | **Importance:** CRITICAL | ||
| 1770 | **Fulfills:** UN-28 (Safe contribution environment) | ||
| 1771 | |||
| 1772 | **Purpose:** Protect contributors from harassment, doxxing, and coordinated attacks. | ||
| 1773 | |||
| 1774 | **Specification:** | ||
| 1775 | |||
| 1776 | * \\ | ||
| 1777 | ** \\ | ||
| 1778 | **1. Privacy Protection: | ||
| 1779 | |||
| 1780 | * **Optional Pseudonymity:** Contributors can use pseudonyms | ||
| 1781 | * **Email Privacy:** Emails never displayed publicly | ||
| 1782 | * **Profile Privacy:** Contributors control what's public | ||
| 1783 | * **IP Logging:** Only for abuse prevention, not public | ||
| 1784 | |||
| 1785 | **2. Harassment Prevention:** | ||
| 1786 | |||
| 1787 | * **Automated Toxicity Detection:** Flag abusive comments | ||
| 1788 | * **Personal Information Detection:** Auto-block doxxing attempts | ||
| 1789 | * **Coordinated Attack Detection:** Identify brigading patterns | ||
| 1790 | * **Rapid Response:** Moderator alerts for harassment | ||
| 1791 | |||
| 1792 | **3. Safety Features:** | ||
| 1793 | |||
| 1794 | * **Block Users:** Contributors can block harassers | ||
| 1795 | * **Private Contributions:** Option to contribute anonymously | ||
| 1796 | * **Report Harassment:** One-click harassment reporting | ||
| 1797 | * **Safety Resources:** Links to support resources | ||
| 1798 | |||
| 1799 | **4. Moderator Tools:** | ||
| 1800 | |||
| 1801 | * **Quick Ban:** Immediately block abusers | ||
| 1802 | * **Pattern Detection:** Identify coordinated attacks | ||
| 1803 | * **Appeal Process:** Fair review of moderation actions | ||
| 1804 | * **Escalation:** Serious threats escalated to authorities | ||
| 1805 | |||
| 1806 | **5. Trusted Contributor Protection:** | ||
| 1807 | |||
| 1808 | * **Enhanced Privacy:** Additional protection for high-profile contributors | ||
| 1809 | * **Verification:** Optional identity verification (not public) | ||
| 1810 | * **Legal Support:** Resources for contributors facing legal threats | ||
| 1811 | |||
| 1812 | **Acceptance Criteria:** | ||
| 1813 | |||
| 1814 | * ✅ Pseudonyms supported | ||
| 1815 | * ✅ Toxicity detection active | ||
| 1816 | * ✅ Doxxing auto-blocked | ||
| 1817 | * ✅ Harassment reporting functional | ||
| 1818 | * ✅ Moderator tools implemented | ||
| 1819 | * ✅ Safety policy published | ||
| 1820 | |||
| 1821 | == Category 5: Continuous Improvement == | ||
| 1822 | |||
| 1823 | FR49: A/B Testing Framework === | ||
| 1824 | |||
| 1825 | **Importance:** CRITICAL | ||
| 1826 | **Fulfills:** Continuous system improvement | ||
| 1827 | |||
| 1828 | **Purpose:** Test and measure improvements to AKEL prompts, algorithms, and workflows. | ||
| 1829 | |||
| 1830 | **Specification:** | ||
| 1831 | |||
| 1832 | **Test Capabilities:** | ||
| 1833 | |||
| 1834 | 1. **Prompt Variations:** | ||
| 1835 | |||
| 1836 | * Test different claim extraction prompts | ||
| 1837 | * Test different verdict generation prompts | ||
| 1838 | * Measure: Accuracy, clarity, completeness | ||
| 1839 | |||
| 1840 | 2. **Algorithm Variations:** | ||
| 1841 | |||
| 1842 | * Test different source scoring algorithms | ||
| 1843 | * Test different confidence calculations | ||
| 1844 | * Measure: Audit accuracy, user satisfaction | ||
| 1845 | |||
| 1846 | 3. **Workflow Variations:** | ||
| 1847 | |||
| 1848 | * Test different quality gate thresholds | ||
| 1849 | * Test different risk tier assignments | ||
| 1850 | * Measure: Publication rate, quality scores | ||
| 1851 | |||
| 1852 | **Implementation:** | ||
| 1853 | |||
| 1854 | * **Traffic Split:** 50/50 or 90/10 splits | ||
| 1855 | * **Randomization:** Consistent per claim (not per user) | ||
| 1856 | * **Metrics Collection:** Automatic for all variants | ||
| 1857 | * **Statistical Significance:** Minimum sample size calculation | ||
| 1858 | * **Rollout:** Winner promoted to 100% traffic | ||
| 1859 | |||
| 1860 | **A/B Test Workflow:** | ||
| 1861 | |||
| 1862 | {{code}} | ||
| 1863 | 1. Hypothesis: "New prompt improves claim extraction" | ||
| 1864 | 2. Design test: Control vs. Variant | ||
| 1865 | 3. Define metrics: Extraction accuracy, completeness | ||
| 1866 | 4. Run test: 7-14 days, minimum 100 claims each | ||
| 1867 | 5. Analyze results: Statistical significance? | ||
| 1868 | 6. Decision: Deploy winner or iterate | ||
| 1869 | {{/code}} | ||
| 1870 | |||
| 1871 | **Acceptance Criteria:** | ||
| 1872 | |||
| 1873 | * ✅ A/B testing framework implemented | ||
| 1874 | * ✅ Can test prompt variations | ||
| 1875 | * ✅ Can test algorithm variations | ||
| 1876 | * ✅ Metrics automatically collected | ||
| 1877 | * ✅ Statistical significance calculated | ||
| 1878 | * ✅ Results inform system improvements | ||
| 1879 | |||
| 1880 | === FR54: Evidence Deduplication === | ||
| 1881 | |||
| 1882 | **Importance:** CRITICAL (POC2/Beta) | ||
| 1883 | **Fulfills:** Accurate evidence counting, quality metrics | ||
| 1884 | |||
| 1885 | **Purpose:** Avoid counting the same source multiple times when it appears in different forms. | ||
| 1886 | |||
| 1887 | **Specification:** | ||
| 1888 | |||
| 1889 | **Deduplication Logic:** | ||
| 1890 | |||
| 1891 | 1. **URL Normalization:** | ||
| 1892 | |||
| 1893 | * Remove tracking parameters (?utm_source=...) | ||
| 1894 | * Normalize http/https | ||
| 1895 | * Normalize www/non-www | ||
| 1896 | * Handle redirects | ||
| 1897 | |||
| 1898 | 2. **Content Similarity:** | ||
| 1899 | |||
| 1900 | * If two sources have >90% text similarity → Same source | ||
| 1901 | * If one is subset of other → Same source | ||
| 1902 | * Use fuzzy matching for minor differences | ||
| 1903 | |||
| 1904 | 3. **Cross-Domain Syndication:** | ||
| 1905 | |||
| 1906 | * Detect wire service content (AP, Reuters) | ||
| 1907 | * Mark as single source if syndicated | ||
| 1908 | * Count original publication only | ||
| 1909 | |||
| 1910 | **Display:** | ||
| 1911 | |||
| 1912 | {{code}} | ||
| 1913 | Evidence Sources (3 unique, 5 total): | ||
| 1914 | |||
| 1915 | 1. Original Article (NYTimes) | ||
| 1916 | - Also appeared in: WashPost, Guardian (syndicated) | ||
| 1917 | |||
| 1918 | 2. Research Paper (Nature) | ||
| 1919 | |||
| 1920 | 3. Official Statement (WHO) | ||
| 1921 | {{/code}} | ||
| 1922 | |||
| 1923 | **Acceptance Criteria:** | ||
| 1924 | |||
| 1925 | * ✅ URL normalization works | ||
| 1926 | * ✅ Content similarity detected | ||
| 1927 | * ✅ Syndicated content identified | ||
| 1928 | * ✅ Unique vs. total counts accurate | ||
| 1929 | * ✅ Improves evidence quality metrics | ||
| 1930 | |||
| 1931 | == Additional Requirements (Lower Importance) == | ||
| 1932 | |||
| 1933 | FR50: OSINT Toolkit Integration === | ||
| 1934 | |||
| 1935 | **Fulfills:** Advanced media verification | ||
| 1936 | |||
| 1937 | **Purpose:** Integrate open-source intelligence tools for advanced verification. | ||
| 1938 | |||
| 1939 | **Tools to Integrate:** | ||
| 1940 | |||
| 1941 | * InVID/WeVerify (video verification) | ||
| 1942 | * Bellingcat toolkit | ||
| 1943 | * Additional TBD based on V1.0 learnings | ||
| 1944 | |||
| 1945 | === FR51: Video Verification System === | ||
| 1946 | |||
| 1947 | **Fulfills:** UN-27 (Visual claims), advanced media verification | ||
| 1948 | |||
| 1949 | **Purpose:** Verify video-based claims. | ||
| 1950 | |||
| 1951 | **Specification:** | ||
| 1952 | |||
| 1953 | * Keyframe extraction | ||
| 1954 | * Reverse video search | ||
| 1955 | * Deepfake detection (AI-powered) | ||
| 1956 | * Metadata analysis | ||
| 1957 | * Acoustic signature analysis | ||
| 1958 | |||
| 1959 | === FR52: Interactive Detection Training === | ||
| 1960 | |||
| 1961 | **Fulfills:** Media literacy education | ||
| 1962 | |||
| 1963 | **Purpose:** Teach users to identify misinformation. | ||
| 1964 | |||
| 1965 | **Specification:** | ||
| 1966 | |||
| 1967 | * Interactive tutorials | ||
| 1968 | * Practice exercises | ||
| 1969 | * Detection quizzes | ||
| 1970 | * Gamification elements | ||
| 1971 | |||
| 1972 | === FR53: Cross-Organizational Sharing === | ||
| 1973 | |||
| 1974 | **Fulfills:** Collaboration with other fact-checkers | ||
| 1975 | |||
| 1976 | **Purpose:** Share findings with IFCN/EFCSN members. | ||
| 1977 | |||
| 1978 | **Specification:** | ||
| 1979 | |||
| 1980 | * API for fact-checking organizations | ||
| 1981 | * Structured data exchange | ||
| 1982 | * Privacy controls | ||
| 1983 | * Attribution requirements | ||
| 1984 | |||
| 1985 | == Summary == | ||
| 1986 | |||
| 1987 | **V1.0 Critical Requirements (Must Have):** | ||
| 1988 | |||
| 1989 | * FR44: ClaimReview Schema ✅ | ||
| 1990 | * FR45: Corrections Notification ✅ | ||
| 1991 | * FR46: Image Verification ✅ | ||
| 1992 | * FR47: Archive.org Integration ✅ | ||
| 1993 | * FR48: Contributor Safety ✅ | ||
| 1994 | * FR49: A/B Testing ✅ | ||
| 1995 | * FR54: Evidence Deduplication ✅ | ||
| 1996 | * NFR11: Quality Assurance Framework ✅ | ||
| 1997 | * NFR12: Security Controls ✅ | ||
| 1998 | * NFR13: Quality Metrics Dashboard ✅ | ||
| 1999 | |||
| 2000 | **V1.1+ (Future):** | ||
| 2001 | |||
| 2002 | * FR50: OSINT Integration | ||
| 2003 | * FR51: Video Verification | ||
| 2004 | * FR52: Detection Training | ||
| 2005 | * FR53: Cross-Org Sharing | ||
| 2006 | |||
| 2007 | **Total:** 11 critical requirements for V1.0 | ||
| 2008 | |||
| 2009 | === FR54: Evidence Deduplication === | ||
| 2010 | |||
| 2011 | **Fulfills:** Accurate evidence counting, quality metrics | ||
| 2012 | |||
| 2013 | **Purpose:** Avoid counting the same source multiple times when it appears in different forms. | ||
| 2014 | |||
| 2015 | **Specification:** | ||
| 2016 | |||
| 2017 | **Deduplication Logic:** | ||
| 2018 | |||
| 2019 | 1. **URL Normalization:** | ||
| 2020 | |||
| 2021 | * Remove tracking parameters (?utm_source=...) | ||
| 2022 | * Normalize http/https | ||
| 2023 | * Normalize www/non-www | ||
| 2024 | * Handle redirects | ||
| 2025 | |||
| 2026 | 2. **Content Similarity:** | ||
| 2027 | |||
| 2028 | * If two sources have >90% text similarity → Same source | ||
| 2029 | * If one is subset of other → Same source | ||
| 2030 | * Use fuzzy matching for minor differences | ||
| 2031 | |||
| 2032 | 3. **Cross-Domain Syndication:** | ||
| 2033 | |||
| 2034 | * Detect wire service content (AP, Reuters) | ||
| 2035 | * Mark as single source if syndicated | ||
| 2036 | * Count original publication only | ||
| 2037 | |||
| 2038 | **Display:** | ||
| 2039 | |||
| 2040 | {{code}} | ||
| 2041 | Evidence Sources (3 unique, 5 total): | ||
| 2042 | |||
| 2043 | 1. Original Article (NYTimes) | ||
| 2044 | - Also appeared in: WashPost, Guardian (syndicated) | ||
| 2045 | |||
| 2046 | 2. Research Paper (Nature) | ||
| 2047 | |||
| 2048 | 3. Official Statement (WHO) | ||
| 2049 | {{/code}} | ||
| 2050 | |||
| 2051 | **Acceptance Criteria:** | ||
| 2052 | |||
| 2053 | * ✅ URL normalization works | ||
| 2054 | * ✅ Content similarity detected | ||
| 2055 | * ✅ Syndicated content identified | ||
| 2056 | * ✅ Unique vs. total counts accurate | ||
| 2057 | * ✅ Improves evidence quality metrics | ||
| 2058 | |||
| 2059 | == Additional Requirements (Lower Importance) == | ||
| 2060 | |||
| 2061 | FR7: Automated Verdicts (Enhanced with Quality Gates) === | ||
| 2062 | |||
| 2063 | **POC1+ Enhancement:** | ||
| 2064 | |||
| 2065 | After AKEL generates verdict, it passes through quality gates: | ||
| 2066 | |||
| 2067 | {{code}} | ||
| 2068 | Workflow: | ||
| 2069 | 1. Extract claims | ||
| 2070 | ↓ | ||
| 2071 | 2. [GATE 1] Validate fact-checkable | ||
| 2072 | ↓ | ||
| 2073 | 3. Generate scenarios | ||
| 2074 | ↓ | ||
| 2075 | 4. Generate verdicts | ||
| 2076 | ↓ | ||
| 2077 | 5. [GATE 4] Validate confidence | ||
| 2078 | ↓ | ||
| 2079 | 6. Display to user | ||
| 2080 | {{/code}} | ||
| 2081 | |||
| 2082 | **Updated Verdict States:** | ||
| 2083 | |||
| 2084 | * PUBLISHED | ||
| 2085 | * INSUFFICIENT_EVIDENCE | ||
| 2086 | * NON_FACTUAL_CLAIM | ||
| 2087 | * PROCESSING | ||
| 2088 | * ERROR | ||
| 2089 | |||
| 2090 | === FR4: Analysis Summary (Enhanced with Quality Metadata) === | ||
| 2091 | |||
| 2092 | **POC1+ Enhancement:** | ||
| 2093 | |||
| 2094 | Display quality indicators: | ||
| 2095 | |||
| 2096 | {{code}} | ||
| 2097 | Analysis Summary: | ||
| 2098 | Verifiable Claims: 3/5 | ||
| 2099 | High Confidence Verdicts: 1 | ||
| 2100 | Medium Confidence: 2 | ||
| 2101 | Evidence Sources: 12 | ||
| 2102 | Avg Source Quality: 0.73 | ||
| 2103 | Quality Score: 8.5/10 | ||
| 2104 | {{/code}} |