Changes for page Requirements

Last modified by Robert Schaub on 2026/02/08 21:32

From 7.1 to 6.1 From 7.5 to 7.4

From version 7.4

edited by Robert Schaub
on 2026/01/20 20:25

Change comment: Renamed back-links.

To version 7.1

edited by Robert Schaub
on 2025/12/24 21:53

Change comment: Imported from XAR

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -1,7 +1,7 @@
  = Requirements =
  {{info}}
--**Phase Assignments:** See [[Requirements Roadmap Matrix>>Archive.FactHarbor.Roadmap.Requirements-Roadmap-Matrix.WebHome]] for which requirements are implemented in which phases.
++**Phase Assignments:** See [[Requirements Roadmap Matrix>>FactHarbor.Roadmap.Requirements-Roadmap-Matrix.WebHome]] for which requirements are implemented in which phases.
  {{/info}}
  **This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.**
@@ -36,7 +36,6 @@
  **Who**: Anyone (no login required)
  **Can**:
--
  * Browse and search claims
  * View scenarios, evidence, verdicts, and confidence scores
  * Flag issues or errors
@@ -44,7 +44,6 @@
  * Submit claims automatically (new claims added if not duplicates)
  **Cannot**:
--
  * Modify content
  * Access edit history details
@@ -55,7 +55,6 @@
  **Who**: Registered users (earns reputation through contributions)
  **Can**:
--
  * Everything a Reader can do
  * Edit claims, evidence, and scenarios
  * Add sources and citations
@@ -64,7 +64,6 @@
  * Earn reputation points for quality contributions
  **Reputation System**:
--
  * New contributors: Limited edit privileges
  * Established contributors (established reputation): Full edit access
  * Trusted contributors (substantial reputation): Can approve certain changes
@@ -72,7 +72,6 @@
  * Reputation lost through: Reverted edits, invalid flags, abuse
  **Cannot**:
--
  * Delete or hide content (only moderators)
  * Override moderation decisions
@@ -83,7 +83,6 @@
  **Who**: Trusted community members with proven track record, appointed by governance board
  **Can**:
--
  * Review flagged content
  * Hide harmful or abusive content
  * Resolve disputes between contributors
@@ -92,7 +92,6 @@
  * Access full audit logs
  **Cannot**:
--
  * Change governance rules
  * Permanently ban users without board approval
  * Override technical quality gates
@@ -106,7 +106,6 @@
  **Not a permanent role**: Contacted externally when needed for contested claims in their domain
  **When used**:
--
  * Medical claims with life/safety implications
  * Legal interpretations with significant impact
  * Scientific claims with high controversy
@@ -113,7 +113,6 @@
  * Technical claims requiring specialized knowledge
  **Process**:
--
  * Moderator identifies need for expert input
  * Contact expert externally (don't require them to be users)
  * Trusted Contributor provides written opinion with sources
@@ -133,13 +133,11 @@
  **Status**: Visible to all users
  **Includes**:
--
  * AI-generated analyses (default state)
  * User-contributed content
  * Edited/improved content
  **Quality Indicators** (displayed with content):
--
  * **Confidence Score**: 0-100% (AI's confidence in analysis)
  * **Source Quality Score**: 0-100% (based on source track record)
  * **Controversy Flag**: If high dispute/edit activity
@@ -149,7 +149,6 @@
  * **Review Status**: AI-generated / Human-reviewed / Expert-validated
  **Automatic Warnings**:
--
  * Confidence < 60%: "Low confidence - use caution"
  * Source quality < 40%: "Sources may be unreliable"
  * High controversy: "Disputed - multiple interpretations exist"
@@ -162,7 +162,6 @@
  **Status**: Not visible to regular users (only to moderators)
  **Reasons**:
--
  * Spam or advertising
  * Personal attacks or harassment
  * Illegal content
@@ -171,7 +171,6 @@
  * Abuse or harmful content
  **Process**:
--
  * Automated detection flags for moderator review
  * Moderator confirms and hides
  * Original author notified with reason
@@ -194,7 +194,6 @@
  **AKEL is the primary system**. Human contributions supplement and train AKEL.
  **AKEL Must**:
--
  * Mark all outputs as AI-generated
  * Display confidence scores prominently
  * Provide source citations
@@ -203,7 +203,6 @@
  * Learn from human corrections
  **When AKEL Makes Errors**:
--
 . Capture the error pattern (what, why, how common)
 . Improve the system (better prompt, model, validation)
 . Re-process affected claims automatically
@@ -234,7 +234,6 @@
  === 4.1 Source Requirements ===
  **Track Record Over Credentials**:
--
  * Sources evaluated by historical accuracy
  * Correction policy matters
  * Independence from conflicts of interest
@@ -241,7 +241,6 @@
  * Methodology transparency
  **Source Quality Database**:
--
  * Automated tracking of source accuracy
  * Correction frequency
  * Reliability score (updated continuously)
@@ -273,7 +273,6 @@
  === 4.4 Confidence Scoring ===
  **Automated confidence calculation based on**:
--
  * Source quality scores
  * Evidence consistency
  * Contradiction detection
@@ -281,7 +281,6 @@
  * Historical accuracy of similar claims
  **Thresholds**:
--
  * < 40%: Too low to publish (needs improvement)
  * 40-60%: Published with "Low confidence" warning
  * 60-80%: Published as standard
@@ -298,7 +298,6 @@
  === 5.1 Risk Score Calculation ===
  **Factors** (weighted algorithm):
--
  * **Domain sensitivity**: Medical, legal, safety auto-flagged higher
  * **Potential impact**: Views, citations, spread
  * **Controversy level**: Flags, disputes, edit wars
@@ -325,7 +325,6 @@
  === 6.1 Error Capture ===
  **When users flag errors or make corrections**:
--
 . What was wrong? (categorize)
 . What should it have been?
 . Why did the system fail? (root cause)
@@ -344,7 +344,6 @@
  === 6.3 Quality Metrics Dashboard ===
  **Track continuously**:
--
  * Error rate by category
  * Source quality distribution
  * Confidence score trends
@@ -370,7 +370,6 @@
  === 7.2 Anomaly Detection ===
  **Automated alerts for**:
--
  * Sudden quality drops
  * Unusual patterns
  * Contradiction clusters
@@ -423,7 +423,6 @@
  **Fulfills**: UN-2 (Context-dependent verification), UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)
  **Automated scenario creation**:
--
  * AKEL analyzes claim and generates likely scenarios (use-cases and contexts)
  * Each scenario includes: assumptions, definitions, boundaries, evidence context
  * Users can flag incorrect scenarios
@@ -490,7 +490,6 @@
  **Purpose**: Provide side-by-side comparison of what a document claims vs. FactHarbor's complete analysis of its credibility
  **Left Panel: Article Summary**:
--
  * Document title, source, and claimed credibility
  * "The Big Picture" - main thesis or position change
  * "Key Findings" - structured summary of document's main claims
@@ -498,7 +498,6 @@
  * "Conclusion" - document's bottom line
  **Right Panel: FactHarbor Analysis Summary**:
--
  * FactHarbor's independent source credibility assessment
  * Claim-by-claim verdicts with confidence scores
  * Methodology assessment (strengths, limitations)
@@ -506,7 +506,6 @@
  * Analysis ID for reference
  **Design Principles**:
--
  * No scrolling required - both panels visible simultaneously
  * Visual distinction between "what they say" and "FactHarbor's analysis"
  * Color coding for verdicts (supported, uncertain, refuted)
@@ -514,7 +514,6 @@
  * Mobile responsive (panels stack vertically on small screens)
  **Implementation Notes**:
--
  * Generated automatically by AKEL for every analyzed document
  * Updates when verdict evolves (maintains version history)
  * Exportable as standalone summary report
@@ -541,8 +541,7 @@
  (% style="font-size:0.9em; color:#666;" %)
  ↑ WELL SUPPORTED • 87% confidence
  [[Click for evidence details →]]
--
--
++(%%)
  )))
  The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
@@ -555,8 +555,7 @@
  ↑ UNCERTAIN • 45% confidence
  Overstated - evidence shows risk reduction, not prevention
  [[Click for details →]]
--
--
++(%%)
  )))
  Dr. Maria Rodriguez, lead researcher, recommends incorporating more olive oil, fish, and vegetables into daily meals.
@@ -569,8 +569,7 @@
  ↑ REFUTED • 15% confidence
  Claim not supported by study design; correlation ≠ causation
  [[Click for counter-evidence →]]
--
--
++(%%)
  )))
  Participants also reported feeling more energetic and experiencing better sleep quality, though these were secondary measures.
@@ -577,7 +577,6 @@
  )))
  **Legend:**
--
  * 🟢 = Well-supported claim (confidence ≥75%)
  * 🟡 = Uncertain claim (confidence 40-74%)
  * 🔴 = Refuted/unsupported claim (confidence <40%)
@@ -596,13 +596,11 @@
  **Confidence:** 87%
  **Evidence Summary:**
--
  * Meta-analysis of 12 RCTs confirms 23-28% risk reduction
  * Consistent findings across multiple populations
  * Published in peer-reviewed journal (high credibility)
  **Uncertainty Factors:**
--
  * Exact percentage varies by study (20-30% range)
  [[View Full Analysis →]]
@@ -609,7 +609,6 @@
  )))
  **Color-Coding System**:
--
  * **Green**: Well-supported claims (confidence ≥75%, strong evidence)
  * **Yellow/Orange**: Uncertain claims (confidence 40-74%, conflicting or limited evidence)
  * **Red**: Refuted or unsupported claims (confidence <40%, contradicted by evidence)
@@ -619,12 +619,8 @@
  (% style="width:100%; border-collapse:collapse;" %)
  |=**Article Text**|=**Status**|=**Analysis**
--|(((
--A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.
--)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Context - no highlighting
--|(((
--//Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups//
--)))|(% style="background-color:#D4EDDA; text-align:center; padding:8px;" %)🟢 **WELL SUPPORTED**|(((
++|(((A recent study published in the Journal of Nutrition has revealed new findings about the Mediterranean diet.)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Context - no highlighting
++|(((//Researchers found that Mediterranean diet followers had a 25% lower risk of heart disease compared to control groups//)))|(% style="background-color:#D4EDDA; text-align:center; padding:8px;" %)🟢 **WELL SUPPORTED**|(((
  **87% confidence**
  Meta-analysis of 12 RCTs confirms 23-28% risk reduction
@@ -631,12 +631,8 @@
  [[View Full Analysis]]
  )))
--|(((
--The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.
--)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Methodology - no highlighting
--|(((
--//Some experts believe this diet can completely prevent heart attacks//
--)))|(% style="background-color:#FFF3CD; text-align:center; padding:8px;" %)🟡 **UNCERTAIN**|(((
++|(((The study, which followed 10,000 participants over five years, showed significant improvements in cardiovascular health markers.)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Methodology - no highlighting
++|(((//Some experts believe this diet can completely prevent heart attacks//)))|(% style="background-color:#FFF3CD; text-align:center; padding:8px;" %)🟡 **UNCERTAIN**|(((
  **45% confidence**
  Overstated - evidence shows risk reduction, not prevention
@@ -643,12 +643,8 @@
  [[View Details]]
  )))
--|(((
--Dr. Rodriguez recommends incorporating more olive oil, fish, and vegetables into daily meals.
--)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Recommendation - no highlighting
--|(((
--//The study proves that saturated fats cause heart disease//
--)))|(% style="background-color:#F8D7DA; text-align:center; padding:8px;" %)🔴 **REFUTED**|(((
++|(((Dr. Rodriguez recommends incorporating more olive oil, fish, and vegetables into daily meals.)))|(% style="text-align:center;" %)Plain text|(% style="font-style:italic; color:#888;" %)Recommendation - no highlighting
++|(((//The study proves that saturated fats cause heart disease//)))|(% style="background-color:#F8D7DA; text-align:center; padding:8px;" %)🔴 **REFUTED**|(((
  **15% confidence**
  Claim not supported by study; correlation ≠ causation
@@ -657,7 +657,6 @@
  )))
  **Design Notes:**
--
  * Highlighted claims use italics to distinguish from plain text
  * Color backgrounds match XWiki message box colors (success/warning/error)
  * Status column shows verdict prominently
@@ -664,7 +664,6 @@
  * Analysis column provides quick summary with link to details
  **User Actions**:
--
  * **Hover** over highlighted claim → Tooltip appears
  * **Click** highlighted claim → Detailed analysis modal/panel
  * **Toggle** button to turn highlighting on/off
@@ -671,18 +671,16 @@
  * **Keyboard**: Tab through highlighted claims
  **Interaction Design**:
--
  * Hover/click on highlighted claim → Show tooltip with:
--* Claim text
--* Verdict (e.g., "WELL SUPPORTED")
--* Confidence score (e.g., "85%")
--* Brief evidence summary
--* Link to detailed analysis
++ * Claim text
++ * Verdict (e.g., "WELL SUPPORTED")
++ * Confidence score (e.g., "85%")
++ * Brief evidence summary
++ * Link to detailed analysis
  * Toggle highlighting on/off (user preference)
  * Adjustable color intensity for accessibility
  **Technical Requirements**:
--
  * Real-time highlighting as page loads (non-blocking)
  * Claim boundary detection (start/end of assertion)
  * Handle nested or overlapping claims
@@ -690,19 +690,16 @@
  * Work with various content formats (HTML, plain text, PDFs)
  **Performance Requirements**:
--
  * Highlighting renders within 500ms of page load
  * No perceptible delay in reading experience
  * Efficient DOM manipulation (avoid reflows)
  **Accessibility**:
--
  * Color-blind friendly palette (use patterns/icons in addition to color)
  * Screen reader compatible (ARIA labels for claim credibility)
  * Keyboard navigation to highlighted claims
  **Implementation Notes**:
--
  * Claims extracted and analyzed by AKEL during initial processing
  * Highlighting data stored as annotations with byte offsets
  * Client-side rendering of highlights based on verdict data
@@ -715,7 +715,6 @@
  **Fulfills**: UN-1 (Fast access to verified content), UN-16 (Clear review status)
  **Simple flow**:
--
 . Claim submitted
 . AKEL processes (automated)
 . If confidence > threshold: Publish (labeled as AI-generated)
@@ -727,7 +727,6 @@
  ==== FR10 — Moderation ====
  **Focus on abuse, not routine quality**:
--
  * Automated abuse detection
  * Moderators handle flags
  * Quick response to harmful content
@@ -798,7 +798,6 @@
  **Purpose:** Ensure extracted claims are factual assertions (not opinions/predictions)
  **Checks:**
--
 . **Factual Statement Test:** Is this verifiable? (Yes/No)
 . **Opinion Detection:** Contains hedging language? ("I think", "probably", "best")
 . **Future Prediction Test:** Makes claims about future events?
@@ -805,7 +805,6 @@
 . **Specificity Score:** Contains specific entities, numbers, dates?
  **Thresholds:**
--
  * Factual: Must be "Yes"
  * Opinion markers: <2 hedging phrases
  * Specificity: ≥3 specific elements
@@ -817,13 +817,11 @@
  **Purpose:** Ensure AI-linked evidence actually relates to claim
  **Checks:**
--
 . **Semantic Similarity Score:** Evidence vs. claim (embeddings)
 . **Entity Overlap:** Shared people/places/things?
 . **Topic Relevance:** Discusses claim subject?
  **Thresholds:**
--
  * Similarity: ≥0.6 (cosine similarity)
  * Entity overlap: ≥1 shared entity
  * Topic relevance: ≥0.5
@@ -835,13 +835,11 @@
  **Purpose:** Validate scenario assumptions are logical and complete
  **Checks:**
--
 . **Completeness:** All required fields populated
 . **Internal Consistency:** Assumptions don't contradict
 . **Distinguishability:** Scenarios meaningfully different
  **Thresholds:**
--
  * Required fields: 100%
  * Contradiction score: <0.3
  * Scenario similarity: <0.8
@@ -853,7 +853,6 @@
  **Purpose:** Only publish high-confidence verdicts
  **Checks:**
--
 . **Evidence Count:** Minimum 2 sources
 . **Source Quality:** Average reliability ≥0.6
 . **Evidence Agreement:** Supporting vs. contradicting ≥0.6
@@ -860,7 +860,6 @@
 . **Uncertainty Factors:** Hedging in reasoning
  **Confidence Tiers:**
--
  * **HIGH (80-100%):** ≥3 sources, ≥0.7 quality, ≥80% agreement
  * **MEDIUM (50-79%):** ≥2 sources, ≥0.6 quality, ≥60% agreement
  * **LOW (0-49%):** <2 sources OR low quality/agreement
@@ -867,13 +867,11 @@
  * **INSUFFICIENT:** <2 sources → DO NOT PUBLISH
  **Implementation Phases:**
--
  * **POC1:** Gates 1 & 4 only (basic validation)
  * **POC2:** All 4 gates (complete framework)
  * **V1.0:** Hardened with <5% hallucination rate
  **Acceptance Criteria:**
--
  * ✅ All gates operational
  * ✅ Hallucination rate <5%
  * ✅ Quality metrics public
@@ -889,7 +889,6 @@
  ==== API Security ====
  **Rate Limiting:**
--
  * **Analysis endpoints:** 100 requests/hour per IP
  * **Read endpoints:** 1,000 requests/hour per IP
  * **Search:** 500 requests/hour per IP
@@ -897,24 +897,21 @@
  * **Burst protection:** Max 10 requests/second
  **Authentication & Authorization:**
--
  * **API Keys:** Required for programmatic access
  * **JWT tokens:** For user sessions (1-hour expiry)
  * **OAuth2:** For third-party integrations
  * **Role-Based Access Control (RBAC):**
--* Public: Read-only access to published claims
--* Contributor: Submit claims, provide evidence
--* Moderator: Review contributions, manage quality
--* Admin: System configuration, user management
++ * Public: Read-only access to published claims
++ * Contributor: Submit claims, provide evidence
++ * Moderator: Review contributions, manage quality
++ * Admin: System configuration, user management
  **CORS Policies:**
--
  * Whitelist approved domains only
  * No wildcard origins in production
  * Credentials required for sensitive endpoints
  **Input Sanitization:**
--
  * Validate all user input against schemas
  * Sanitize HTML/JavaScript in text submissions
  * Prevent SQL injection (use parameterized queries)
@@ -922,12 +922,11 @@
  * Max request size: 10MB
  * File upload restrictions: Whitelist file types, scan for malware
------
++---
  ==== Data Security ====
  **Encryption at Rest:**
--
  * Database encryption using AES-256
  * Encrypted backups
  * Key management via cloud provider KMS (AWS KMS, Google Cloud KMS)
@@ -934,7 +934,6 @@
  * Regular key rotation (90-day cycle)
  **Encryption in Transit:**
--
  * HTTPS/TLS 1.3 only (no TLS 1.0/1.1)
  * Strong cipher suites only
  * HSTS (HTTP Strict Transport Security) enabled
@@ -941,7 +941,6 @@
  * Certificate pinning for mobile apps
  **Secure Credential Storage:**
--
  * Passwords hashed with bcrypt (cost factor 12+)
  * API keys encrypted in database
  * Secrets stored in environment variables (never in code)
@@ -948,13 +948,12 @@
  * Use secrets manager (AWS Secrets Manager, HashiCorp Vault)
  **Data Privacy:**
--
  * Minimal data collection (privacy by design)
  * User data deletion on request (GDPR compliance)
  * PII encryption in database
  * Anonymize logs (no PII in log files)
------
++---
  ==== Application Security ====
@@ -972,7 +972,6 @@
 . **Server-Side Request Forgery:** URL validation, whitelist domains
  **Security Headers:**
--
  * `Content-Security-Policy`: Strict CSP to prevent XSS
  * `X-Frame-Options`: DENY (prevent clickjacking)
  * `X-Content-Type-Options`: nosniff
@@ -980,7 +980,6 @@
  * `Permissions-Policy`: Restrict browser features
  **Dependency Vulnerability Scanning:**
--
  * **Tools:** Snyk, Dependabot, npm audit, pip-audit
  * **Frequency:** Daily automated scans
  * **Action:** Patch critical vulnerabilities within 24 hours
@@ -987,34 +987,30 @@
  * **Policy:** No known high/critical CVEs in production
  **Security Audits:**
--
  * **Internal:** Quarterly security reviews
  * **External:** Annual penetration testing by certified firm
  * **Bug Bounty:** Public bug bounty program (V1.1+)
  * **Compliance:** SOC 2 Type II certification target (V1.5)
------
++---
  ==== Operational Security ====
  **DDoS Protection:**
--
  * CloudFlare or AWS Shield
  * Rate limiting at CDN layer
  * Automatic IP blocking for abuse patterns
  **Monitoring & Alerting:**
--
  * Real-time security event monitoring
  * Alerts for:
--* Failed login attempts (>5 in 10 minutes)
--* API abuse patterns
--* Unusual data access patterns
--* Security scan detections
++ * Failed login attempts (>5 in 10 minutes)
++ * API abuse patterns
++ * Unusual data access patterns
++ * Security scan detections
  * Integration with SIEM (Security Information and Event Management)
  **Incident Response:**
--
  * Documented incident response plan
  * Security incident classification (P1-P4)
  * On-call rotation for security issues
@@ -1022,18 +1022,16 @@
  * Public disclosure policy (coordinated disclosure)
  **Backup & Recovery:**
--
  * Daily encrypted backups
  * 30-day retention period
  * Tested recovery procedures (quarterly)
  * Disaster recovery plan (RTO: 4 hours, RPO: 1 hour)
------
++---
  ==== Compliance & Standards ====
  **GDPR Compliance:**
--
  * User consent management
  * Right to access data
  * Right to deletion
@@ -1041,7 +1041,6 @@
  * Privacy policy published
  **Accessibility:**
--
  * WCAG 2.1 AA compliance
  * Screen reader compatibility
  * Keyboard navigation
@@ -1048,7 +1048,6 @@
  * Alt text for images
  **Browser Support:**
--
  * Modern browsers only (Chrome/Edge/Firefox/Safari latest 2 versions)
  * No IE11 support
@@ -1075,18 +1075,16 @@
  **Core Metrics to Display:**
--* \\
--** \\
--**1. Verdict Quality Metrics
++**1. Verdict Quality Metrics**
  **TIGERScore (Fact-Checking Quality):**
--
  * **Definition:** Measures how well generated verdicts match expert fact-checker judgments
  * **Scale:** 0-100 (higher is better)
  * **Calculation:** Using TIGERScore framework (Truth-conditional accuracy, Informativeness, Generality, Evaluativeness, Relevance)
  * **Target:** Average ≥80 for production release
  * **Display:**
--{{code}}Verdict Quality (TIGERScore):
++{{code}}
++Verdict Quality (TIGERScore):
  Overall: 84.2 ▲ (+2.1 from last month)
  Distribution:
@@ -1094,18 +1094,19 @@
   Good (60-80): 28%
   Needs Improvement (<60): 5%
--Trend: [Graph showing improvement over time]{{/code}}
++Trend: [Graph showing improvement over time]
++{{/code}}
  **2. Hallucination & Faithfulness Metrics**
  **AlignScore (Faithfulness to Evidence):**
--
  * **Definition:** Measures how well verdicts align with actual evidence content
  * **Scale:** 0-1 (higher is better)
  * **Purpose:** Detect AI hallucinations (making claims not supported by evidence)
  * **Target:** Average ≥0.85, hallucination rate <5%
  * **Display:**
--{{code}}Evidence Faithfulness (AlignScore):
++{{code}}
++Evidence Faithfulness (AlignScore):
  Average: 0.87 ▼ (-0.02 from last month)
  Hallucination Rate: 4.2%
@@ -1112,24 +1112,24 @@
   - Claims without evidence support: 3.1%
   - Misrepresented evidence: 1.1%
--Action: Prompt engineering review scheduled{{/code}}
++Action: Prompt engineering review scheduled
++{{/code}}
  **3. Evidence Quality Metrics**
  **Source Reliability:**
--
  * Average source quality score (0-1 scale)
  * Distribution of high/medium/low quality sources
  * Publisher track record trends
  **Evidence Coverage:**
--
  * Average number of sources per claim
  * Percentage of claims with ≥2 sources (EFCSN minimum)
  * Geographic diversity of sources
  **Display:**
--{{code}}Evidence Quality:
++{{code}}
++Evidence Quality:
  Average Sources per Claim: 4.2
  Claims with ≥2 sources: 94% (EFCSN compliant)
@@ -1139,23 +1139,24 @@
   Medium quality (0.5-0.8): 43%
   Low quality (<0.5): 9%
--Geographic Diversity: 23 countries represented{{/code}}
++Geographic Diversity: 23 countries represented
++{{/code}}
  **4. Contributor Consensus Metrics** (when human reviewers involved)
  **Inter-Rater Reliability (IRR):**
--
  * **Calculation:** Cohen's Kappa or Fleiss' Kappa for multiple raters
  * **Scale:** 0-1 (higher is better)
  * **Interpretation:**
--* >0.8: Almost perfect agreement
--* 0.6-0.8: Substantial agreement
--* 0.4-0.6: Moderate agreement
--* <0.4: Poor agreement
++ * >0.8: Almost perfect agreement
++ * 0.6-0.8: Substantial agreement
++ * 0.4-0.6: Moderate agreement
++ * <0.4: Poor agreement
  * **Target:** Maintain ≥0.7 (substantial agreement)
  **Display:**
--{{code}}Contributor Consensus:
++{{code}}
++Contributor Consensus:
  Inter-Rater Reliability (IRR): 0.73 (Substantial agreement)
   - Verdict agreement: 78%
@@ -1163,9 +1163,10 @@
   - Scenario structure agreement: 69%
  Cases requiring moderator review: 12
--Moderator override rate: 8%{{/code}}
++Moderator override rate: 8%
++{{/code}}
------
++---
  ==== Quality Dashboard Implementation ====
@@ -1172,7 +1172,6 @@
  **Dashboard Location:** `/quality-metrics`
  **Update Frequency:**
--
  * **POC2:** Weekly manual updates
  * **Beta 0:** Daily automated updates
  * **V1.0:** Real-time metrics (updated hourly)
@@ -1222,7 +1222,7 @@
  {{/code}}
------
++---
  ==== Continuous Improvement Feedback Loop ====
@@ -1229,36 +1229,31 @@
  **How Metrics Inform AKEL Improvements:**
 . **Identify Weak Areas:**
++ * Low TIGERScore → Review prompt engineering
++ * High hallucination → Strengthen evidence grounding
++ * Low IRR → Clarify evaluation criteria
--* Low TIGERScore → Review prompt engineering
--* High hallucination → Strengthen evidence grounding
--* Low IRR → Clarify evaluation criteria
--
 . **A/B Testing Integration:**
++ * Test prompt variations
++ * Measure impact on quality metrics
++ * Deploy winners automatically
--* Test prompt variations
--* Measure impact on quality metrics
--* Deploy winners automatically
--
 . **Alert Thresholds:**
++ * TIGERScore drops below 75 → Alert team
++ * Hallucination rate exceeds 7% → Pause auto-publishing
++ * IRR below 0.6 → Moderator training needed
--* TIGERScore drops below 75 → Alert team
--* Hallucination rate exceeds 7% → Pause auto-publishing
--* IRR below 0.6 → Moderator training needed
--
 . **Monthly Quality Reviews:**
++ * Analyze trends
++ * Identify systematic issues
++ * Plan prompt improvements
++ * Update AKEL models
--* Analyze trends
--* Identify systematic issues
--* Plan prompt improvements
--* Update AKEL models
++---
------
--
  ==== Metric Calculation Details ====
  **TIGERScore Implementation:**
--
  * Reference: https://github.com/TIGER-AI-Lab/TIGERScore
  * Input: Generated verdict + reference verdict (from expert)
  * Output: 0-100 score across 5 dimensions
@@ -1265,7 +1265,6 @@
  * Requires: Test set of expert-reviewed claims (minimum 100)
  **AlignScore Implementation:**
--
  * Reference: https://github.com/yuh-zha/AlignScore
  * Input: Generated verdict + source evidence text
  * Output: 0-1 faithfulness score
@@ -1272,12 +1272,11 @@
  * Calculation: Semantic alignment between claim and evidence
  **Source Quality Scoring:**
--
  * Use existing source reliability database (e.g., NewsGuard, MBFC)
  * Factor in: Publication history, corrections record, transparency
  * Scale: 0-1 (weighted average across sources)
------
++---
  ==== Integration Points ====
@@ -1305,13 +1305,11 @@
  == 14. Related Pages ==
  **Non-Functional Requirements (see Section 9):**
--
  * [[NFR11 — AKEL Quality Assurance Framework>>#NFR11]]
  * [[NFR12 — Security Controls>>#NFR12]]
  * [[NFR13 — Quality Metrics Transparency>>#NFR13]]
  **Other Requirements:**
--
  * [[User Needs>>FactHarbor.Specification.Requirements.User Needs.WebHome]]
  * [[V1.0 Requirements>>FactHarbor.Specification.Requirements.V10.]]
  * [[Gap Analysis>>FactHarbor.Specification.Requirements.GapAnalysis]]
@@ -1320,8 +1320,8 @@
  * [[Architecture>>FactHarbor.Specification.Architecture.WebHome]] - How requirements are implemented
  * [[Data Model>>FactHarbor.Specification.Data Model.WebHome]] - Data structures supporting requirements
  * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] - User interaction workflows
--* [[AKEL>>Archive.FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements
--* [[Global Rules>>Archive.FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]]
++* [[AKEL>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements
++* [[Global Rules>>FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]]
  * [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
  = V0.9.70 Additional Requirements =
@@ -1379,7 +1379,6 @@
  **FactHarbor-Specific Mapping:**
  **Likelihood Score to Rating Scale:**
--
  * 80-100% likelihood → 5 (Highly Supported)
  * 60-79% likelihood → 4 (Supported)
  * 40-59% likelihood → 3 (Mixed/Uncertain)
@@ -1387,7 +1387,6 @@
  * 0-19% likelihood → 1 (Refuted)
  **Multiple Scenarios Handling:**
--
  * If claim has multiple scenarios with different verdicts, generate **separate ClaimReview** for each scenario
  * Add `disambiguatingDescription` field explaining scenario context
  * Example: "Scenario: If interpreted as referring to 2023 data..."
@@ -1439,9 +1439,7 @@
  ==== Notification Mechanisms ====
--* \\
--** \\
--**1. In-Page Banner:
++**1. In-Page Banner:**
  Display prominent banner on claim page:
@@ -1461,10 +1461,10 @@
  * Public changelog at `/claims/{id}/corrections`
  * Displays for each correction:
--* Date/time of correction
--* What changed (before/after comparison)
--* Why changed (reason if provided)
--* Who made change (AKEL auto-update vs. contributor override)
++ * Date/time of correction
++ * What changed (before/after comparison)
++ * Why changed (reason if provided)
++ * Who made change (AKEL auto-update vs. contributor override)
  **3. Email Notifications (opt-in):**
@@ -1523,25 +1523,23 @@
  **Purpose:** Find earlier uses of the image to verify context
  **Implementation:**
--
  * Integrate APIs:
--* **Google Vision AI** (reverse search)
--* **TinEye** (oldest known uses)
--* **Bing Visual Search** (broad coverage)
++ * **Google Vision AI** (reverse search)
++ * **TinEye** (oldest known uses)
++ * **Bing Visual Search** (broad coverage)
  **Process:**
--
 . Extract image from claim or user upload
 . Query multiple reverse search services
 . Analyze results for:
++ * Earliest known publication
++ * Original context (what was it really showing?)
++ * Publication timeline
++ * Geographic spread
--* Earliest known publication
--* Original context (what was it really showing?)
--* Publication timeline
--* Geographic spread
--
  **Output:**
--{{code}}Reverse Image Search Results:
++{{code}}
++Reverse Image Search Results:
  Earliest known use: 2019-03-15 (5 years before claim)
  Original context: "Photo from 2019 flooding in Mumbai"
@@ -1554,9 +1554,10 @@
  • 2020-07-22: Bangladesh monsoon
  • 2024-10-15: Current claim (misattributed)
--[View full timeline]{{/code}}
++[View full timeline]
++{{/code}}
------
++---
  **Method 2: AI Manipulation Detection**
@@ -1563,41 +1563,36 @@
  **Purpose:** Detect deepfakes, face swaps, and digital alterations
  **Implementation:**
--
  * Integrate detection services:
--* **Sensity AI** (deepfake detection)
--* **Reality Defender** (multimodal analysis)
--* **AWS Rekognition** (face detection inconsistencies)
++ * **Sensity AI** (deepfake detection)
++ * **Reality Defender** (multimodal analysis)
++ * **AWS Rekognition** (face detection inconsistencies)
  **Detection Categories:**
--
 . **Face Manipulation:**
++ * Deepfake face swaps
++ * Expression manipulation
++ * Identity replacement
--* Deepfake face swaps
--* Expression manipulation
--* Identity replacement
--
 . **Image Manipulation:**
++ * Copy-paste artifacts
++ * Clone stamp detection
++ * Content-aware fill detection
++ * JPEG compression inconsistencies
--* Copy-paste artifacts
--* Clone stamp detection
--* Content-aware fill detection
--* JPEG compression inconsistencies
--
 . **AI Generation:**
++ * Detect fully AI-generated images
++ * Identify generation artifacts
++ * Check for model signatures
--* Detect fully AI-generated images
--* Identify generation artifacts
--* Check for model signatures
--
  **Confidence Scoring:**
--
  * **HIGH (80-100%):** Strong evidence of manipulation
  * **MEDIUM (50-79%):** Suspicious artifacts detected
  * **LOW (0-49%):** Minor inconsistencies or inconclusive
  **Output:**
--{{code}}Manipulation Analysis:
++{{code}}
++Manipulation Analysis:
  Face Manipulation: LOW RISK (12%)
  Image Editing: MEDIUM RISK (64%)
@@ -1606,9 +1606,10 @@
  AI Generation: LOW RISK (8%)
--⚠️ Possible manipulation detected. Manual review recommended.{{/code}}
++⚠️ Possible manipulation detected. Manual review recommended.
++{{/code}}
------
++---
  **Method 3: Metadata Analysis (EXIF)**
@@ -1615,7 +1615,6 @@
  **Purpose:** Extract technical details that may reveal manipulation or misattribution
  **Extracted Data:**
--
  * **Camera/Device:** Make, model, software
  * **Timestamps:** Original date, modification dates
  * **Location:** GPS coordinates (if present)
@@ -1623,7 +1623,6 @@
  * **File Properties:** Resolution, compression, format conversions
  **Red Flags:**
--
  * Metadata completely stripped (suspicious)
  * Timestamp conflicts with claimed date
  * GPS location conflicts with claimed location
@@ -1631,7 +1631,8 @@
  * Creation date after modification date (impossible)
  **Output:**
--{{code}}Image Metadata:
++{{code}}
++Image Metadata:
  Camera: iPhone 14 Pro
  Original date: 2023-08-12 14:32:15
@@ -1643,20 +1643,19 @@
  Claim says: "Taken in Los Angeles"
  EXIF says: New York City
--⚠️ Edited 14 months after capture{{/code}}
++⚠️ Edited 14 months after capture
++{{/code}}
------
++---
  ==== Verification Workflow ====
  **Automatic Triggers:**
--
 . User submits claim with image
 . Article being analyzed contains images
 . Social media post includes photos
  **Process:**
--
 . Extract images from content
 . Run all 3 verification methods in parallel
 . Aggregate results into confidence score
@@ -1691,16 +1691,14 @@
  ==== Cost Considerations ====
  **API Costs (estimated per image):**
--
  * Google Vision AI: $0.001-0.003
  * TinEye: $0.02 (commercial API)
  * Sensity AI: $0.05-0.10
  * AWS Rekognition: $0.001-0.002
--**Total per image:** $0.07-0.15**
++**Total per image:** ~$0.07-0.15
  **Mitigation Strategies:**
--
  * Cache results for duplicate images
  * Use free tier quotas where available
  * Prioritize higher-value claims for deep analysis
@@ -1727,7 +1727,6 @@
  **Automatic Archiving:**
  When AKEL links evidence:
--
 . Check if URL already archived (Wayback Machine API)
 . If not, submit for archiving (Save Page Now API)
 . Store both original URL and archive URL
@@ -1762,10 +1762,8 @@
  * ✅ API rate limits respected
  * ✅ Archive status visible in evidence display
--== Category 4: Community Safety ==
++== Category 4: Community Safety ===== FR48: Contributor Safety Framework ===
-- FR48: Contributor Safety Framework ===
--
  **Importance:** CRITICAL
  **Fulfills:** UN-28 (Safe contribution environment)
@@ -1773,9 +1773,7 @@
  **Specification:**
--* \\
--** \\
--**1. Privacy Protection:
++**1. Privacy Protection:**
  * **Optional Pseudonymity:** Contributors can use pseudonyms
  * **Email Privacy:** Emails never displayed publicly
@@ -1818,10 +1818,8 @@
  * ✅ Moderator tools implemented
  * ✅ Safety policy published
--== Category 5: Continuous Improvement ==
++== Category 5: Continuous Improvement ===== FR49: A/B Testing Framework ===
-- FR49: A/B Testing Framework ===
--
  **Importance:** CRITICAL
  **Fulfills:** Continuous system improvement
@@ -1832,23 +1832,20 @@
  **Test Capabilities:**
 . **Prompt Variations:**
++ * Test different claim extraction prompts
++ * Test different verdict generation prompts
++ * Measure: Accuracy, clarity, completeness
--* Test different claim extraction prompts
--* Test different verdict generation prompts
--* Measure: Accuracy, clarity, completeness
--
 . **Algorithm Variations:**
++ * Test different source scoring algorithms
++ * Test different confidence calculations
++ * Measure: Audit accuracy, user satisfaction
--* Test different source scoring algorithms
--* Test different confidence calculations
--* Measure: Audit accuracy, user satisfaction
--
 . **Workflow Variations:**
++ * Test different quality gate thresholds
++ * Test different risk tier assignments
++ * Measure: Publication rate, quality scores
--* Test different quality gate thresholds
--* Test different risk tier assignments
--* Measure: Publication rate, quality scores
--
  **Implementation:**
  * **Traffic Split:** 50/50 or 90/10 splits
@@ -1889,24 +1889,21 @@
  **Deduplication Logic:**
 . **URL Normalization:**
++ * Remove tracking parameters (?utm_source=...)
++ * Normalize http/https
++ * Normalize www/non-www
++ * Handle redirects
--* Remove tracking parameters (?utm_source=...)
--* Normalize http/https
--* Normalize www/non-www
--* Handle redirects
--
 . **Content Similarity:**
++ * If two sources have >90% text similarity → Same source
++ * If one is subset of other → Same source
++ * Use fuzzy matching for minor differences
--* If two sources have >90% text similarity → Same source
--* If one is subset of other → Same source
--* Use fuzzy matching for minor differences
--
 . **Cross-Domain Syndication:**
++ * Detect wire service content (AP, Reuters)
++ * Mark as single source if syndicated
++ * Count original publication only
--* Detect wire service content (AP, Reuters)
--* Mark as single source if syndicated
--* Count original publication only
--
  **Display:**
  {{code}}
@@ -1928,16 +1928,13 @@
  * ✅ Unique vs. total counts accurate
  * ✅ Improves evidence quality metrics
--== Additional Requirements (Lower Importance) ==
++== Additional Requirements (Lower Importance) ===== FR50: OSINT Toolkit Integration ===
-- FR50: OSINT Toolkit Integration ===
--
  **Fulfills:** Advanced media verification
  **Purpose:** Integrate open-source intelligence tools for advanced verification.
  **Tools to Integrate:**
--
  * InVID/WeVerify (video verification)
  * Bellingcat toolkit
  * Additional TBD based on V1.0 learnings
@@ -1949,7 +1949,6 @@
  **Purpose:** Verify video-based claims.
  **Specification:**
--
  * Keyframe extraction
  * Reverse video search
  * Deepfake detection (AI-powered)
@@ -1963,7 +1963,6 @@
  **Purpose:** Teach users to identify misinformation.
  **Specification:**
--
  * Interactive tutorials
  * Practice exercises
  * Detection quizzes
@@ -1976,7 +1976,6 @@
  **Purpose:** Share findings with IFCN/EFCSN members.
  **Specification:**
--
  * API for fact-checking organizations
  * Structured data exchange
  * Privacy controls
@@ -2017,24 +2017,21 @@
  **Deduplication Logic:**
 . **URL Normalization:**
++ * Remove tracking parameters (?utm_source=...)
++ * Normalize http/https
++ * Normalize www/non-www
++ * Handle redirects
--* Remove tracking parameters (?utm_source=...)
--* Normalize http/https
--* Normalize www/non-www
--* Handle redirects
--
 . **Content Similarity:**
++ * If two sources have >90% text similarity → Same source
++ * If one is subset of other → Same source
++ * Use fuzzy matching for minor differences
--* If two sources have >90% text similarity → Same source
--* If one is subset of other → Same source
--* Use fuzzy matching for minor differences
--
 . **Cross-Domain Syndication:**
++ * Detect wire service content (AP, Reuters)
++ * Mark as single source if syndicated
++ * Count original publication only
--* Detect wire service content (AP, Reuters)
--* Mark as single source if syndicated
--* Count original publication only
--
  **Display:**
  {{code}}
@@ -2056,10 +2056,8 @@
  * ✅ Unique vs. total counts accurate
  * ✅ Improves evidence quality metrics
--== Additional Requirements (Lower Importance) ==
++== Additional Requirements (Lower Importance) ===== FR7: Automated Verdicts (Enhanced with Quality Gates) ===
-- FR7: Automated Verdicts (Enhanced with Quality Gates) ===
--
  **POC1+ Enhancement:**
  After AKEL generates verdict, it passes through quality gates:
@@ -2080,7 +2080,6 @@
  {{/code}}
  **Updated Verdict States:**
--
  * PUBLISHED
  * INSUFFICIENT_EVIDENCE
  * NON_FACTUAL_CLAIM
@@ -2102,3 +2102,4 @@
   Avg Source Quality: 0.73
   Quality Score: 8.5/10
  {{/code}}
++

Changes for page Requirements

Summary

Details

Applications

Navigation

Need help?