Requirements

This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.

Core Philosophy: Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically.

Navigation

• User Needs - What users need from FactHarbor (drives these requirements)
• This page - How we fulfill those needs through system design

1. Roles

Fulfills: UN-12 (Submit claims), UN-13 (Cite verdicts), UN-14 (API access)

FactHarbor uses three simple roles plus a reputation system.

1.1 Reader

Who: Anyone (no login required)

Can:

• Browse and search claims
• View scenarios, evidence, verdicts, and confidence scores
• Flag issues or errors
• Use filters, search, and visualization tools
• Submit claims automatically (new claims added if not duplicates)

Cannot:

• Modify content
• Access edit history details

User Needs served: UN-1 (Trust assessment), UN-2 (Claim verification), UN-3 (Article summary with FactHarbor analysis summary), UN-4 (Social media fact-checking), UN-5 (Source tracing), UN-7 (Evidence transparency), UN-8 (Understanding disagreement), UN-12 (Submit claims), UN-17 (In-article highlighting)

1.2 Contributor

Who: Registered users (earns reputation through contributions)

Can:

• Everything a Reader can do
• Edit claims, evidence, and scenarios
• Add sources and citations
• Suggest improvements to AI-generated content
• Participate in discussions
• Earn reputation points for quality contributions

Reputation System:

• New contributors: Limited edit privileges
• Established contributors (established reputation): Full edit access
• Trusted contributors (substantial reputation): Can approve certain changes
• Reputation earned through: Accepted edits, helpful flags, quality contributions
• Reputation lost through: Reverted edits, invalid flags, abuse

Cannot:

• Delete or hide content (only moderators)
• Override moderation decisions

User Needs served: UN-13 (Cite and contribute)

1.3 Moderator

Who: Trusted community members with proven track record, appointed by governance board

Can:

• Review flagged content
• Hide harmful or abusive content
• Resolve disputes between contributors
• Issue warnings or temporary bans
• Make final decisions on content disputes
• Access full audit logs

Cannot:

• Change governance rules
• Permanently ban users without board approval
• Override technical quality gates

Note: Small team (3-5 initially), supported by automated moderation tools.

1.4 Domain Trusted Contributors (Optional, Task-Specific)

Who: Subject matter specialists invited for specific high-stakes disputes

Not a permanent role: Contacted externally when needed for contested claims in their domain

When used:

• Medical claims with life/safety implications
• Legal interpretations with significant impact
• Scientific claims with high controversy
• Technical claims requiring specialized knowledge

Process:

• Moderator identifies need for expert input
• Contact expert externally (don't require them to be users)
• Trusted Contributor provides written opinion with sources
• Opinion added to claim record
• Trusted Contributor acknowledged in claim

User Needs served: UN-16 (Expert validation status)

2. Content States

Fulfills: UN-1 (Trust indicators), UN-16 (Review status transparency)

FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping.

2.1 Published

Status: Visible to all users

Includes:

• AI-generated analyses (default state)
• User-contributed content
• Edited/improved content

Quality Indicators (displayed with content):

• Confidence Score: 0-100% (AI's confidence in analysis)
• Source Quality Score: 0-100% (based on source track record)
• Controversy Flag: If high dispute/edit activity
• Completeness Score: % of expected fields filled
• Last Updated: Date of most recent change
• Edit Count: Number of revisions
• Review Status: AI-generated / Human-reviewed / Expert-validated

Automatic Warnings:

• Confidence < 60%: "Low confidence - use caution"
• Source quality < 40%: "Sources may be unreliable"
• High controversy: "Disputed - multiple interpretations exist"
• Medical/Legal/Safety domain: "Seek professional advice"

User Needs served: UN-1 (Trust score), UN-9 (Methodology transparency), ~UN-15 (Evolution timeline - Deferred)~, UN-16 (Review status)

2.2 Hidden

Status: Not visible to regular users (only to moderators)

Reasons:

• Spam or advertising
• Personal attacks or harassment
• Illegal content
• Privacy violations
• Deliberate misinformation (verified)
• Abuse or harmful content

Process:

• Automated detection flags for moderator review
• Moderator confirms and hides
• Original author notified with reason
• Can appeal to board if disputes moderator decision

Note: Content is hidden, not deleted (for audit trail)

3. Contribution Rules

3.1 All Contributors Must

• Provide sources for factual claims
• Use clear, neutral language in FactHarbor's own summaries
• Respect others and maintain civil discourse
• Accept community feedback constructively
• Focus on improving quality, not protecting ego

3.2 AKEL (AI System)

AKEL is the primary system. Human contributions supplement and train AKEL.

AKEL Must:

• Mark all outputs as AI-generated
• Display confidence scores prominently
• Provide source citations
• Flag uncertainty clearly
• Identify contradictions in evidence
• Learn from human corrections

When AKEL Makes Errors:

1. Capture the error pattern (what, why, how common)
   2. Improve the system (better prompt, model, validation)
   3. Re-process affected claims automatically
   4. Measure improvement (did quality increase?)

Human Role: Train AKEL through corrections, not replace AKEL

3.3 Contributors Should

• Improve clarity and structure
• Add missing sources
• Flag errors for system improvement
• Suggest better ways to present information
• Participate in quality discussions

3.4 Moderators Must

• Be impartial
• Document moderation decisions
• Respond to appeals promptly
• Use automated tools to scale efforts
• Focus on abuse/harm, not routine quality control

4. Quality Standards

Fulfills: UN-5 (Source reliability), UN-6 (Publisher track records), UN-7 (Evidence transparency), UN-9 (Methodology transparency)

4.1 Source Requirements

Track Record Over Credentials:

• Sources evaluated by historical accuracy
• Correction policy matters
• Independence from conflicts of interest
• Methodology transparency

Source Quality Database:

• Automated tracking of source accuracy
• Correction frequency
• Reliability score (updated continuously)
• Users can see source track record

No automatic trust for government, academia, or media - all evaluated by track record.

User Needs served: UN-5 (Source provenance), UN-6 (Publisher reliability)

4.2 Claim Requirements

• Clear subject and assertion
• Verifiable with available information
• Sourced (or explicitly marked as needing sources)
• Neutral language in FactHarbor summaries
• Appropriate context provided

User Needs served: UN-2 (Claim extraction and verification)

4.3 Evidence Requirements

• Publicly accessible (or explain why not)
• Properly cited with attribution
• Relevant to claim being evaluated
• Original source preferred over secondary

User Needs served: UN-7 (Evidence transparency)

4.4 Confidence Scoring

Automated confidence calculation based on:

• Source quality scores
• Evidence consistency
• Contradiction detection
• Completeness of analysis
• Historical accuracy of similar claims

Thresholds:

• < 40%: Too low to publish (needs improvement)
• 40-60%: Published with "Low confidence" warning
• 60-80%: Published as standard
• 80-100%: Published as "High confidence"

User Needs served: UN-1 (Trust assessment), UN-9 (Methodology transparency)

5. Automated Risk Scoring

Fulfills: UN-10 (Manipulation detection), UN-16 (Appropriate review level)

Replace manual risk tiers with continuous automated scoring.

5.1 Risk Score Calculation

Factors (weighted algorithm):

• Domain sensitivity: Medical, legal, safety auto-flagged higher
• Potential impact: Views, citations, spread
• Controversy level: Flags, disputes, edit wars
• Uncertainty: Low confidence, contradictory evidence
• Source reliability: Track record of sources used

Score: 0-100 (higher = more risk)

5.2 Automated Actions

• Score > 80: Flag for moderator review before publication
• Score 60-80: Publish with prominent warnings
• Score 40-60: Publish with standard warnings
• Score < 40: Publish normally

Continuous monitoring: Risk score recalculated as new information emerges

User Needs served: UN-10 (Detect manipulation tactics), UN-16 (Review status)

6. System Improvement Process

Core principle: Fix the system, not just the data.

6.1 Error Capture

When users flag errors or make corrections:

1. What was wrong? (categorize)
   2. What should it have been?
   3. Why did the system fail? (root cause)
   4. How common is this pattern?
   5. Store in ErrorPattern table (improvement queue)

6.2 Continuous Improvement Cycle

1. Review: Analyze top error patterns
   2. Develop: Create fix (prompt, model, validation)
   3. Test: Validate fix on sample claims
   4. Deploy: Roll out if quality improves
   5. Re-process: Automatically update affected claims
   6. Monitor: Track quality metrics

6.3 Quality Metrics Dashboard

Track continuously:

• Error rate by category
• Source quality distribution
• Confidence score trends
• User flag rate (issues found)
• Correction acceptance rate
• Re-work rate
• Claims processed per hour

Goal: continuous improvement in error rate

7. Automated Quality Monitoring

Replace manual audit sampling with automated monitoring.

7.1 Continuous Metrics

• Source quality: Track record database
• Consistency: Contradiction detection
• Clarity: Readability scores
• Completeness: Field validation
• Accuracy: User corrections tracked

7.2 Anomaly Detection

Automated alerts for:

• Sudden quality drops
• Unusual patterns
• Contradiction clusters
• Source reliability changes
• User behavior anomalies

7.3 Targeted Review

• Review only flagged items
• Random sampling for calibration (not quotas)
• Learn from corrections to improve automation

8. Functional Requirements

This section defines specific features that fulfill user needs.

8.1 Claim Intake & Normalization

FR1 — Claim Intake

Fulfills: UN-2 (Claim extraction), UN-4 (Quick fact-checking), UN-12 (Submit claims)

• Users submit claims via simple form or API
• Claims can be text, URL, or image
• Duplicate detection (semantic similarity)
• Auto-categorization by domain

FR2 — Claim Normalization

Fulfills: UN-2 (Claim verification)

• Standardize to clear assertion format
• Extract key entities (who, what, when, where)
• Identify claim type (factual, predictive, evaluative)
• Link to existing similar claims

FR3 — Claim Classification

Fulfills: UN-11 (Filtered research)

• Domain: Politics, Science, Health, etc.
• Type: Historical fact, current stat, prediction, etc.
• Risk score: Automated calculation
• Complexity: Simple, moderate, complex

8.2 Scenario System

FR4 — Scenario Generation

Fulfills: UN-2 (Context-dependent verification), UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)

Automated scenario creation:

• AKEL analyzes claim and generates likely scenarios (use-cases and contexts)
• Each scenario includes: assumptions, definitions, boundaries, evidence context
• Users can flag incorrect scenarios
• System learns from corrections

Key Concept: Scenarios represent different interpretations or contexts (e.g., "Clinical trials with healthy adults" vs. "Real-world data with diverse populations")

FR5 — Evidence Linking

Fulfills: UN-5 (Source tracing), UN-7 (Evidence transparency)

• Automated evidence discovery from sources
• Relevance scoring
• Contradiction detection
• Source quality assessment

FR6 — Scenario Comparison

Fulfills: UN-3 (Article summary with FactHarbor analysis summary), UN-8 (Understanding disagreement)

• Side-by-side comparison interface
• Highlight key differences between scenarios
• Show evidence supporting each scenario
• Display confidence scores per scenario

8.3 Verdicts & Analysis

FR7 — Automated Verdicts

Fulfills: UN-1 (Trust score), UN-2 (Verification verdicts), UN-3 (Article summary with FactHarbor analysis summary), UN-13 (Cite verdicts)

• AKEL generates verdict based on evidence within each scenario
• Likelihood range displayed (e.g., "0.70-0.85 (likely true)") - NOT binary true/false
• Uncertainty factors explicitly listed (e.g., "Small sample sizes", "Long-term effects unknown")
• Confidence score displayed prominently
• Source quality indicators shown
• Contradictions noted
• Uncertainty acknowledged

Key Innovation: Detailed probabilistic verdicts with explicit uncertainty, not binary judgments

FR8 — Time Evolution

Fulfills: UN-15 (Verdict evolution timeline)

• Claims and verdicts update as new evidence emerges
• Version history maintained for all verdicts
• Changes highlighted
• Confidence score trends visible
• Users can see "as of date X, what did we know?"

8.4 User Interface & Presentation

FR12 — Two-Panel Summary View (Article Summary with FactHarbor Analysis Summary)

Fulfills: UN-3 (Article Summary with FactHarbor Analysis Summary)

Purpose: Provide side-by-side comparison of what a document claims vs. FactHarbor's complete analysis of its credibility

Left Panel: Article Summary:

• Document title, source, and claimed credibility
• "The Big Picture" - main thesis or position change
• "Key Findings" - structured summary of document's main claims
• "Reasoning" - document's explanation for positions
• "Conclusion" - document's bottom line

Right Panel: FactHarbor Analysis Summary:

• FactHarbor's independent source credibility assessment
• Claim-by-claim verdicts with confidence scores
• Methodology assessment (strengths, limitations)
• Overall verdict on document quality
• Analysis ID for reference

Design Principles:

• No scrolling required - both panels visible simultaneously
• Visual distinction between "what they say" and "FactHarbor's analysis"
• Color coding for verdicts (supported, uncertain, refuted)
• Confidence percentages clearly visible
• Mobile responsive (panels stack vertically on small screens)

Implementation Notes:

• Generated automatically by AKEL for every analyzed document
• Updates when verdict evolves (maintains version history)
• Exportable as standalone summary report
• Shareable via permanent URL

FR13 — In-Article Claim Highlighting

Fulfills: UN-17 (In-article claim highlighting)

Purpose: Enable readers to quickly assess claim credibility while reading by visually highlighting factual claims with color-coded indicators

Visual Example: Article with Highlighted Claims

Legend:

• 🟢 = Well-supported claim (confidence ≥75%)
• 🟡 = Uncertain claim (confidence 40-74%)
• 🔴 = Refuted/unsupported claim (confidence <40%)
• Plain text = Non-factual content (context, opinions, recommendations)

Tooltip on Hover/Click

Color-Coding System:

• Green: Well-supported claims (confidence ≥75%, strong evidence)
• Yellow/Orange: Uncertain claims (confidence 40-74%, conflicting or limited evidence)
• Red: Refuted or unsupported claims (confidence <40%, contradicted by evidence)
• Gray/Neutral: Non-factual content (opinions, questions, procedural text)

Interactive Highlighting Example (Detailed View)

Design Notes:

• Highlighted claims use italics to distinguish from plain text
• Color backgrounds match XWiki message box colors (success/warning/error)
• Status column shows verdict prominently
• Analysis column provides quick summary with link to details

User Actions:

• Hover over highlighted claim → Tooltip appears
• Click highlighted claim → Detailed analysis modal/panel
• Toggle button to turn highlighting on/off
• Keyboard: Tab through highlighted claims

Interaction Design:

• Hover/click on highlighted claim → Show tooltip with:
• Claim text
• Verdict (e.g., "WELL SUPPORTED")
• Confidence score (e.g., "85%")
• Brief evidence summary
• Link to detailed analysis
• Toggle highlighting on/off (user preference)
• Adjustable color intensity for accessibility

Technical Requirements:

• Real-time highlighting as page loads (non-blocking)
• Claim boundary detection (start/end of assertion)
• Handle nested or overlapping claims
• Preserve original article formatting
• Work with various content formats (HTML, plain text, PDFs)

Performance Requirements:

• Highlighting renders within 500ms of page load
• No perceptible delay in reading experience
• Efficient DOM manipulation (avoid reflows)

Accessibility:

• Color-blind friendly palette (use patterns/icons in addition to color)
• Screen reader compatible (ARIA labels for claim credibility)
• Keyboard navigation to highlighted claims

Implementation Notes:

• Claims extracted and analyzed by AKEL during initial processing
• Highlighting data stored as annotations with byte offsets
• Client-side rendering of highlights based on verdict data
• Mobile responsive (tap instead of hover)

8.5 Workflow & Moderation

FR9 — Publication Workflow

Fulfills: UN-1 (Fast access to verified content), UN-16 (Clear review status)

Simple flow:

1. Claim submitted
   2. AKEL processes (automated)
   3. If confidence > threshold: Publish (labeled as AI-generated)
   4. If confidence < threshold: Flag for improvement
   5. If risk score > threshold: Flag for moderator

No multi-stage approval process

FR10 — Moderation

Focus on abuse, not routine quality:

• Automated abuse detection
• Moderators handle flags
• Quick response to harmful content
• Minimal involvement in routine content

FR11 — Audit Trail

Fulfills: UN-14 (API access to histories), UN-15 (Evolution tracking)

• All edits logged
• Version history public
• Moderation decisions documented
• System improvements tracked

9. Non-Functional Requirements

9.1 NFR1 — Performance

Fulfills: UN-4 (Fast fact-checking), UN-11 (Responsive filtering)

• Claim processing: < 30 seconds
• Search response: < 2 seconds
• Page load: < 3 seconds
• 99% uptime

9.2 NFR2 — Scalability

Fulfills: UN-14 (API access at scale)

• Handle 10,000 claims initially
• Scale to 1M+ claims
• Support 100K+ concurrent users
• Automated processing scales linearly

9.3 NFR3 — Transparency

Fulfills: UN-7 (Evidence transparency), UN-9 (Methodology transparency), UN-13 (Citable verdicts), UN-15 (Evolution visibility)

• All algorithms open source
• All data exportable
• All decisions documented
• Quality metrics public

9.4 NFR4 — Security & Privacy

• Follow Privacy Policy
• Secure authentication
• Data encryption
• Regular security audits

9.5 NFR5 — Maintainability

• Modular architecture
• Automated testing
• Continuous integration
• Comprehensive documentation

NFR11: AKEL Quality Assurance Framework

Fulfills: AI safety, IFCN methodology transparency

Specification:

Multi-layer AI quality gates to detect hallucinations, low-confidence results, and logical inconsistencies.

Quality Gate 1: Claim Extraction Validation

Purpose: Ensure extracted claims are factual assertions (not opinions/predictions)

Checks:

1. Factual Statement Test: Is this verifiable? (Yes/No)
   2. Opinion Detection: Contains hedging language? ("I think", "probably", "best")
   3. Future Prediction Test: Makes claims about future events?
   4. Specificity Score: Contains specific entities, numbers, dates?

Thresholds:

• Factual: Must be "Yes"
• Opinion markers: <2 hedging phrases
• Specificity: ≥3 specific elements

Action if Failed: Flag as "Non-verifiable", do NOT generate verdict

Quality Gate 2: Evidence Relevance Validation

Purpose: Ensure AI-linked evidence actually relates to claim

Checks:

1. Semantic Similarity Score: Evidence vs. claim (embeddings)
   2. Entity Overlap: Shared people/places/things?
   3. Topic Relevance: Discusses claim subject?

Thresholds:

• Similarity: ≥0.6 (cosine similarity)
• Entity overlap: ≥1 shared entity
• Topic relevance: ≥0.5

Action if Failed: Discard irrelevant evidence

Quality Gate 3: Scenario Coherence Check

Purpose: Validate scenario assumptions are logical and complete

Checks:

1. Completeness: All required fields populated
   2. Internal Consistency: Assumptions don't contradict
   3. Distinguishability: Scenarios meaningfully different

Thresholds:

• Required fields: 100%
• Contradiction score: <0.3
• Scenario similarity: <0.8

Action if Failed: Merge duplicates, reduce confidence -20%

Quality Gate 4: Verdict Confidence Assessment

Purpose: Only publish high-confidence verdicts

Checks:

1. Evidence Count: Minimum 2 sources
   2. Source Quality: Average reliability ≥0.6
   3. Evidence Agreement: Supporting vs. contradicting ≥0.6
   4. Uncertainty Factors: Hedging in reasoning

Confidence Tiers:

• HIGH (80-100%): ≥3 sources, ≥0.7 quality, ≥80% agreement
• MEDIUM (50-79%): ≥2 sources, ≥0.6 quality, ≥60% agreement
• LOW (0-49%): <2 sources OR low quality/agreement
• INSUFFICIENT: <2 sources → DO NOT PUBLISH

Implementation Phases:

• POC1: Gates 1 & 4 only (basic validation)
• POC2: All 4 gates (complete framework)
• V1.0: Hardened with <5% hallucination rate

Acceptance Criteria:

• ✅ All gates operational
• ✅ Hallucination rate <5%
• ✅ Quality metrics public

NFR12: Security Controls

Fulfills: Data protection, system integrity, user privacy, production readiness

Purpose: Protect FactHarbor systems, user data, and operations from security threats, ensuring production-grade security posture.

Specification:

API Security

Rate Limiting:

• Analysis endpoints: 100 requests/hour per IP
• Read endpoints: 1,000 requests/hour per IP
• Search: 500 requests/hour per IP
• Authenticated users: 5x higher limits
• Burst protection: Max 10 requests/second

Authentication & Authorization:

• API Keys: Required for programmatic access
• JWT tokens: For user sessions (1-hour expiry)
• OAuth2: For third-party integrations
• Role-Based Access Control (RBAC):
• Public: Read-only access to published claims
• Contributor: Submit claims, provide evidence
• Moderator: Review contributions, manage quality
• Admin: System configuration, user management

CORS Policies:

• Whitelist approved domains only
• No wildcard origins in production
• Credentials required for sensitive endpoints

Input Sanitization:

• Validate all user input against schemas
• Sanitize HTML/JavaScript in text submissions
• Prevent SQL injection (use parameterized queries)
• Prevent command injection (no shell execution of user input)
• Max request size: 10MB
• File upload restrictions: Whitelist file types, scan for malware

-

Data Security

Encryption at Rest:

• Database encryption using AES-256
• Encrypted backups
• Key management via cloud provider KMS (AWS KMS, Google Cloud KMS)
• Regular key rotation (90-day cycle)

Encryption in Transit:

• HTTPS/TLS 1.3 only (no TLS 1.0/1.1)
• Strong cipher suites only
• HSTS (HTTP Strict Transport Security) enabled
• Certificate pinning for mobile apps

Secure Credential Storage:

• Passwords hashed with bcrypt (cost factor 12+)
• API keys encrypted in database
• Secrets stored in environment variables (never in code)
• Use secrets manager (AWS Secrets Manager, HashiCorp Vault)

Data Privacy:

• Minimal data collection (privacy by design)
• User data deletion on request (GDPR compliance)
• PII encryption in database
• Anonymize logs (no PII in log files)

-

Application Security

OWASP Top 10 Compliance:

1. Broken Access Control: RBAC implementation, path traversal prevention
   2. Cryptographic Failures: Strong encryption, secure key management
   3. Injection: Parameterized queries, input validation
   4. Insecure Design: Security review of all features
   5. Security Misconfiguration: Hardened defaults, security headers
   6. Vulnerable Components: Dependency scanning (see below)
   7. Authentication Failures: Strong password policy, MFA support
   8. Data Integrity Failures: Signature verification, checksums
   9. Security Logging Failures: Comprehensive audit logs
   10. Server-Side Request Forgery: URL validation, whitelist domains

Security Headers:

• `Content-Security-Policy`: Strict CSP to prevent XSS
• `X-Frame-Options`: DENY (prevent clickjacking)
• `X-Content-Type-Options`: nosniff
• `Referrer-Policy`: strict-origin-when-cross-origin
• `Permissions-Policy`: Restrict browser features

Dependency Vulnerability Scanning:

• Tools: Snyk, Dependabot, npm audit, pip-audit
• Frequency: Daily automated scans
• Action: Patch critical vulnerabilities within 24 hours
• Policy: No known high/critical CVEs in production

Security Audits:

• Internal: Quarterly security reviews
• External: Annual penetration testing by certified firm
• Bug Bounty: Public bug bounty program (V1.1+)
• Compliance: SOC 2 Type II certification target (V1.5)

-

Operational Security

DDoS Protection:

• CloudFlare or AWS Shield
• Rate limiting at CDN layer
• Automatic IP blocking for abuse patterns

Monitoring & Alerting:

• Real-time security event monitoring
• Alerts for:
• Failed login attempts (>5 in 10 minutes)
• API abuse patterns
• Unusual data access patterns
• Security scan detections
• Integration with SIEM (Security Information and Event Management)

Incident Response:

• Documented incident response plan
• Security incident classification (P1-P4)
• On-call rotation for security issues
• Post-mortem for all security incidents
• Public disclosure policy (coordinated disclosure)

Backup & Recovery:

• Daily encrypted backups
• 30-day retention period
• Tested recovery procedures (quarterly)
• Disaster recovery plan (RTO: 4 hours, RPO: 1 hour)

-

Compliance & Standards

GDPR Compliance:

• User consent management
• Right to access data
• Right to deletion
• Data portability
• Privacy policy published

Accessibility:

• WCAG 2.1 AA compliance
• Screen reader compatibility
• Keyboard navigation
• Alt text for images

Browser Support:

• Modern browsers only (Chrome/Edge/Firefox/Safari latest 2 versions)
• No IE11 support

Acceptance Criteria:

• ✅ Passes OWASP ZAP security scan (no high/critical findings)
• ✅ All dependencies with known vulnerabilities patched
• ✅ Penetration test completed with no critical findings
• ✅ Rate limiting blocks abuse attempts
• ✅ Encryption at rest and in transit verified
• ✅ Security headers scored A+ on securityheaders.com
• ✅ Incident response plan documented and tested
• ✅ 95% uptime over 30-day period

NFR13: Quality Metrics Transparency

Fulfills: User trust, transparency, continuous improvement, IFCN methodology transparency

Purpose: Provide transparent, measurable quality metrics that demonstrate AKEL's performance and build user trust in automated fact-checking.

Specification:

Component: Public Quality Dashboard

Core Metrics to Display:

• • 1. Verdict Quality Metrics

TIGERScore (Fact-Checking Quality):

• Definition: Measures how well generated verdicts match expert fact-checker judgments
• Scale: 0-100 (higher is better)
• Calculation: Using TIGERScore framework (Truth-conditional accuracy, Informativeness, Generality, Evaluativeness, Relevance)
• Target: Average ≥80 for production release
• Display:
  Verdict Quality (TIGERScore):
  Overall: 84.2 ▲ (+2.1 from last month)
  
  Distribution:
   Excellent (>80): 67%
   Good (60-80): 28%
   Needs Improvement (<60): 5%
  
  Trend: [Graph showing improvement over time]

2. Hallucination & Faithfulness Metrics

AlignScore (Faithfulness to Evidence):

• Definition: Measures how well verdicts align with actual evidence content
• Scale: 0-1 (higher is better)
• Purpose: Detect AI hallucinations (making claims not supported by evidence)
• Target: Average ≥0.85, hallucination rate <5%
• Display:
  Evidence Faithfulness (AlignScore):
  Average: 0.87 ▼ (-0.02 from last month)
  
  Hallucination Rate: 4.2%
  - Claims without evidence support: 3.1%
  - Misrepresented evidence: 1.1%
  
  Action: Prompt engineering review scheduled

3. Evidence Quality Metrics

Source Reliability:

• Average source quality score (0-1 scale)
• Distribution of high/medium/low quality sources
• Publisher track record trends

Evidence Coverage:

• Average number of sources per claim
• Percentage of claims with ≥2 sources (EFCSN minimum)
• Geographic diversity of sources

Display:
Evidence Quality:

Average Sources per Claim: 4.2
Claims with ≥2 sources: 94% (EFCSN compliant)

Source Quality Distribution:
 High quality (>0.8): 48%
 Medium quality (0.5-0.8): 43%
 Low quality (<0.5): 9%

Geographic Diversity: 23 countries represented

4. Contributor Consensus Metrics (when human reviewers involved)

Inter-Rater Reliability (IRR):

• Calculation: Cohen's Kappa or Fleiss' Kappa for multiple raters
• Scale: 0-1 (higher is better)
• Interpretation:
• >0.8: Almost perfect agreement
• 0.6-0.8: Substantial agreement
• 0.4-0.6: Moderate agreement
• <0.4: Poor agreement
• Target: Maintain ≥0.7 (substantial agreement)

Display:
Contributor Consensus:

Inter-Rater Reliability (IRR): 0.73 (Substantial agreement)
- Verdict agreement: 78%
- Evidence quality agreement: 71%
- Scenario structure agreement: 69%

Cases requiring moderator review: 12
Moderator override rate: 8%

-

Quality Dashboard Implementation

Dashboard Location: `/quality-metrics`

Update Frequency:

• POC2: Weekly manual updates
• Beta 0: Daily automated updates
• V1.0: Real-time metrics (updated hourly)

Dashboard Sections:

1. Overview: Key metrics at a glance
   2. Verdict Quality: TIGERScore trends and distributions
   3. Evidence Analysis: Source quality and coverage
   4. AI Performance: Hallucination rates, AlignScore
   5. Human Oversight: Contributor consensus, review rates
   6. System Health: Processing times, error rates, uptime

Example Dashboard Layout:

-

Continuous Improvement Feedback Loop

How Metrics Inform AKEL Improvements:

1. Identify Weak Areas:

• Low TIGERScore → Review prompt engineering
• High hallucination → Strengthen evidence grounding
• Low IRR → Clarify evaluation criteria

2. A/B Testing Integration:

• Test prompt variations
• Measure impact on quality metrics
• Deploy winners automatically

3. Alert Thresholds:

• TIGERScore drops below 75 → Alert team
• Hallucination rate exceeds 7% → Pause auto-publishing
• IRR below 0.6 → Moderator training needed

4. Monthly Quality Reviews:

• Analyze trends
• Identify systematic issues
• Plan prompt improvements
• Update AKEL models

-

Metric Calculation Details

TIGERScore Implementation:

• Reference: https://github.com/TIGER-AI-Lab/TIGERScore
• Input: Generated verdict + reference verdict (from expert)
• Output: 0-100 score across 5 dimensions
• Requires: Test set of expert-reviewed claims (minimum 100)

AlignScore Implementation:

• Reference: https://github.com/yuh-zha/AlignScore
• Input: Generated verdict + source evidence text
• Output: 0-1 faithfulness score
• Calculation: Semantic alignment between claim and evidence

Source Quality Scoring:

• Use existing source reliability database (e.g., NewsGuard, MBFC)
• Factor in: Publication history, corrections record, transparency
• Scale: 0-1 (weighted average across sources)

-

Integration Points

• NFR11: AKEL Quality Assurance - Metrics validate quality gate effectiveness
• FR49: A/B Testing - Metrics measure test success
• FR11: Audit Trail - Source of quality data
• NFR3: Transparency - Public metrics build trust

Acceptance Criteria:

• ✅ All core metrics implemented and calculating correctly
• ✅ Dashboard updates daily (Beta 0) or hourly (V1.0)
• ✅ Alerts trigger when metrics degrade beyond thresholds
• ✅ Monthly quality report auto-generates
• ✅ Dashboard is publicly accessible (no login required)
• ✅ Mobile-responsive dashboard design
• ✅ Metrics inform quarterly AKEL improvement planning

13. Requirements Traceability

For full traceability matrix showing which requirements fulfill which user needs, see:

• User Needs - Section 8 includes comprehensive mapping tables

14. Related Pages

Non-Functional Requirements (see Section 9):

• NFR11 — AKEL Quality Assurance Framework
• NFR12 — Security Controls
• NFR13 — Quality Metrics Transparency

Other Requirements:

• User Needs
• V1.0 Requirements
• Gap Analysis

• User Needs - What users need (drives these requirements)
• Architecture - How requirements are implemented
• Data Model - Data structures supporting requirements
• Workflows - User interaction workflows
• AKEL - AI system fulfilling automation requirements
• Global Rules
• Privacy Policy

V0.9.70 Additional Requirements

Functional Requirements (Additional)

FR44: ClaimReview Schema Implementation

Fulfills: UN-13 (Cite FactHarbor Verdicts), UN-14 (API Access for Integration), UN-26 (Search Engine Visibility)

Purpose: Generate valid ClaimReview structured data for every published analysis to enable Google/Bing search visibility and fact-check discovery.

Specification:

Component: Schema.org Markup Generator

FactHarbor must generate valid ClaimReview structured data following Schema.org specifications for every published claim analysis.

Required JSON-LD Schema:

FactHarbor-Specific Mapping:

Likelihood Score to Rating Scale:

• 80-100% likelihood → 5 (Highly Supported)
• 60-79% likelihood → 4 (Supported)
• 40-59% likelihood → 3 (Mixed/Uncertain)
• 20-39% likelihood → 2 (Questionable)
• 0-19% likelihood → 1 (Refuted)

Multiple Scenarios Handling:

• If claim has multiple scenarios with different verdicts, generate separate ClaimReview for each scenario
• Add `disambiguatingDescription` field explaining scenario context
• Example: "Scenario: If interpreted as referring to 2023 data..."

Implementation Requirements

1. Auto-generate on claim publication
   2. Embed in HTML `<head>` section as JSON-LD script
   3. Validate against Schema.org validator before publishing
   4. Submit to Google Search Console for indexing
   5. Update automatically when verdict changes (integrate with FR8: Time Evolution)

Integration Points

• FR7: Automated Verdicts - Source of rating data and claim text
• FR8: Time Evolution - Triggers schema updates when verdicts change
• FR11: Audit Trail - Logs all schema generation and update events

Resources

• ClaimReview Project: https://www.claimreviewproject.com
• Schema.org ClaimReview: https://schema.org/ClaimReview
• Google Fact Check Guidelines: https://developers.google.com/search/docs/appearance/fact-check

Acceptance Criteria:

• ✅ Passes Google Structured Data Testing Tool
• ✅ Appears in Google Fact Check Explorer within 48 hours of publication
• ✅ Valid JSON-LD syntax (no errors)
• ✅ All required fields populated with correct data types
• ✅ Handles multi-scenario claims correctly (separate ClaimReview per scenario)

FR45: User Corrections Notification System

Fulfills: IFCN Principle 5 (Open & Honest Corrections), EFCSN compliance

Purpose: When any claim analysis is corrected, notify users who previously viewed the claim to maintain transparency and build trust.

Specification:

Component: Corrections Visibility Framework

Correction Types:

1. Major Correction: Verdict changes category (e.g., "Supported" → "Refuted")
   2. Significant Correction: Likelihood score changes >20%
   3. Minor Correction: Evidence additions, source quality updates
   4. Scenario Addition: New scenario added to existing claim

Notification Mechanisms

• • 1. In-Page Banner:

Display prominent banner on claim page:

2. Correction Log Page:

• Public changelog at `/claims/{id}/corrections`
• Displays for each correction:
• Date/time of correction
• What changed (before/after comparison)
• Why changed (reason if provided)
• Who made change (AKEL auto-update vs. contributor override)

3. Email Notifications (opt-in):

• Send to users who bookmarked or shared the claim
• Subject: "FactHarbor Correction: [Claim title]"
• Include summary of changes
• Link to updated analysis

4. RSS/API Feed:

• Corrections feed at `/corrections.rss`
• API endpoint: `GET /api/corrections?since={timestamp}`
• Enables external monitoring by journalists and researchers

Display Rules

• Show banner on ALL pages displaying the claim (search results, related claims, embeddings)
• Banner persists for 30 days after correction
• "Corrections" count badge on claim card
• Timestamp on every verdict: "Last updated: [datetime]"

IFCN Compliance Requirements

• Corrections policy published at `/corrections-policy`
• User can report suspected errors via `/report-error/{claim_id}`
• Link to IFCN complaint process (if FactHarbor becomes signatory)
• Scrupulous transparency: Never silently edit analyses

Integration Points

• FR8: Time Evolution - Triggers corrections when verdicts change
• FR11: Audit Trail - Source of correction data and change history
• NFR3: Transparency - Public correction log demonstrates commitment

Acceptance Criteria:

• ✅ Banner appears within 60 seconds of correction
• ✅ Correction log is permanent and publicly accessible
• ✅ Email notifications deliver within 5 minutes
• ✅ RSS feed updates in real-time
• ✅ Mobile-responsive banner design
• ✅ Accessible (screen reader compatible)

FR46: Image Verification System

Fulfills: UN-27 (Visual Claim Verification)

Purpose: Verify authenticity and context of images shared with claims to detect manipulation, misattribution, and out-of-context usage.

Specification:

Component: Multi-Method Image Verification

Method 1: Reverse Image Search

Purpose: Find earlier uses of the image to verify context

Implementation:

• Integrate APIs:
• Google Vision AI (reverse search)
• TinEye (oldest known uses)
• Bing Visual Search (broad coverage)

Process:

1. Extract image from claim or user upload
   2. Query multiple reverse search services
   3. Analyze results for:

• Earliest known publication
• Original context (what was it really showing?)
• Publication timeline
• Geographic spread

Output:
Reverse Image Search Results:

Earliest known use: 2019-03-15 (5 years before claim)
Original context: "Photo from 2019 flooding in Mumbai"
This claim uses it for: "2024 hurricane damage in Florida"

⚠️ Image is OUT OF CONTEXT

Found in 47 other articles:
• 2019-03-15: Mumbai floods (original)
• 2020-07-22: Bangladesh monsoon
• 2024-10-15: Current claim (misattributed)

[View full timeline]

-

Method 2: AI Manipulation Detection

Purpose: Detect deepfakes, face swaps, and digital alterations

Implementation:

• Integrate detection services:
• Sensity AI (deepfake detection)
• Reality Defender (multimodal analysis)
• AWS Rekognition (face detection inconsistencies)

Detection Categories:

1. Face Manipulation:

• Deepfake face swaps
• Expression manipulation
• Identity replacement

2. Image Manipulation:

• Copy-paste artifacts
• Clone stamp detection
• Content-aware fill detection
• JPEG compression inconsistencies

3. AI Generation:

• Detect fully AI-generated images
• Identify generation artifacts
• Check for model signatures

Confidence Scoring:

• HIGH (80-100%): Strong evidence of manipulation
• MEDIUM (50-79%): Suspicious artifacts detected
• LOW (0-49%): Minor inconsistencies or inconclusive

Output:
Manipulation Analysis:

Face Manipulation: LOW RISK (12%)
Image Editing: MEDIUM RISK (64%)
• Clone stamp artifacts detected in sky region
• JPEG compression inconsistent between objects

AI Generation: LOW RISK (8%)

⚠️ Possible manipulation detected. Manual review recommended.

-

Method 3: Metadata Analysis (EXIF)

Purpose: Extract technical details that may reveal manipulation or misattribution

Extracted Data:

• Camera/Device: Make, model, software
• Timestamps: Original date, modification dates
• Location: GPS coordinates (if present)
• Editing History: Software used, edit count
• File Properties: Resolution, compression, format conversions

Red Flags:

• Metadata completely stripped (suspicious)
• Timestamp conflicts with claimed date
• GPS location conflicts with claimed location
• Multiple edit rounds (hiding something?)
• Creation date after modification date (impossible)

Output:
Image Metadata:

Camera: iPhone 14 Pro
Original date: 2023-08-12 14:32:15
Location: 40.7128°N, 74.0060°W (New York City)
Modified: 2024-10-15 08:45:22
Software: Adobe Photoshop 2024

⚠️ Location conflicts with claim
Claim says: "Taken in Los Angeles"
EXIF says: New York City

⚠️ Edited 14 months after capture

-

Verification Workflow

Automatic Triggers:

1. User submits claim with image
   2. Article being analyzed contains images
   3. Social media post includes photos

Process:

1. Extract images from content
   2. Run all 3 verification methods in parallel
   3. Aggregate results into confidence score
   4. Generate human-readable summary
   5. Display prominently in analysis

Display Integration:

Show image verification panel in claim analysis:

Integration Points

• FR7: Automated Verdicts - Image verification affects claim credibility
• FR4: Analysis Summary - Image findings included in summary
• UN-27: Visual Claim Verification - Direct fulfillment

Cost Considerations

API Costs (estimated per image):

• Google Vision AI: $0.001-0.003
• TinEye: $0.02 (commercial API)
• Sensity AI: $0.05-0.10
• AWS Rekognition: $0.001-0.002

Total per  $0.07-0.15

Mitigation Strategies:

• Cache results for duplicate images
• Use free tier quotas where available
• Prioritize higher-value claims for deep analysis
• Offer premium verification as paid tier

Acceptance Criteria:

• ✅ Reverse image search finds original sources
• ✅ Manipulation detection accuracy >80% on test dataset
• ✅ EXIF extraction works for major image formats (JPEG, PNG, HEIC)
• ✅ Results display within 10 seconds
• ✅ Mobile-friendly image comparison interface
• ✅ False positive rate <15%

FR47: Archive.org Integration

Importance: CRITICAL
Fulfills: Evidence persistence, FR5 (Evidence linking) 

Purpose: Ensure evidence remains accessible even if original sources are deleted.

Specification:

Automatic Archiving:

When AKEL links evidence:

1. Check if URL already archived (Wayback Machine API)
   2. If not, submit for archiving (Save Page Now API)
   3. Store both original URL and archive URL
   4. Display both to users

Archive Display:

Fallback Logic:

• If original URL unavailable → Auto-redirect to archive
• If archive unavailable → Display warning
• If both unavailable → Flag for manual review

API Integration:

• Use Wayback Machine Availability API
• Use Save Page Now API (SPNv2)
• Rate limiting: 15 requests/minute (Wayback limit)

Acceptance Criteria:

• ✅ All evidence URLs auto-archived
• ✅ Archive links displayed to users
• ✅ Fallback to archive if original unavailable
• ✅ API rate limits respected
• ✅ Archive status visible in evidence display

Category 4: Community Safety

 FR48: Contributor Safety Framework ===

Importance: CRITICAL
Fulfills: UN-28 (Safe contribution environment) 

Purpose: Protect contributors from harassment, doxxing, and coordinated attacks.

Specification:

• • 1. Privacy Protection:

• Optional Pseudonymity: Contributors can use pseudonyms
• Email Privacy: Emails never displayed publicly
• Profile Privacy: Contributors control what's public
• IP Logging: Only for abuse prevention, not public

2. Harassment Prevention:

• Automated Toxicity Detection: Flag abusive comments
• Personal Information Detection: Auto-block doxxing attempts
• Coordinated Attack Detection: Identify brigading patterns
• Rapid Response: Moderator alerts for harassment

3. Safety Features:

• Block Users: Contributors can block harassers
• Private Contributions: Option to contribute anonymously
• Report Harassment: One-click harassment reporting
• Safety Resources: Links to support resources

4. Moderator Tools:

• Quick Ban: Immediately block abusers
• Pattern Detection: Identify coordinated attacks
• Appeal Process: Fair review of moderation actions
• Escalation: Serious threats escalated to authorities

5. Trusted Contributor Protection:

• Enhanced Privacy: Additional protection for high-profile contributors
• Verification: Optional identity verification (not public)
• Legal Support: Resources for contributors facing legal threats

Acceptance Criteria:

• ✅ Pseudonyms supported
• ✅ Toxicity detection active
• ✅ Doxxing auto-blocked
• ✅ Harassment reporting functional
• ✅ Moderator tools implemented
• ✅ Safety policy published

Category 5: Continuous Improvement

 FR49: A/B Testing Framework ===

Importance: CRITICAL
Fulfills: Continuous system improvement 

Purpose: Test and measure improvements to AKEL prompts, algorithms, and workflows.

Specification:

Test Capabilities:

1. Prompt Variations:

• Test different claim extraction prompts
• Test different verdict generation prompts
• Measure: Accuracy, clarity, completeness

2. Algorithm Variations:

• Test different source scoring algorithms
• Test different confidence calculations
• Measure: Audit accuracy, user satisfaction

3. Workflow Variations:

• Test different quality gate thresholds
• Test different risk tier assignments
• Measure: Publication rate, quality scores

Implementation:

• Traffic Split: 50/50 or 90/10 splits
• Randomization: Consistent per claim (not per user)
• Metrics Collection: Automatic for all variants
• Statistical Significance: Minimum sample size calculation
• Rollout: Winner promoted to 100% traffic

A/B Test Workflow:

Acceptance Criteria:

• ✅ A/B testing framework implemented
• ✅ Can test prompt variations
• ✅ Can test algorithm variations
• ✅ Metrics automatically collected
• ✅ Statistical significance calculated
• ✅ Results inform system improvements

FR54: Evidence Deduplication

Importance: CRITICAL (POC2/Beta)
Fulfills: Accurate evidence counting, quality metrics 

Purpose: Avoid counting the same source multiple times when it appears in different forms.

Specification:

Deduplication Logic:

1. URL Normalization:

• Remove tracking parameters (?utm_source=...)
• Normalize http/https
• Normalize www/non-www
• Handle redirects

2. Content Similarity:

• If two sources have >90% text similarity → Same source
• If one is subset of other → Same source
• Use fuzzy matching for minor differences

3. Cross-Domain Syndication:

• Detect wire service content (AP, Reuters)
• Mark as single source if syndicated
• Count original publication only

Display:

Acceptance Criteria:

• ✅ URL normalization works
• ✅ Content similarity detected
• ✅ Syndicated content identified
• ✅ Unique vs. total counts accurate
• ✅ Improves evidence quality metrics

Additional Requirements (Lower Importance)

 FR50: OSINT Toolkit Integration ===

Fulfills: Advanced media verification 

Purpose: Integrate open-source intelligence tools for advanced verification.

Tools to Integrate:

• InVID/WeVerify (video verification)
• Bellingcat toolkit
• Additional TBD based on V1.0 learnings

FR51: Video Verification System

Fulfills: UN-27 (Visual claims), advanced media verification 

Purpose: Verify video-based claims.

Specification:

• Keyframe extraction
• Reverse video search
• Deepfake detection (AI-powered)
• Metadata analysis
• Acoustic signature analysis

FR52: Interactive Detection Training

Fulfills: Media literacy education 

Purpose: Teach users to identify misinformation.

Specification:

• Interactive tutorials
• Practice exercises
• Detection quizzes
• Gamification elements

FR53: Cross-Organizational Sharing

Fulfills: Collaboration with other fact-checkers 

Purpose: Share findings with IFCN/EFCSN members.

Specification:

• API for fact-checking organizations
• Structured data exchange
• Privacy controls
• Attribution requirements

Summary

V1.0 Critical Requirements (Must Have):

• FR44: ClaimReview Schema ✅
• FR45: Corrections Notification ✅
• FR46: Image Verification ✅
• FR47: Archive.org Integration ✅
• FR48: Contributor Safety ✅
• FR49: A/B Testing ✅
• FR54: Evidence Deduplication ✅
• NFR11: Quality Assurance Framework ✅
• NFR12: Security Controls ✅
• NFR13: Quality Metrics Dashboard ✅

V1.1+ (Future):

• FR50: OSINT Integration
• FR51: Video Verification
• FR52: Detection Training
• FR53: Cross-Org Sharing

Total: 11 critical requirements for V1.0

FR54: Evidence Deduplication

Fulfills: Accurate evidence counting, quality metrics 

Purpose: Avoid counting the same source multiple times when it appears in different forms.

Specification:

Deduplication Logic:

1. URL Normalization:

• Remove tracking parameters (?utm_source=...)
• Normalize http/https
• Normalize www/non-www
• Handle redirects

2. Content Similarity:

• If two sources have >90% text similarity → Same source
• If one is subset of other → Same source
• Use fuzzy matching for minor differences

3. Cross-Domain Syndication:

• Detect wire service content (AP, Reuters)
• Mark as single source if syndicated
• Count original publication only

Display:

Acceptance Criteria:

• ✅ URL normalization works
• ✅ Content similarity detected
• ✅ Syndicated content identified
• ✅ Unique vs. total counts accurate
• ✅ Improves evidence quality metrics

Additional Requirements (Lower Importance)

 FR7: Automated Verdicts (Enhanced with Quality Gates) ===

POC1+ Enhancement:

After AKEL generates verdict, it passes through quality gates:

Updated Verdict States:

• PUBLISHED
• INSUFFICIENT_EVIDENCE
• NON_FACTUAL_CLAIM
• PROCESSING
• ERROR

FR4: Analysis Summary (Enhanced with Quality Metadata)

POC1+ Enhancement:

Display quality indicators: