Requirements

This page defines Roles, Content States, Rules, and System Principles for FactHarbor.
Core Philosophy: Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically.

1. Roles

FactHarbor uses three simple roles plus a reputation system.

1.1 Reader

Who: Anyone (no login required)
Can:

• Browse and search claims
• View scenarios, evidence, verdicts, and confidence scores
• Flag issues or errors
• Use filters, search, and visualization tools
• Submit claims automatically (new claims added if not duplicates)
  Cannot:
• Modify content
• Access edit history details

1.2 Contributor

Who: Registered users (earns reputation through contributions)
Can:

• Everything a Reader can do
• Edit claims, evidence, and scenarios
• Add sources and citations
• Suggest improvements to AI-generated content
• Participate in discussions
• Earn reputation points for quality contributions
  Reputation System:
• New contributors: Limited edit privileges
• Established contributors (established reputation): Full edit access
• Trusted contributors (substantial reputation): Can approve certain changes
• Reputation earned through: Accepted edits, helpful flags, quality contributions
• Reputation lost through: Reverted edits, invalid flags, abuse
  Cannot:
• Delete or hide content (only moderators)
• Override moderation decisions

1.3 Moderator

Who: Trusted community members with proven track record, appointed by governance board
Can:

• Review flagged content
• Hide harmful or abusive content
• Resolve disputes between contributors
• Issue warnings or temporary bans
• Make final decisions on content disputes
• Access full audit logs
  Cannot:
• Change governance rules
• Permanently ban users without board approval
• Override technical quality gates
  Note: Small team (3-5 initially), supported by automated moderation tools.

1.4 Domain Trusted Contributors (Optional, Task-Specific)

Who: Subject matter specialists invited for specific high-stakes disputes
Not a permanent role: Contacted externally when needed for contested claims in their domain
When used:

• Medical claims with life/safety implications
• Legal interpretations with significant impact
• Scientific claims with high controversy
• Technical claims requiring specialized knowledge
  Process:
• Moderator identifies need for expert input
• Contact expert externally (don't require them to be users)
• Trusted Contributor provides written opinion with sources
• Opinion added to claim record
• Trusted Contributor acknowledged in claim

2. Content States

FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping.

2.1 Published

Status: Visible to all users
Includes:

• AI-generated analyses (default state)
• User-contributed content
• Edited/improved content
  Quality Indicators (displayed with content):
• Confidence Score: 0-100% (AI's confidence in analysis)
• Source Quality Score: 0-100% (based on source track record)
• Controversy Flag: If high dispute/edit activity
• Completeness Score: % of expected fields filled
• Last Updated: Date of most recent change
• Edit Count: Number of revisions
  Automatic Warnings:
• Confidence < 60%: "Low confidence - use caution"
• Source quality < 40%: "Sources may be unreliable"
• High controversy: "Disputed - multiple interpretations exist"
• Medical/Legal/Safety domain: "Seek professional advice"

2.2 Hidden

Status: Not visible to regular users (only to moderators)
Reasons:

• Spam or advertising
• Personal attacks or harassment
• Illegal content
• Privacy violations
• Deliberate misinformation (verified)
• Abuse or harmful content
  Process:
• Automated detection flags for moderator review
• Moderator confirms and hides
• Original author notified with reason
• Can appeal to board if disputes moderator decision
  Note: Content is hidden, not deleted (for audit trail)

3. Contribution Rules

3.1 All Contributors Must

• Provide sources for factual claims
• Use clear, neutral language in FactHarbor's own summaries
• Respect others and maintain civil discourse
• Accept community feedback constructively
• Focus on improving quality, not protecting ego

3.2 AKEL (AI System)

AKEL is the primary system. Human contributions supplement and train AKEL.
AKEL Must:

• Mark all outputs as AI-generated
• Display confidence scores prominently
• Provide source citations
• Flag uncertainty clearly
• Identify contradictions in evidence
• Learn from human corrections
  When AKEL Makes Errors:

1. Capture the error pattern (what, why, how common)
   2. Improve the system (better prompt, model, validation)
   3. Re-process affected claims automatically
   4. Measure improvement (did quality increase?)
   Human Role: Train AKEL through corrections, not replace AKEL

3.3 Contributors Should

• Improve clarity and structure
• Add missing sources
• Flag errors for system improvement
• Suggest better ways to present information
• Participate in quality discussions

3.4 Moderators Must

• Be impartial
• Document moderation decisions
• Respond to appeals promptly
• Use automated tools to scale efforts
• Focus on abuse/harm, not routine quality control

4. Quality Standards

4.1 Source Requirements

Track Record Over Credentials:

• Sources evaluated by historical accuracy
• Correction policy matters
• Independence from conflicts of interest
• Methodology transparency
  Source Quality Database:
• Automated tracking of source accuracy
• Correction frequency
• Reliability score (updated continuously)
• Users can see source track record
  No automatic trust for government, academia, or media - all evaluated by track record.

4.2 Claim Requirements

• Clear subject and assertion
• Verifiable with available information
• Sourced (or explicitly marked as needing sources)
• Neutral language in FactHarbor summaries
• Appropriate context provided

4.3 Evidence Requirements

• Publicly accessible (or explain why not)
• Properly cited with attribution
• Relevant to claim being evaluated
• Original source preferred over secondary

4.4 Confidence Scoring

Automated confidence calculation based on:

• Source quality scores
• Evidence consistency
• Contradiction detection
• Completeness of analysis
• Historical accuracy of similar claims
  Thresholds:
• < 40%: Too low to publish (needs improvement)
• 40-60%: Published with "Low confidence" warning
• 60-80%: Published as standard
• 80-100%: Published as "High confidence"

5. Automated Risk Scoring

Replace manual risk tiers with continuous automated scoring.

5.1 Risk Score Calculation

Factors (weighted algorithm):

• Domain sensitivity: Medical, legal, safety auto-flagged higher
• Potential impact: Views, citations, spread
• Controversy level: Flags, disputes, edit wars
• Uncertainty: Low confidence, contradictory evidence
• Source reliability: Track record of sources used
  Score: 0-100 (higher = more risk)

5.2 Automated Actions

• Score > 80: Flag for moderator review before publication
• Score 60-80: Publish with prominent warnings
• Score 40-60: Publish with standard warnings
• Score < 40: Publish normally
  Continuous monitoring: Risk score recalculated as new information emerges

6. System Improvement Process

Core principle: Fix the system, not just the data.

6.1 Error Capture

When users flag errors or make corrections:

1. What was wrong? (categorize)
   2. What should it have been?
   3. Why did the system fail? (root cause)
   4. How common is this pattern?
   5. Store in ErrorPattern table (improvement queue)

6.2 Weekly Improvement Cycle

1. Review: Analyze top error patterns
   2. Develop: Create fix (prompt, model, validation)
   3. Test: Validate fix on sample claims
   4. Deploy: Roll out if quality improves
   5. Re-process: Automatically update affected claims
   6. Monitor: Track quality metrics

6.3 Quality Metrics Dashboard

Track continuously:

• Error rate by category
• Source quality distribution
• Confidence score trends
• User flag rate (issues found)
• Correction acceptance rate
• Re-work rate
• Claims processed per hour
  Goal: 10% monthly improvement in error rate

7. Automated Quality Monitoring

Replace manual audit sampling with automated monitoring.

7.1 Continuous Metrics

• Source quality: Track record database
• Consistency: Contradiction detection
• Clarity: Readability scores
• Completeness: Field validation
• Accuracy: User corrections tracked

7.2 Anomaly Detection

Automated alerts for:

• Sudden quality drops
• Unusual patterns
• Contradiction clusters
• Source reliability changes
• User behavior anomalies

7.3 Targeted Review

• Review only flagged items
• Random sampling for calibration (not quotas)
• Learn from corrections to improve automation

8. Claim Intake & Normalization

8.1 FR1 – Claim Intake

• Users submit claims via simple form or API
• Claims can be text, URL, or image
• Duplicate detection (semantic similarity)
• Auto-categorization by domain

8.2 FR2 – Claim Normalization

• Standardize to clear assertion format
• Extract key entities (who, what, when, where)
• Identify claim type (factual, predictive, evaluative)
• Link to existing similar claims

8.3 FR3 – Claim Classification

• Domain: Politics, Science, Health, etc.
• Type: Historical fact, current stat, prediction, etc.
• Risk score: Automated calculation
• Complexity: Simple, moderate, complex

9. Scenario System

9.1 FR4 – Scenario Generation

Automated scenario creation:

• AKEL analyzes claim and generates likely scenarios
• Each scenario includes: assumptions, evidence, conclusion
• Users can flag incorrect scenarios
• System learns from corrections

9.2 FR5 – Evidence Linking

• Automated evidence discovery from sources
• Relevance scoring
• Contradiction detection
• Source quality assessment

9.3 FR6 – Scenario Comparison

• Side-by-side comparison interface
• Highlight key differences
• Show evidence supporting each
• Display confidence scores

10. Verdicts & Analysis

10.1 FR7 – Automated Verdicts

• AKEL generates verdict based on evidence
• Confidence score displayed prominently
• Source quality indicators
• Contradictions noted
• Uncertainty acknowledged

10.2 FR8 – Time Evolution

• Claims update as new evidence emerges
• Version history maintained
• Changes highlighted
• Confidence score trends visible

11. Workflow & Moderation

11.1 FR9 – Publication Workflow

Simple flow:

1. Claim submitted
   2. AKEL processes (automated)
   3. If confidence > threshold: Publish
   4. If confidence < threshold: Flag for improvement
   5. If risk score > threshold: Flag for moderator
   No multi-stage approval process

11.2 FR10 – Moderation

Focus on abuse, not routine quality:

• Automated abuse detection
• Moderators handle flags
• Quick response to harmful content
• Minimal involvement in routine content

11.3 FR11 – Audit Trail

• All edits logged
• Version history public
• Moderation decisions documented
• System improvements tracked

12. Technical Requirements

12.1 NFR1 – Performance

• Claim processing: < 30 seconds
• Search response: < 2 seconds
• Page load: < 3 seconds
• 99% uptime

12.2 NFR2 – Scalability

• Handle 10,000 claims initially
• Scale to 1M+ claims
• Support 100K+ concurrent users
• Automated processing scales linearly

12.3 NFR3 – Transparency

• All algorithms open source
• All data exportable
• All decisions documented
• Quality metrics public

12.4 NFR4 – Security & Privacy

• Follow Privacy Policy
• Secure authentication
• Data encryption
• Regular security audits

12.5 NFR5 – Maintainability

• Modular architecture
• Automated testing
• Continuous integration
• Comprehensive documentation

13. MVP Scope

Phase 1 (Months 1-3): Read-Only MVP
Build:

• Automated claim analysis
• Confidence scoring
• Source evaluation
• Browse/search interface
• User flagging system
  Goal: Prove AI quality before adding user editing
  Phase 2 (Months 4-6): User Contributions
  Add only if needed:
• Simple editing (Wikipedia-style)
• Reputation system
• Basic moderation
  Phase 3 (Months 7-12): Refinement
• Continuous quality improvement
• Feature additions based on real usage
• Scale infrastructure
  Deferred:
• Federation (until multiple successful instances exist)
• Complex contribution workflows (focus on automation)
• Extensive role hierarchy (keep simple)

14. Success Metrics

System Quality (track weekly):

• Error rate by category (target: -10%/month)
• Average confidence score (target: increase)
• Source quality distribution (target: more high-quality)
• Contradiction detection rate (target: increase)
  Efficiency (track monthly):
• Claims processed per hour (target: increase)
• Human hours per claim (target: decrease)
• Automation coverage (target: >90%)
• Re-work rate (target: <5%)
  User Satisfaction (track quarterly):
• User flag rate (issues found)
• Correction acceptance rate (flags valid)
• Return user rate
• Trust indicators (surveys)

15. Related Pages

• Architecture
• Data Model
• Workflows
• Global Rules
• Privacy Policy