Automation

How FactHarbor scales through automated claim evaluation.

1. Automation Philosophy

FactHarbor is automation-first: AKEL (AI Knowledge Extraction Layer) makes all content decisions. Humans monitor system performance and improve algorithms.
Why automation:

• Scale: Can process millions of claims
• Consistency: Same evaluation criteria applied uniformly
• Transparency: Algorithms are auditable
• Speed: Results in <20 seconds typically
  See Automation Philosophy for detailed principles.

2. Claim Processing Flow

2.1 User Submits Claim

• User provides claim text + source URLs
• System validates format
• Assigns processing ID
• Queues for AKEL processing

2.2 AKEL Processing

AKEL automatically:

1. Parses claim into testable components
   2. Extracts evidence from sources
   3. Scores source credibility
   4. Evaluates claim against evidence
   5. Generates verdict with confidence score
   6. Assigns risk tier (A/B/C)
   7. Publishes result
   Processing time: Typically <20 seconds
   No human approval required - publication is automatic

2.3 Publication States

Processing: AKEL working on claim (not visible to public)
Published: AKEL completed evaluation (public)

• Verdict displayed with confidence score
• Evidence and sources shown
• Risk tier indicated
• Users can report issues
  Flagged: AKEL identified issue requiring moderator attention (still public)
• Low confidence below threshold
• Detected manipulation attempt
• Unusual pattern
• Moderator reviews and may take action

3. Risk Tiers

Risk tiers classify claims by potential impact and guide audit sampling rates.

3.1 Tier A (High Risk)

Domains: Medical, legal, elections, safety, security
Characteristics:

• High potential for harm if incorrect
• Complex specialized knowledge required
• Often subject to regulation
  Publication: AKEL publishes automatically with prominent risk warning
  Audit rate: Higher sampling recommended

3.2 Tier B (Medium Risk)

Domains: Complex policy, science, causality claims
Characteristics:

• Moderate potential impact
• Requires careful evidence evaluation
• Multiple valid interpretations possible
  Publication: AKEL publishes automatically with standard risk label
  Audit rate: Moderate sampling recommended

3.3 Tier C (Low Risk)

Domains: Definitions, established facts, historical data
Characteristics:

• Low potential for harm
• Well-documented information
• Clear right/wrong answers typically
  Publication: AKEL publishes by default
  Audit rate: Lower sampling recommended

4. Quality Gates

AKEL applies quality gates before publication. If any fail, claim is flagged (not blocked - still published).
Quality gates:

• Sufficient evidence extracted (≥2 sources)
• Sources meet minimum credibility threshold
• Confidence score calculable
• No detected manipulation patterns
• Claim parseable into testable form
  Failed gates: Claim published with flag for moderator review

5. Automation Levels

Automation Maturity Progression

graph TD
    POC[Level 0 POC Demo CURRENT]
    R05[Level 0.5 Limited Production]
    R10[Level 1.0 Full Production]
    R20[Level 2.0+ Distributed Intelligence]

    POC --> R05
    R05 --> R10
    R10 --> R20

Level Descriptions

Level	Name	Key Features
Level 0	POC/Demo (CURRENT)	All content auto-analyzed, AKEL generates verdicts, no risk tier filtering, single-user demo mode
Level 0.5	Limited Production	Multi-user support, risk tier classification, basic sampling audit, algorithm improvement focus
Level 1.0	Full Production	All tiers auto-published, clear risk labels, reduced sampling, mature algorithms
Level 2.0+	Distributed	Federated multi-node, cross-node audits, advanced patterns, strategic sampling only

Current Implementation (v2.6.33)

Feature	POC Target	Actual Status
AKEL auto-analysis	Yes	Implemented
Verdict generation	Yes	Implemented (7-point scale)
Quality Gates	Basic	Gates 1 and 4 implemented
Risk tiers	Yes	Not implemented
Sampling audits	High sampling	Not implemented
User system	Demo only	Anonymous only

Key Principles

Across All Levels:

• AKEL makes all publication decisions
• No human approval gates
• Humans monitor metrics and improve algorithms
• Risk tiers guide audit priorities, not publication
• Sampling audits inform improvements

FactHarbor progresses through automation maturity levels:
Release 0.5 (Proof-of-Concept): Tier C only, human review required
Release 1.0 (Initial): Tier B/C auto-published, Tier A flagged for review
Release 2.0 (Mature): All tiers auto-published with risk labels, sampling audits
See Automation Roadmap for detailed progression.

6. Human Role

Humans do NOT review content for approval. Instead:
Monitoring: Watch aggregate performance metrics
Improvement: Fix algorithms when patterns show issues
Exception handling: Review AKEL-flagged items
Governance: Set policies AKEL applies
See Contributor Processes for how to improve the system.

7. Moderation

Moderators handle items AKEL flags:
Abuse detection: Spam, manipulation, harassment
Safety issues: Content that could cause immediate harm
System gaming: Attempts to manipulate scoring
Action: May temporarily hide content, ban users, or propose algorithm improvements
Does NOT: Routinely review claims or override verdicts
See Organisational Model for moderator role details.

8. Continuous Improvement

Performance monitoring: Track AKEL accuracy, speed, coverage
Issue identification: Find systematic errors from metrics
Algorithm updates: Deploy improvements to fix patterns
A/B testing: Validate changes before full rollout
Retrospectives: Learn from failures systematically
See Continuous Improvement for improvement cycle.

9. Scalability

Automation enables FactHarbor to scale:

• Millions of claims processable
• Consistent quality at any volume
• Cost efficiency through automation
• Rapid iteration on algorithms
  Without automation: Human review doesn't scale, creates bottlenecks, introduces inconsistency.

10. Transparency

All automation is transparent:

• Algorithm parameters documented
• Evaluation criteria public
• Source scoring rules explicit
• Confidence calculations explained
• Performance metrics visible
  See System Performance Metrics for what we measure.