Skip to Content

Wiki source code of Automation

Version 2.1 by Robert Schaub on 2025/12/18 12:54
Show last authors
1 = Automation =
2 **How FactHarbor scales through automated claim evaluation.**
3 == 1. Automation Philosophy ==
4 FactHarbor is **automation-first**: AKEL (AI Knowledge Extraction Layer) makes all content decisions. Humans monitor system performance and improve algorithms.
5 **Why automation:**
6 * **Scale**: Can process millions of claims
7 * **Consistency**: Same evaluation criteria applied uniformly
8 * **Transparency**: Algorithms are auditable
9 * **Speed**: Results in <20 seconds typically
10 See [[Automation Philosophy>>FactHarbor.Organisation.Automation-Philosophy]] for detailed principles.
11 == 2. Claim Processing Flow ==
12 === 2.1 User Submits Claim ===
13 * User provides claim text + source URLs
14 * System validates format
15 * Assigns processing ID
16 * Queues for AKEL processing
17 === 2.2 AKEL Processing ===
18 **AKEL automatically:**
19 1. Parses claim into testable components
20 2. Extracts evidence from sources
21 3. Scores source credibility
22 4. Evaluates claim against evidence
23 5. Generates verdict with confidence score
24 6. Assigns risk tier (A/B/C)
25 7. Publishes result
26 **Processing time**: Typically <20 seconds
27 **No human approval required** - publication is automatic
28 === 2.3 Publication States ===
29 **Processing**: AKEL working on claim (not visible to public)
30 **Published**: AKEL completed evaluation (public)
31 * Verdict displayed with confidence score
32 * Evidence and sources shown
33 * Risk tier indicated
34 * Users can report issues
35 **Flagged**: AKEL identified issue requiring moderator attention (still public)
36 * Low confidence below threshold
37 * Detected manipulation attempt
38 * Unusual pattern
39 * Moderator reviews and may take action
40 == 3. Risk Tiers ==
41 Risk tiers classify claims by potential impact and guide audit sampling rates.
42 === 3.1 Tier A (High Risk) ===
43 **Domains**: Medical, legal, elections, safety, security
44 **Characteristics**:
45 * High potential for harm if incorrect
46 * Complex specialized knowledge required
47 * Often subject to regulation
48 **Publication**: AKEL publishes automatically with prominent risk warning
49 **Audit rate**: Higher sampling recommended
50 === 3.2 Tier B (Medium Risk) ===
51 **Domains**: Complex policy, science, causality claims
52 **Characteristics**:
53 * Moderate potential impact
54 * Requires careful evidence evaluation
55 * Multiple valid interpretations possible
56 **Publication**: AKEL publishes automatically with standard risk label
57 **Audit rate**: Moderate sampling recommended
58 === 3.3 Tier C (Low Risk) ===
59 **Domains**: Definitions, established facts, historical data
60 **Characteristics**:
61 * Low potential for harm
62 * Well-documented information
63 * Clear right/wrong answers typically
64 **Publication**: AKEL publishes by default
65 **Audit rate**: Lower sampling recommended
66 == 4. Quality Gates ==
67 AKEL applies quality gates before publication. If any fail, claim is **flagged** (not blocked - still published).
68 **Quality gates**:
69 * Sufficient evidence extracted (≥2 sources)
70 * Sources meet minimum credibility threshold
71 * Confidence score calculable
72 * No detected manipulation patterns
73 * Claim parseable into testable form
74 **Failed gates**: Claim published with flag for moderator review
75 == 5. Automation Levels ==
76 {{include reference="FactHarbor.Specification.Diagrams.Automation Level.WebHome"/}}
77 FactHarbor progresses through automation maturity levels:
78 **Release 0.5** (Proof-of-Concept): Tier C only, human review required
79 **Release 1.0** (Initial): Tier B/C auto-published, Tier A flagged for review
80 **Release 2.0** (Mature): All tiers auto-published with risk labels, sampling audits
81 See [[Automation Roadmap>>FactHarbor.Specification.Diagrams.Automation Roadmap.WebHome]] for detailed progression.
82
83 == 5.5 Automation Roadmap ==
84
85 {{include reference="FactHarbor.Specification.Diagrams.Automation Roadmap.WebHome"/}}
86
87 == 6. Human Role ==
88 Humans do NOT review content for approval. Instead:
89 **Monitoring**: Watch aggregate performance metrics
90 **Improvement**: Fix algorithms when patterns show issues
91 **Exception handling**: Review AKEL-flagged items
92 **Governance**: Set policies AKEL applies
93 See [[Contributor Processes>>FactHarbor.Organisation.Contributor-Processes]] for how to improve the system.
94
95 == 6.5 Manual vs Automated Matrix ==
96
97 {{include reference="FactHarbor.Specification.Diagrams.Manual vs Automated matrix.WebHome"/}}
98
99 == 7. Moderation ==
100 Moderators handle items AKEL flags:
101 **Abuse detection**: Spam, manipulation, harassment
102 **Safety issues**: Content that could cause immediate harm
103 **System gaming**: Attempts to manipulate scoring
104 **Action**: May temporarily hide content, ban users, or propose algorithm improvements
105 **Does NOT**: Routinely review claims or override verdicts
106 See [[Organisational Model>>FactHarbor.Organisation.Organisational-Model]] for moderator role details.
107 == 8. Continuous Improvement ==
108 **Performance monitoring**: Track AKEL accuracy, speed, coverage
109 **Issue identification**: Find systematic errors from metrics
110 **Algorithm updates**: Deploy improvements to fix patterns
111 **A/B testing**: Validate changes before full rollout
112 **Retrospectives**: Learn from failures systematically
113 See [[Continuous Improvement>>FactHarbor.Organisation.How-We-Work-Together.Continuous-Improvement]] for improvement cycle.
114 == 9. Scalability ==
115 Automation enables FactHarbor to scale:
116 * **Millions of claims** processable
117 * **Consistent quality** at any volume
118 * **Cost efficiency** through automation
119 * **Rapid iteration** on algorithms
120 Without automation: Human review doesn't scale, creates bottlenecks, introduces inconsistency.
121 == 10. Transparency ==
122 All automation is transparent:
123 * **Algorithm parameters** documented
124 * **Evaluation criteria** public
125 * **Source scoring rules** explicit
126 * **Confidence calculations** explained
127 * **Performance metrics** visible
128 See [[System Performance Metrics>>FactHarbor.Specification.System-Performance-Metrics]] for what we measure.