Skip to Content

Wiki source code of Automation Philosophy

Last modified by Robert Schaub on 2025/12/23 18:00
Show last authors
1 = Automation Philosophy =
2 **Core Principle**: AKEL is primary. Humans monitor, improve, and handle exceptions.
3 == 1. The Principle ==
4 **FactHarbor is AI-first, not AI-assisted.**
5 This is not:
6 * ❌ "AI helps humans make better decisions"
7 * ❌ "Humans review AI recommendations"
8 * ❌ "AI drafts, humans approve"
9 This is:
10 * ✅ "AI makes decisions, humans improve the AI"
11 * ✅ "Humans monitor metrics, not individual outputs"
12 * ✅ "Fix the system, not the data"
13 == 2. Why This Matters ==
14 === 2.1 Scalability ===
15 **Human review doesn't scale**:
16 * 1 person can review ~100 claims/day carefully
17 * FactHarbor aims for millions of claims
18 * Would need 10,000+ reviewers
19 * Impossible to maintain consistency
20 **Algorithmic processing scales**:
21 * AKEL processes 1 claim or 1 million claims with same consistency
22 * Cost per claim approaches zero at scale
23 * Quality improves with more data
24 * 24/7 availability
25 === 2.2 Consistency ===
26 **Human judgment varies**:
27 * Different reviewers apply criteria differently
28 * Same reviewer makes different decisions on different days
29 * Influenced by fatigue, mood, recent examples
30 * Unconscious biases affect decisions
31 **Algorithmic processing is consistent**:
32 * Same input → same output, always
33 * Rules applied uniformly
34 * No mood, fatigue, or bias
35 * Predictable behavior
36 === 2.3 Transparency ===
37 **Human judgment is opaque**:
38 * "I just know" - hard to explain
39 * Expertise in human head
40 * Can't audit thought process
41 * Difficult to improve systematically
42 **Algorithmic processing is transparent**:
43 * Code can be audited
44 * Parameters are documented
45 * Decision logic is explicit
46 * Changes are tracked
47 * Can test "what if" scenarios
48 === 2.4 Improvement ===
49 **Improving human judgment**:
50 * Train each person individually
51 * Hope training transfers consistently
52 * Subjective quality assessment
53 * Slow iteration
54 **Improving algorithms**:
55 * Change code once, affects all decisions
56 * Test on historical data before deploying
57 * Measure improvement objectively
58 * Rapid iteration (deploy multiple times per week)
59 == 3. The Human Role ==
60 Humans in FactHarbor are **system architects**, not **content judges**.
61 === 3.1 What Humans Do ===
62 **Monitor** system performance:
63 * Watch dashboards showing aggregate metrics
64 * Identify when metrics fall outside acceptable ranges
65 * Spot patterns in errors or edge cases
66 * Track user feedback trends
67 **Improve** algorithms and policies:
68 * Analyze systematic errors
69 * Propose algorithm improvements
70 * Update policies based on learning
71 * Test changes before deployment
72 * Document learnings
73 **Handle** exceptions:
74 * Items AKEL explicitly flags for review
75 * System gaming attempts
76 * Abuse and harassment
77 * Legal/safety emergencies
78 **Govern** the system:
79 * Set risk tier policies
80 * Define acceptable performance ranges
81 * Allocate resources
82 * Make strategic decisions
83 === 3.2 What Humans Do NOT Do ===
84 **Review** individual claims for correctness:
85 * ❌ "Let me check if this verdict is right"
86 * ❌ "I'll approve these before publication"
87 * ❌ "This needs human judgment"
88 **Override** AKEL decisions routinely:
89 * ❌ "AKEL got this wrong, I'll fix it"
90 * ❌ "I disagree with this verdict"
91 * ❌ "This source should be rated higher"
92 **Act as** approval gates:
93 * ❌ "All claims must be human-approved"
94 * ❌ "High-risk claims need review"
95 * ❌ "Quality assurance before publication"
96 **Why not?** Because this defeats the purpose and doesn't scale.
97 == 4. When Humans Intervene ==
98 === 4.1 Legitimate Interventions ===
99 **Humans should intervene when**:
100 ==== AKEL explicitly flags for review ====:
101 * AKEL's confidence is too low
102 * Detected potential manipulation
103 * Unusual pattern requiring human judgment
104 * Clear policy: "Flag if confidence <X"
105 ==== System metrics show problems ====:
106 * Processing time suddenly increases
107 * Error rate jumps
108 * Confidence distribution shifts
109 * User feedback becomes negative
110 ==== Systematic bias detected ====:
111 * Metrics show pattern of unfairness
112 * Particular domains consistently scored oddly
113 * Source types systematically mis-rated
114 ==== Legal/safety emergency ====:
115 * Legal takedown required
116 * Imminent harm to individuals
117 * Security breach
118 * Compliance violation
119 === 4.2 Illegitimate Interventions ===
120 **Humans should NOT intervene for**:
121 ==== "I disagree with this verdict" ====:
122 * Problem: Your opinion vs AKEL's analysis
123 * Solution: If AKEL is systematically wrong, fix the algorithm
124 * Action: Gather data, propose algorithm improvement
125 ==== "This source should rank higher" ====:
126 * Problem: Subjective preference
127 * Solution: Fix scoring rules systematically
128 * Action: Analyze why AKEL scored it lower, adjust scoring algorithm if justified
129 ==== "Manual quality gate" ====:
130 * Problem: Creates bottleneck, defeats automation
131 * Solution: Improve AKEL's quality to not need human gate
132 * Action: Set quality thresholds in algorithm, not human review
133 ==== "I know better than the algorithm" ====:
134 * Problem: Doesn't scale, introduces bias
135 * Solution: Teach the algorithm what you know
136 * Action: Update training data, adjust parameters, document expertise in policy
137 == 5. Fix the System, Not the Data ==
138 **Fundamental principle**: When AKEL makes mistakes, improve AKEL, don't fix individual outputs.
139 === 5.1 Why? ===
140 **Fixing individual outputs**:
141 * Doesn't prevent future similar errors
142 * Doesn't scale (too many outputs)
143 * Creates inconsistency
144 * Hides systematic problems
145 **Fixing the system**:
146 * Prevents future similar errors
147 * Scales automatically
148 * Maintains consistency
149 * Surfaces and resolves root causes
150 === 5.2 Process ===
151 **When you see a "wrong" AKEL decision**:
152 ==== Document it ====:
153 * What was the claim?
154 * What did AKEL decide?
155 * What should it have decided?
156 * Why do you think it's wrong?
157 ==== Investigate ====:
158 * Is this a one-off, or a pattern?
159 * Check similar claims - same issue?
160 * What caused AKEL to decide this way?
161 * What rule/parameter needs changing?
162 ==== Propose systematic fix ====:
163 * Algorithm change?
164 * Policy clarification?
165 * Training data update?
166 * Parameter adjustment?
167 ==== Test the fix ====:
168 * Run on historical data
169 * Does it fix this case?
170 * Does it break other cases?
171 * What's the overall impact?
172 ==== Deploy and monitor ====:
173 * Gradual rollout
174 * Watch metrics closely
175 * Gather feedback
176 * Iterate if needed
177 == 6. Balancing Automation and Human Values ==
178 === 6.1 Algorithms Embody Values ===
179 **Important**: Automation doesn't mean "value-free"
180 **Algorithms encode human values**:
181 * Which evidence types matter most?
182 * How much weight to peer review?
183 * What constitutes "high risk"?
184 * When to flag for human review?
185 **These are human choices**, implemented in code.
186 === 6.2 Human Governance of Automation ===
187 **Humans set**:
188 * ✅ Risk tier policies (what's high-risk?)
189 * ✅ Evidence weighting (what types of evidence matter?)
190 * ✅ Source scoring criteria (what makes a source credible?)
191 * ✅ Moderation policies (what's abuse?)
192 * ✅ Bias mitigation strategies
193 **AKEL applies**:
194 * ✅ These policies consistently
195 * ✅ At scale
196 * ✅ Transparently
197 * ✅ Without subjective variation
198 === 6.3 Continuous Value Alignment ===
199 **Ongoing process**:
200 * Monitor: Are outcomes aligned with values?
201 * Analyze: Where do values and outcomes diverge?
202 * Adjust: Update policies or algorithms
203 * Test: Validate alignment improved
204 * Repeat: Values alignment is never "done"
205 == 7. Cultural Implications ==
206 === 7.1 Mindset Shift Required ===
207 **From**: "I'm a content expert who reviews claims"
208 **To**: "I'm a system architect who improves algorithms"
209 **From**: "Good work means catching errors"
210 **To**: "Good work means preventing errors systematically"
211 **From**: "I trust my judgment"
212 **To**: "I make my judgment codifiable and testable"
213 === 7.2 New Skills Needed ===
214 **Less emphasis on**:
215 * Individual content judgment
216 * Manual review skills
217 * Subjective expertise application
218 **More emphasis on**:
219 * Data analysis and metrics interpretation
220 * Algorithm design and optimization
221 * Policy formulation
222 * Testing and validation
223 * Documentation and knowledge transfer
224 === 7.3 Job Satisfaction Sources ===
225 **Satisfaction comes from**:
226 * ✅ Seeing metrics improve after your changes
227 * ✅ Building systems that help millions
228 * ✅ Solving systematic problems elegantly
229 * ✅ Continuous learning and improvement
230 * ✅ Transparent, auditable impact
231 **Not from**:
232 * ❌ Being the expert who makes final call
233 * ❌ Manual review and approval
234 * ❌ Gatekeeping
235 * ❌ Individual heroics
236 == 8. Trust and Automation ==
237 === 8.1 Building Trust in AKEL ===
238 **Users trust AKEL when**:
239 * Transparent: How decisions are made is documented
240 * Consistent: Same inputs → same outputs
241 * Measurable: Performance metrics are public
242 * Improvable: Clear process for getting better
243 * Governed: Human oversight of policies, not outputs
244 === 8.2 What Trust Does NOT Mean ===
245 **Trust in automation ≠**:
246 * ❌ "Never makes mistakes" (impossible)
247 * ❌ "Better than any human could ever be" (unnecessary)
248 * ❌ "Beyond human understanding" (must be understandable)
249 * ❌ "Set it and forget it" (requires continuous improvement)
250 **Trust in automation =**:
251 * ✅ Mistakes are systematic, not random
252 * ✅ Mistakes can be detected and fixed systematically
253 * ✅ Performance continuously improves
254 * ✅ Decision process is transparent and auditable
255 == 9. Edge Cases and Exceptions ==
256 === 9.1 Some Things Still Need Humans ===
257 **AKEL flags for human review when**:
258 * Confidence below threshold
259 * Detected manipulation attempt
260 * Novel situation not seen before
261 * Explicit policy requires human judgment
262 **Humans handle**:
263 * Items AKEL flags
264 * Not routine review
265 === 9.2 Learning from Exceptions ===
266 **When humans handle an exception**:
267 1. Resolve the immediate case
268 2. Document: What made this exceptional?
269 3. Analyze: Could AKEL have handled this?
270 4. Improve: Update AKEL to handle similar cases
271 5. Monitor: Did exception rate decrease?
272 **Goal**: Fewer exceptions over time as AKEL learns.
273 ---
274 **Remember**: AKEL is primary. You improve the SYSTEM. The system improves the CONTENT.
275 == 10. Related Pages ==
276 * [[Governance>>FactHarbor.Organisation.Governance.WebHome]] - How AKEL is governed
277 * [[Contributor Processes>>FactHarbor.Organisation.Contributor-Processes]] - How to improve the system
278 * [[Organisational Model>>FactHarbor.Organisation.Organisational-Model]] - Team structure and roles
279 * [[System Performance Metrics>>FactHarbor.Specification.System-Performance-Metrics]] - What we monitor