Skip to Content

Changes for page Automation Philosophy

Last modified by Robert Schaub on 2026/02/08 08:29
From version 1.1
edited by Robert Schaub
on 2026/01/20 20:44
Change comment: Imported from XAR
To version 1.3
edited by Robert Schaub
on 2026/02/08 08:29
Change comment: Renamed back-links.

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -FactHarbor.Organisation.WebHome
1 +WebHome
Content
... ... @@ -1,8 +1,12 @@
1 1  = Automation Philosophy =
2 +
2 2  **Core Principle**: AKEL is primary. Humans monitor, improve, and handle exceptions.
4 +
3 3  == 1. The Principle ==
6 +
4 4  **FactHarbor is AI-first, not AI-assisted.**
5 5  This is not:
9 +
6 6  * ❌ "AI helps humans make better decisions"
7 7  * ❌ "Humans review AI recommendations"
8 8  * ❌ "AI drafts, humans approve"
... ... @@ -10,10 +10,14 @@
10 10  * ✅ "AI makes decisions, humans improve the AI"
11 11  * ✅ "Humans monitor metrics, not individual outputs"
12 12  * ✅ "Fix the system, not the data"
17 +
13 13  == 2. Why This Matters ==
19 +
14 14  === 2.1 Scalability ===
21 +
15 15  **Human review doesn't scale**:
16 -* 1 person can review ~100 claims/day carefully
23 +
24 +* 1 person can review 100 claims/day carefully
17 17  * FactHarbor aims for millions of claims
18 18  * Would need 10,000+ reviewers
19 19  * Impossible to maintain consistency
... ... @@ -22,8 +22,11 @@
22 22  * Cost per claim approaches zero at scale
23 23  * Quality improves with more data
24 24  * 24/7 availability
33 +
25 25  === 2.2 Consistency ===
35 +
26 26  **Human judgment varies**:
37 +
27 27  * Different reviewers apply criteria differently
28 28  * Same reviewer makes different decisions on different days
29 29  * Influenced by fatigue, mood, recent examples
... ... @@ -33,8 +33,11 @@
33 33  * Rules applied uniformly
34 34  * No mood, fatigue, or bias
35 35  * Predictable behavior
47 +
36 36  === 2.3 Transparency ===
49 +
37 37  **Human judgment is opaque**:
51 +
38 38  * "I just know" - hard to explain
39 39  * Expertise in human head
40 40  * Can't audit thought process
... ... @@ -45,8 +45,11 @@
45 45  * Decision logic is explicit
46 46  * Changes are tracked
47 47  * Can test "what if" scenarios
62 +
48 48  === 2.4 Improvement ===
64 +
49 49  **Improving human judgment**:
66 +
50 50  * Train each person individually
51 51  * Hope training transfers consistently
52 52  * Subjective quality assessment
... ... @@ -56,10 +56,15 @@
56 56  * Test on historical data before deploying
57 57  * Measure improvement objectively
58 58  * Rapid iteration (deploy multiple times per week)
76 +
59 59  == 3. The Human Role ==
78 +
60 60  Humans in FactHarbor are **system architects**, not **content judges**.
80 +
61 61  === 3.1 What Humans Do ===
82 +
62 62  **Monitor** system performance:
84 +
63 63  * Watch dashboards showing aggregate metrics
64 64  * Identify when metrics fall outside acceptable ranges
65 65  * Spot patterns in errors or edge cases
... ... @@ -80,8 +80,11 @@
80 80  * Define acceptable performance ranges
81 81  * Allocate resources
82 82  * Make strategic decisions
105 +
83 83  === 3.2 What Humans Do NOT Do ===
107 +
84 84  **Review** individual claims for correctness:
109 +
85 85  * ❌ "Let me check if this verdict is right"
86 86  * ❌ "I'll approve these before publication"
87 87  * ❌ "This needs human judgment"
... ... @@ -94,50 +94,92 @@
94 94  * ❌ "High-risk claims need review"
95 95  * ❌ "Quality assurance before publication"
96 96  **Why not?** Because this defeats the purpose and doesn't scale.
122 +
97 97  == 4. When Humans Intervene ==
124 +
98 98  === 4.1 Legitimate Interventions ===
126 +
99 99  **Humans should intervene when**:
100 -==== AKEL explicitly flags for review ====:
128 +
129 +==== AKEL explicitly flags for review ====
130 +
131 +:
132 +
101 101  * AKEL's confidence is too low
102 102  * Detected potential manipulation
103 103  * Unusual pattern requiring human judgment
104 104  * Clear policy: "Flag if confidence <X"
105 -==== System metrics show problems ====:
137 +
138 +==== System metrics show problems ====
139 +
140 +:
141 +
106 106  * Processing time suddenly increases
107 107  * Error rate jumps
108 108  * Confidence distribution shifts
109 109  * User feedback becomes negative
110 -==== Systematic bias detected ====:
146 +
147 +==== Systematic bias detected ====
148 +
149 +:
150 +
111 111  * Metrics show pattern of unfairness
112 112  * Particular domains consistently scored oddly
113 113  * Source types systematically mis-rated
114 -==== Legal/safety emergency ====:
154 +
155 +==== Legal/safety emergency ====
156 +
157 +:
158 +
115 115  * Legal takedown required
116 116  * Imminent harm to individuals
117 117  * Security breach
118 118  * Compliance violation
163 +
119 119  === 4.2 Illegitimate Interventions ===
165 +
120 120  **Humans should NOT intervene for**:
121 -==== "I disagree with this verdict" ====:
167 +
168 +==== "I disagree with this verdict" ====
169 +
170 +:
171 +
122 122  * Problem: Your opinion vs AKEL's analysis
123 123  * Solution: If AKEL is systematically wrong, fix the algorithm
124 124  * Action: Gather data, propose algorithm improvement
125 -==== "This source should rank higher" ====:
175 +
176 +==== "This source should rank higher" ====
177 +
178 +:
179 +
126 126  * Problem: Subjective preference
127 127  * Solution: Fix scoring rules systematically
128 128  * Action: Analyze why AKEL scored it lower, adjust scoring algorithm if justified
129 -==== "Manual quality gate" ====:
183 +
184 +==== "Manual quality gate" ====
185 +
186 +:
187 +
130 130  * Problem: Creates bottleneck, defeats automation
131 131  * Solution: Improve AKEL's quality to not need human gate
132 132  * Action: Set quality thresholds in algorithm, not human review
133 -==== "I know better than the algorithm" ====:
191 +
192 +==== "I know better than the algorithm" ====
193 +
194 +:
195 +
134 134  * Problem: Doesn't scale, introduces bias
135 135  * Solution: Teach the algorithm what you know
136 136  * Action: Update training data, adjust parameters, document expertise in policy
199 +
137 137  == 5. Fix the System, Not the Data ==
201 +
138 138  **Fundamental principle**: When AKEL makes mistakes, improve AKEL, don't fix individual outputs.
203 +
139 139  === 5.1 Why? ===
205 +
140 140  **Fixing individual outputs**:
207 +
141 141  * Doesn't prevent future similar errors
142 142  * Doesn't scale (too many outputs)
143 143  * Creates inconsistency
... ... @@ -147,44 +147,73 @@
147 147  * Scales automatically
148 148  * Maintains consistency
149 149  * Surfaces and resolves root causes
217 +
150 150  === 5.2 Process ===
219 +
151 151  **When you see a "wrong" AKEL decision**:
152 -==== Document it ====:
221 +
222 +==== Document it ====
223 +
224 +:
225 +
153 153  * What was the claim?
154 154  * What did AKEL decide?
155 155  * What should it have decided?
156 156  * Why do you think it's wrong?
157 -==== Investigate ====:
230 +
231 +==== Investigate ====
232 +
233 +:
234 +
158 158  * Is this a one-off, or a pattern?
159 159  * Check similar claims - same issue?
160 160  * What caused AKEL to decide this way?
161 161  * What rule/parameter needs changing?
162 -==== Propose systematic fix ====:
239 +
240 +==== Propose systematic fix ====
241 +
242 +:
243 +
163 163  * Algorithm change?
164 164  * Policy clarification?
165 165  * Training data update?
166 166  * Parameter adjustment?
167 -==== Test the fix ====:
248 +
249 +==== Test the fix ====
250 +
251 +:
252 +
168 168  * Run on historical data
169 169  * Does it fix this case?
170 170  * Does it break other cases?
171 171  * What's the overall impact?
172 -==== Deploy and monitor ====:
257 +
258 +==== Deploy and monitor ====
259 +
260 +:
261 +
173 173  * Gradual rollout
174 174  * Watch metrics closely
175 175  * Gather feedback
176 176  * Iterate if needed
266 +
177 177  == 6. Balancing Automation and Human Values ==
268 +
178 178  === 6.1 Algorithms Embody Values ===
270 +
179 179  **Important**: Automation doesn't mean "value-free"
180 180  **Algorithms encode human values**:
273 +
181 181  * Which evidence types matter most?
182 182  * How much weight to peer review?
183 183  * What constitutes "high risk"?
184 184  * When to flag for human review?
185 185  **These are human choices**, implemented in code.
279 +
186 186  === 6.2 Human Governance of Automation ===
281 +
187 187  **Humans set**:
283 +
188 188  * ✅ Risk tier policies (what's high-risk?)
189 189  * ✅ Evidence weighting (what types of evidence matter?)
190 190  * ✅ Source scoring criteria (what makes a source credible?)
... ... @@ -195,15 +195,21 @@
195 195  * ✅ At scale
196 196  * ✅ Transparently
197 197  * ✅ Without subjective variation
294 +
198 198  === 6.3 Continuous Value Alignment ===
296 +
199 199  **Ongoing process**:
298 +
200 200  * Monitor: Are outcomes aligned with values?
201 201  * Analyze: Where do values and outcomes diverge?
202 202  * Adjust: Update policies or algorithms
203 203  * Test: Validate alignment improved
204 204  * Repeat: Values alignment is never "done"
304 +
205 205  == 7. Cultural Implications ==
306 +
206 206  === 7.1 Mindset Shift Required ===
308 +
207 207  **From**: "I'm a content expert who reviews claims"
208 208  **To**: "I'm a system architect who improves algorithms"
209 209  **From**: "Good work means catching errors"
... ... @@ -210,8 +210,11 @@
210 210  **To**: "Good work means preventing errors systematically"
211 211  **From**: "I trust my judgment"
212 212  **To**: "I make my judgment codifiable and testable"
315 +
213 213  === 7.2 New Skills Needed ===
317 +
214 214  **Less emphasis on**:
319 +
215 215  * Individual content judgment
216 216  * Manual review skills
217 217  * Subjective expertise application
... ... @@ -221,8 +221,11 @@
221 221  * Policy formulation
222 222  * Testing and validation
223 223  * Documentation and knowledge transfer
329 +
224 224  === 7.3 Job Satisfaction Sources ===
331 +
225 225  **Satisfaction comes from**:
333 +
226 226  * ✅ Seeing metrics improve after your changes
227 227  * ✅ Building systems that help millions
228 228  * ✅ Solving systematic problems elegantly
... ... @@ -233,16 +233,23 @@
233 233  * ❌ Manual review and approval
234 234  * ❌ Gatekeeping
235 235  * ❌ Individual heroics
344 +
236 236  == 8. Trust and Automation ==
346 +
237 237  === 8.1 Building Trust in AKEL ===
348 +
238 238  **Users trust AKEL when**:
350 +
239 239  * Transparent: How decisions are made is documented
240 240  * Consistent: Same inputs → same outputs
241 241  * Measurable: Performance metrics are public
242 242  * Improvable: Clear process for getting better
243 243  * Governed: Human oversight of policies, not outputs
356 +
244 244  === 8.2 What Trust Does NOT Mean ===
358 +
245 245  **Trust in automation ≠**:
360 +
246 246  * ❌ "Never makes mistakes" (impossible)
247 247  * ❌ "Better than any human could ever be" (unnecessary)
248 248  * ❌ "Beyond human understanding" (must be understandable)
... ... @@ -252,9 +252,13 @@
252 252  * ✅ Mistakes can be detected and fixed systematically
253 253  * ✅ Performance continuously improves
254 254  * ✅ Decision process is transparent and auditable
370 +
255 255  == 9. Edge Cases and Exceptions ==
372 +
256 256  === 9.1 Some Things Still Need Humans ===
374 +
257 257  **AKEL flags for human review when**:
376 +
258 258  * Confidence below threshold
259 259  * Detected manipulation attempt
260 260  * Novel situation not seen before
... ... @@ -262,8 +262,11 @@
262 262  **Humans handle**:
263 263  * Items AKEL flags
264 264  * Not routine review
384 +
265 265  === 9.2 Learning from Exceptions ===
386 +
266 266  **When humans handle an exception**:
388 +
267 267  1. Resolve the immediate case
268 268  2. Document: What made this exceptional?
269 269  3. Analyze: Could AKEL have handled this?
... ... @@ -271,9 +271,11 @@
271 271  5. Monitor: Did exception rate decrease?
272 272  **Goal**: Fewer exceptions over time as AKEL learns.
273 273  ---
274 -**Remember**: AKEL is primary. You improve the SYSTEM. The system improves the CONTENT.
396 +**Remember**: AKEL is primary. You improve the SYSTEM. The system improves the CONTENT.--
397 +
275 275  == 10. Related Pages ==
276 -* [[Governance>>FactHarbor.Organisation.Governance.WebHome]] - How AKEL is governed
399 +
400 +* [[Governance>>Archive.FactHarbor 2026\.02\.08.Organisation.Governance.WebHome]] - How AKEL is governed
277 277  * [[Contributor Processes>>FactHarbor.Organisation.Contributor-Processes]] - How to improve the system
278 278  * [[Organisational Model>>FactHarbor.Organisation.Organisational-Model]] - Team structure and roles
279 279  * [[System Performance Metrics>>FactHarbor.Specification.System-Performance-Metrics]] - What we monitor