Wiki source code of Automation
Last modified by Robert Schaub on 2026/02/08 21:22
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | = Automation = | ||
| 2 | |||
| 3 | Automation in FactHarbor amplifies human capability while implementing risk-based oversight. | ||
| 4 | |||
| 5 | This chapter defines: | ||
| 6 | |||
| 7 | * Risk-based publication model | ||
| 8 | * Quality gates for AI-generated content | ||
| 9 | * What must remain human-only | ||
| 10 | * What AI (AKEL) can draft and publish | ||
| 11 | * What can be fully automated | ||
| 12 | * How automation evolves through POC → Beta 0 → Release 1.0 | ||
| 13 | |||
| 14 | == 1. POC v1 (AI-Generated Publication Demonstration) == | ||
| 15 | |||
| 16 | The goal of POC v1 is to validate the automated reasoning capabilities and demonstrate AI-generated content publication. | ||
| 17 | |||
| 18 | === 1.1 Workflow === | ||
| 19 | |||
| 20 | 1. **Input**: User pastes a block of raw text. | ||
| 21 | 1. **Deep Analysis (Background)**: The system autonomously performs the full pipeline **before** displaying the text: | ||
| 22 | |||
| 23 | * Extraction & Normalisation | ||
| 24 | * Scenario & Sub-query generation | ||
| 25 | * Evidence retrieval with **contradiction search** | ||
| 26 | * Quality gate validation | ||
| 27 | * Verdict computation | ||
| 28 | |||
| 29 | 1. **Visualisation (Extraction & Marking)**: The system displays the text with claims extracted and marked. | ||
| 30 | |||
| 31 | * **Verdict-Based Coloring**: The extraction highlights (e.g. Orange/Green) are chosen **according to the computed verdict** for each claim. | ||
| 32 | * **AI-Generated Label**: Clear indication that content is AI-produced | ||
| 33 | |||
| 34 | 1. **Inspection**: User clicks a highlighted claim to see the **Reasoning Trail**, showing exactly which evidence and sub-queries led to that verdict. | ||
| 35 | |||
| 36 | === 1.2 Technical Scope === | ||
| 37 | |||
| 38 | * **AI-Generated Publication**: Content published as Mode 2 (AI-Generated, no prior human review) | ||
| 39 | * **Quality Gates Active**: All automated quality checks enforced | ||
| 40 | * **Contradiction Search Demonstrated**: Shows counter-evidence and reservation detection | ||
| 41 | * **Risk Tier Classification**: POC shows tier assignment (demo purposes) | ||
| 42 | * **No Human Approval Gate**: Demonstrates scalable AI publication | ||
| 43 | * **Structured Sub-Queries**: Logic generated by decomposing claims into the FactHarbor data model | ||
| 44 | |||
| 45 | == 2. Publication Model == | ||
| 46 | |||
| 47 | FactHarbor implements a risk-based publication model with three modes: | ||
| 48 | |||
| 49 | === 2.1 Mode 1: Draft-Only === | ||
| 50 | |||
| 51 | **Mode 1 (Draft-Only)**: Failed quality gates or high-risk content pending expert review. Internal review queue only. | ||
| 52 | |||
| 53 | See [[AKEL Publication Modes>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome#HPublicationModes]] for detailed mode specifications. | ||
| 54 | |||
| 55 | |||
| 56 | === 2.2 Mode 2: AI-Generated (Public) === | ||
| 57 | |||
| 58 | **Mode 2 (AI-Generated, Published)**: Passed all quality gates, risk tier B or C, clearly labeled as AI-generated. Users can request human review. | ||
| 59 | |||
| 60 | See [[AKEL Publication Modes>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome#HPublicationModes]] for detailed requirements. | ||
| 61 | |||
| 62 | |||
| 63 | === 2.3 Mode 3: Human-Reviewed === | ||
| 64 | |||
| 65 | **Mode 3 (Human-Reviewed, Published)**: Validated by human reviewers or experts, highest trust level. Required for Tier A content publication. | ||
| 66 | |||
| 67 | See [[AKEL Publication Modes>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome#HPublicationModes]] for detailed requirements. | ||
| 68 | |||
| 69 | |||
| 70 | == 3. Risk tiers and Automation Levels == | ||
| 71 | |||
| 72 | |||
| 73 | Risk tiers determine review requirements and automation levels. See [[Governance>>FactHarbor.Organisation.Governance.WebHome#HRiskTierPolicyGovernance]] for tier policy governance. | ||
| 74 | |||
| 75 | === 3.1 Tier A (High Risk) === | ||
| 76 | |||
| 77 | * **Domains**: Medical, legal, elections, safety, security | ||
| 78 | * **Automation**: AI can draft, human review required for "Human-Reviewed" status | ||
| 79 | * **AI publication**: Allowed with prominent disclaimers and warnings | ||
| 80 | * **Audit rate**: Recommendation: 30-50% | ||
| 81 | |||
| 82 | === 3.2 Tier B (Medium Risk) === | ||
| 83 | |||
| 84 | * **Domains**: Complex policy, science, causality claims | ||
| 85 | * **Automation**: AI can draft and publish (Mode 2) | ||
| 86 | * **Human review**: Optional, audit-based | ||
| 87 | * **Audit rate**: Recommendation: 10-20% | ||
| 88 | |||
| 89 | === 3.3 Tier C (Low Risk) === | ||
| 90 | |||
| 91 | * **Domains**: Definitions, established facts, historical data | ||
| 92 | * **Automation**: AI publication default | ||
| 93 | * **Human review**: On request or via sampling | ||
| 94 | * **Audit rate**: Recommendation: 5-10% | ||
| 95 | |||
| 96 | == 4. Human-Only Tasks == | ||
| 97 | |||
| 98 | These require human judgment and cannot be automated: | ||
| 99 | |||
| 100 | * **Ethical boundary decisions** (especially medical, political, psychological harm assessment) | ||
| 101 | * **Dispute resolution** between conflicting expert opinions | ||
| 102 | * **Governance policy** setting and enforcement | ||
| 103 | * **Final authority** on Tier A "Human-Reviewed" status | ||
| 104 | * **Audit system oversight** and quality standard definition | ||
| 105 | * **Risk tier policy** adjustments based on societal context | ||
| 106 | |||
| 107 | == 5. AI-Draft with Audit (Semi-Automated) == | ||
| 108 | |||
| 109 | AKEL drafts these; humans validate via sampling audits: | ||
| 110 | |||
| 111 | * **Scenario structures** (definitions, assumptions, context) | ||
| 112 | * **Evaluation methods** and reasoning chains | ||
| 113 | * **Evidence relevance** assessment and ranking | ||
| 114 | * **Reliability scoring** and source evaluation | ||
| 115 | * **Verdict reasoning** with uncertainty quantification | ||
| 116 | * **Contradiction and reservation** identification | ||
| 117 | * **Scenario comparison** explanations | ||
| 118 | * **Public summaries** and accessibility text | ||
| 119 | |||
| 120 | Most Tier B and C content remains in AI-draft status unless: | ||
| 121 | |||
| 122 | * Users request human review | ||
| 123 | * Audits identify errors | ||
| 124 | * High engagement triggers review | ||
| 125 | * Community flags issues | ||
| 126 | |||
| 127 | == 6. Fully Automated Structural Tasks == | ||
| 128 | |||
| 129 | These require no human interpretation: | ||
| 130 | |||
| 131 | * **Claim normalization** (canonical form generation) | ||
| 132 | * **Duplicate detection** (vector embeddings, clustering) | ||
| 133 | * **Evidence metadata extraction** (dates, authors, publication info) | ||
| 134 | * **Basic reliability heuristics** (source reputation scoring) | ||
| 135 | * **Contradiction detection** (conflicting statements across sources) | ||
| 136 | * **Re-evaluation triggers** (new evidence, source updates) | ||
| 137 | * **Layout generation** (diagrams, summaries, UI presentation) | ||
| 138 | * **Federation integrity checks** (cross-node data validation) | ||
| 139 | |||
| 140 | == 7. Quality Gates (Automated) == | ||
| 141 | |||
| 142 | |||
| 143 | Before AI-generated publication (Mode 2), content must pass four automated quality gates: | ||
| 144 | |||
| 145 | 1. **Source Quality** - Primary sources verified, citations complete | ||
| 146 | 2. **Contradiction Search** (MANDATORY) - Counter-evidence actively sought | ||
| 147 | 3. **Uncertainty Quantification** - Confidence scores calculated | ||
| 148 | 4. **Structural Validation** - Required fields present, format valid | ||
| 149 | |||
| 150 | See [[AKEL Quality Gates>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome#HQualityGates]] for complete gate specifications. | ||
| 151 | |||
| 152 | |||
| 153 | == 8. Audit System == | ||
| 154 | |||
| 155 | Instead of reviewing all AI output, systematic sampling audits ensure quality: | ||
| 156 | |||
| 157 | === 8.1 Stratified Sampling === | ||
| 158 | |||
| 159 | * Risk tier (A > B > C sampling rates) | ||
| 160 | * Confidence scores (low confidence → more audits) | ||
| 161 | * Traffic/engagement (popular content audited more) | ||
| 162 | * Novelty (new topics/claim types prioritized) | ||
| 163 | * User flags and disagreement signals | ||
| 164 | |||
| 165 | === 8.2 Continuous Improvement Loop === | ||
| 166 | |||
| 167 | Audit findings improve: | ||
| 168 | |||
| 169 | * Query templates | ||
| 170 | * Source reliability weights | ||
| 171 | * Contradiction detection algorithms | ||
| 172 | * Risk tier assignment rules | ||
| 173 | * Bubble detection heuristics | ||
| 174 | |||
| 175 | === 8.3 Transparency === | ||
| 176 | |||
| 177 | * Audit statistics published | ||
| 178 | * Accuracy rates by tier reported | ||
| 179 | * System improvements documented | ||
| 180 | |||
| 181 | == 9. Automation Roadmap == | ||
| 182 | |||
| 183 | Automation capabilities increase with system maturity while maintaining quality oversight. | ||
| 184 | |||
| 185 | === 9.1 POC (Current Focus) === | ||
| 186 | |||
| 187 | **Automated:** | ||
| 188 | |||
| 189 | * Claim normalization | ||
| 190 | * Scenario template generation | ||
| 191 | * Evidence metadata extraction | ||
| 192 | * Simple verdict drafts | ||
| 193 | * **AI-generated publication** (Mode 2, with quality gates) | ||
| 194 | * **Contradiction search** | ||
| 195 | * **Risk tier assignment** | ||
| 196 | |||
| 197 | **Human:** | ||
| 198 | |||
| 199 | * High-risk content validation (Tier A) | ||
| 200 | * Sampling audits across all tiers | ||
| 201 | * Quality standard refinement | ||
| 202 | * Governance decisions | ||
| 203 | |||
| 204 | === 9.2 Beta 0 (Enhanced Automation) === | ||
| 205 | |||
| 206 | **Automated:** | ||
| 207 | |||
| 208 | * Detailed scenario generation | ||
| 209 | * Advanced evidence reliability scoring | ||
| 210 | * Cross-scenario comparisons | ||
| 211 | * Multi-source contradiction detection | ||
| 212 | * Internal Truth Landscape generation | ||
| 213 | * **Increased AI-draft coverage** (more Tier B content) | ||
| 214 | |||
| 215 | **Human:** | ||
| 216 | |||
| 217 | * Tier A final approval | ||
| 218 | * Audit sampling (continued) | ||
| 219 | * Expert validation of complex domains | ||
| 220 | * Quality improvement oversight | ||
| 221 | |||
| 222 | === 9.3 Release 1.0 (High Automation) === | ||
| 223 | |||
| 224 | **Automated:** | ||
| 225 | |||
| 226 | * Full scenario generation (comprehensive) | ||
| 227 | * Bayesian verdict scoring across scenarios | ||
| 228 | * Multi-scenario summary generation | ||
| 229 | * Anomaly detection across federated nodes | ||
| 230 | * AKEL-assisted cross-node synchronization | ||
| 231 | * **Most Tier B and all Tier C** auto-published | ||
| 232 | |||
| 233 | **Human:** | ||
| 234 | |||
| 235 | * Tier A oversight (still required) | ||
| 236 | * Strategic audits (lower sampling rates, higher value) | ||
| 237 | * Ethical decisions and policy | ||
| 238 | * Conflict resolution | ||
| 239 | |||
| 240 | == 10. Automation Levels Diagram == | ||
| 241 | |||
| 242 | {{include reference="Archive.FactHarbor 2026\.01\.20.Specification.Diagrams.Automation Level.WebHome"/}} | ||
| 243 | |||
| 244 | |||
| 245 | == 11. Automation Roadmap Diagram == | ||
| 246 | |||
| 247 | {{include reference="Archive.FactHarbor 2026\.01\.20.Specification.Diagrams.Automation Roadmap.WebHome"/}} | ||
| 248 | |||
| 249 | |||
| 250 | == 12. Manual vs Automated Matrix == | ||
| 251 | |||
| 252 | {{include reference="Archive.FactHarbor 2026\.01\.20.Specification.Diagrams.Manual vs Automated matrix.WebHome"/}} | ||
| 253 | |||
| 254 | |||
| 255 | == 13. Related Pages == | ||
| 256 | |||
| 257 | * [[AKEL (AI Knowledge Extraction Layer)>>Archive.FactHarbor 2026\.01\.20.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] | ||
| 258 | * [[Requirements (Roles)>>Archive.FactHarbor 2026\.01\.20.Specification.Requirements.WebHome]] | ||
| 259 | * [[Workflows>>Archive.FactHarbor 2026\.01\.20.Specification.Workflows.WebHome]] | ||
| 260 | * [[Governance>>Archive.FactHarbor.Organisation.Governance.WebHome]] |