Changes for page Automation
Last modified by Robert Schaub on 2025/12/24 20:34
From version 7.2
edited by Robert Schaub
on 2025/12/16 20:25
on 2025/12/16 20:25
Change comment:
Update document after refactoring.
Summary
-
Page properties (2 modified, 0 added, 0 removed)
Details
- Page properties
-
- Parent
-
... ... @@ -1,1 +1,1 @@ 1 -FactHarbor.Archive.FactHarbor.Specification.WebHome 1 +FactHarbor.Archive.FactHarbor V0\.9\.18.Specification.WebHome - Content
-
... ... @@ -3,6 +3,7 @@ 3 3 Automation in FactHarbor amplifies human capability while implementing risk-based oversight. 4 4 5 5 This chapter defines: 6 + 6 6 * Risk-based publication model 7 7 * Quality gates for AI-generated content 8 8 * What must remain human-only ... ... @@ -18,14 +18,18 @@ 18 18 19 19 1. **Input**: User pastes a block of raw text. 20 20 1. **Deep Analysis (Background)**: The system autonomously performs the full pipeline **before** displaying the text: 22 + 21 21 * Extraction & Normalisation 22 22 * Scenario & Sub-query generation 23 23 * Evidence retrieval with **contradiction search** 24 24 * Quality gate validation 25 25 * Verdict computation 28 + 26 26 1. **Visualisation (Extraction & Marking)**: The system displays the text with claims extracted and marked. 30 + 27 27 * **Verdict-Based Coloring**: The extraction highlights (e.g. Orange/Green) are chosen **according to the computed verdict** for each claim. 28 28 * **AI-Generated Label**: Clear indication that content is AI-produced 33 + 29 29 1. **Inspection**: User clicks a highlighted claim to see the **Reasoning Trail**, showing exactly which evidence and sub-queries led to that verdict. 30 30 31 31 === Technical Scope === ... ... @@ -44,11 +44,13 @@ 44 44 FactHarbor implements a risk-based publication model with three modes: 45 45 46 46 === Mode 1: Draft-Only === 52 + 47 47 * Failed quality gates 48 48 * High-risk content pending expert review 49 49 * Internal review queue only 50 50 51 51 === Mode 2: AI-Generated (Public) === 58 + 52 52 * Passed all quality gates 53 53 * Risk tier B or C 54 54 * Clear AI-generated labeling ... ... @@ -55,6 +55,7 @@ 55 55 * Users can request human review 56 56 57 57 === Mode 3: Human-Reviewed === 65 + 58 58 * Validated by human reviewers/experts 59 59 * "Human-Reviewed" status badge 60 60 * Required for Tier A content publication ... ... @@ -66,6 +66,7 @@ 66 66 == Risk Tiers and Automation Levels == 67 67 68 68 === Tier A (High Risk) === 77 + 69 69 * **Domains**: Medical, legal, elections, safety, security 70 70 * **Automation**: AI can draft, human review required for "Human-Reviewed" status 71 71 * **AI publication**: Allowed with prominent disclaimers and warnings ... ... @@ -72,6 +72,7 @@ 72 72 * **Audit rate**: Recommendation: 30-50% 73 73 74 74 === Tier B (Medium Risk) === 84 + 75 75 * **Domains**: Complex policy, science, causality claims 76 76 * **Automation**: AI can draft and publish (Mode 2) 77 77 * **Human review**: Optional, audit-based ... ... @@ -78,6 +78,7 @@ 78 78 * **Audit rate**: Recommendation: 10-20% 79 79 80 80 === Tier C (Low Risk) === 91 + 81 81 * **Domains**: Definitions, established facts, historical data 82 82 * **Automation**: AI publication default 83 83 * **Human review**: On request or via sampling ... ... @@ -112,6 +112,7 @@ 112 112 * **Public summaries** and accessibility text 113 113 114 114 Most Tier B and C content remains in AI-draft status unless: 126 + 115 115 * Users request human review 116 116 * Audits identify errors 117 117 * High engagement triggers review ... ... @@ -139,26 +139,30 @@ 139 139 Before AI-draft publication (Mode 2), content must pass: 140 140 141 141 1. **Source Quality Gate** 142 - * Primary sources verified 143 - * Citations complete and accessible 144 - * Source reliability scored 145 145 155 +* Primary sources verified 156 +* Citations complete and accessible 157 +* Source reliability scored 158 + 146 146 2. **Contradiction Search Gate** (MANDATORY) 147 - * Counter-evidence actively sought 148 - * Reservations and limitations identified 149 - * Bubble detection (echo chambers, conspiracy theories) 150 - * Diverse perspective verification 151 151 161 +* Counter-evidence actively sought 162 +* Reservations and limitations identified 163 +* Bubble detection (echo chambers, conspiracy theories) 164 +* Diverse perspective verification 165 + 152 152 3. **Uncertainty Quantification Gate** 153 - * Confidence scores calculated 154 - * Limitations stated 155 - * Data gaps disclosed 156 156 168 +* Confidence scores calculated 169 +* Limitations stated 170 +* Data gaps disclosed 171 + 157 157 4. **Structural Integrity Gate** 158 - * No hallucinations detected 159 - * Logic chain valid 160 - * References verifiable 161 161 174 +* No hallucinations detected 175 +* Logic chain valid 176 +* References verifiable 177 + 162 162 See [[AKEL page>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed quality gate specifications. 163 163 164 164 ---- ... ... @@ -168,6 +168,7 @@ 168 168 Instead of reviewing all AI output, systematic sampling audits ensure quality: 169 169 170 170 === Stratified Sampling === 187 + 171 171 * Risk tier (A > B > C sampling rates) 172 172 * Confidence scores (low confidence → more audits) 173 173 * Traffic/engagement (popular content audited more) ... ... @@ -175,7 +175,9 @@ 175 175 * User flags and disagreement signals 176 176 177 177 === Continuous Improvement Loop === 195 + 178 178 Audit findings improve: 197 + 179 179 * Query templates 180 180 * Source reliability weights 181 181 * Contradiction detection algorithms ... ... @@ -183,6 +183,7 @@ 183 183 * Bubble detection heuristics 184 184 185 185 === Transparency === 205 + 186 186 * Audit statistics published 187 187 * Accuracy rates by tier reported 188 188 * System improvements documented ... ... @@ -196,6 +196,7 @@ 196 196 === POC (Current Focus) === 197 197 198 198 **Automated:** 219 + 199 199 * Claim normalization 200 200 * Scenario template generation 201 201 * Evidence metadata extraction ... ... @@ -205,6 +205,7 @@ 205 205 * **Risk tier assignment** 206 206 207 207 **Human:** 229 + 208 208 * High-risk content validation (Tier A) 209 209 * Sampling audits across all tiers 210 210 * Quality standard refinement ... ... @@ -213,6 +213,7 @@ 213 213 === Beta 0 (Enhanced Automation) === 214 214 215 215 **Automated:** 238 + 216 216 * Detailed scenario generation 217 217 * Advanced evidence reliability scoring 218 218 * Cross-scenario comparisons ... ... @@ -221,6 +221,7 @@ 221 221 * **Increased AI-draft coverage** (more Tier B content) 222 222 223 223 **Human:** 247 + 224 224 * Tier A final approval 225 225 * Audit sampling (continued) 226 226 * Expert validation of complex domains ... ... @@ -229,6 +229,7 @@ 229 229 === Release 1.0 (High Automation) === 230 230 231 231 **Automated:** 256 + 232 232 * Full scenario generation (comprehensive) 233 233 * Bayesian verdict scoring across scenarios 234 234 * Multi-scenario summary generation ... ... @@ -237,6 +237,7 @@ 237 237 * **Most Tier B and all Tier C** auto-published 238 238 239 239 **Human:** 265 + 240 240 * Tier A oversight (still required) 241 241 * Strategic audits (lower sampling rates, higher value) 242 242 * Ethical decisions and policy ... ... @@ -246,13 +246,13 @@ 246 246 247 247 == Automation Levels Diagram == 248 248 249 -{{include reference=" Test.FactHarborV09.Specification.Diagrams.Automation Level.WebHome"/}}275 +{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.Automation Level.WebHome"/}} 250 250 251 251 ---- 252 252 253 253 == Automation Roadmap Diagram == 254 254 255 -{{include reference=" Test.FactHarborV09.Specification.Diagrams.Automation Roadmap.WebHome"/}}281 +{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.Automation Roadmap.WebHome"/}} 256 256 257 257 ---- 258 258 ... ... @@ -268,4 +268,3 @@ 268 268 * [[Requirements (Roles)>>FactHarbor.Specification.Requirements.WebHome]] 269 269 * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] 270 270 * [[Governance>>FactHarbor.Organisation.Governance]] 271 -