Skip to Content

Changes for page Automation

Last modified by Robert Schaub on 2025/12/24 20:33
From version 1.3
edited by Robert Schaub
on 2025/12/16 20:40
Change comment: Update document after refactoring.
To version 1.1
edited by Robert Schaub
on 2025/12/16 20:34
Change comment: Imported from XAR

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -Test.FactHarbor V0\.9\.23 Lost Data.Specification.WebHome
1 +Test.FactHarborV09.Specification.WebHome
Content
... ... @@ -3,7 +3,6 @@
3 3  Automation in FactHarbor amplifies human capability while implementing risk-based oversight.
4 4  
5 5  This chapter defines:
6 -
7 7  * Risk-based publication model
8 8  * Quality gates for AI-generated content
9 9  * What must remain human-only
... ... @@ -19,18 +19,14 @@
19 19  
20 20  1. **Input**: User pastes a block of raw text.
21 21  1. **Deep Analysis (Background)**: The system autonomously performs the full pipeline **before** displaying the text:
22 -
23 23  * Extraction & Normalisation
24 24  * Scenario & Sub-query generation
25 25  * Evidence retrieval with **contradiction search**
26 26  * Quality gate validation
27 27  * Verdict computation
28 -
29 29  1. **Visualisation (Extraction & Marking)**: The system displays the text with claims extracted and marked.
30 -
31 31  * **Verdict-Based Coloring**: The extraction highlights (e.g. Orange/Green) are chosen **according to the computed verdict** for each claim.
32 32  * **AI-Generated Label**: Clear indication that content is AI-produced
33 -
34 34  1. **Inspection**: User clicks a highlighted claim to see the **Reasoning Trail**, showing exactly which evidence and sub-queries led to that verdict.
35 35  
36 36  === 1.2 Technical Scope ===
... ... @@ -42,18 +42,17 @@
42 42  * **No Human Approval Gate**: Demonstrates scalable AI publication
43 43  * **Structured Sub-Queries**: Logic generated by decomposing claims into the FactHarbor data model
44 44  
40 +
45 45  == 2. Publication Model ==
46 46  
47 47  FactHarbor implements a risk-based publication model with three modes:
48 48  
49 49  === 2.1 Mode 1: Draft-Only ===
50 -
51 51  * Failed quality gates
52 52  * High-risk content pending expert review
53 53  * Internal review queue only
54 54  
55 55  === 2.2 Mode 2: AI-Generated (Public) ===
56 -
57 57  * Passed all quality gates
58 58  * Risk tier B or C
59 59  * Clear AI-generated labeling
... ... @@ -60,18 +60,16 @@
60 60  * Users can request human review
61 61  
62 62  === 2.3 Mode 3: Human-Reviewed ===
63 -
64 64  * Validated by human reviewers/experts
65 65  * "Human-Reviewed" status badge
66 66  * Required for Tier A content publication
67 67  
68 -See [[AKEL page>>Test.FactHarbor V0\.9\.23 Lost Data.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed publication mode descriptions.
61 +See [[AKEL page>>Test.FactHarborV09.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed publication mode descriptions.
69 69  
70 70  
71 71  == 3. Risk Tiers and Automation Levels ==
72 72  
73 73  === 3.1 Tier A (High Risk) ===
74 -
75 75  * **Domains**: Medical, legal, elections, safety, security
76 76  * **Automation**: AI can draft, human review required for "Human-Reviewed" status
77 77  * **AI publication**: Allowed with prominent disclaimers and warnings
... ... @@ -78,7 +78,6 @@
78 78  * **Audit rate**: Recommendation: 30-50%
79 79  
80 80  === 3.2 Tier B (Medium Risk) ===
81 -
82 82  * **Domains**: Complex policy, science, causality claims
83 83  * **Automation**: AI can draft and publish (Mode 2)
84 84  * **Human review**: Optional, audit-based
... ... @@ -85,12 +85,12 @@
85 85  * **Audit rate**: Recommendation: 10-20%
86 86  
87 87  === 3.3 Tier C (Low Risk) ===
88 -
89 89  * **Domains**: Definitions, established facts, historical data
90 90  * **Automation**: AI publication default
91 91  * **Human review**: On request or via sampling
92 92  * **Audit rate**: Recommendation: 5-10%
93 93  
84 +
94 94  == 4. Human-Only Tasks ==
95 95  
96 96  These require human judgment and cannot be automated:
... ... @@ -102,6 +102,7 @@
102 102  * **Audit system oversight** and quality standard definition
103 103  * **Risk tier policy** adjustments based on societal context
104 104  
96 +
105 105  == 5. AI-Draft with Audit (Semi-Automated) ==
106 106  
107 107  AKEL drafts these; humans validate via sampling audits:
... ... @@ -116,12 +116,12 @@
116 116  * **Public summaries** and accessibility text
117 117  
118 118  Most Tier B and C content remains in AI-draft status unless:
119 -
120 120  * Users request human review
121 121  * Audits identify errors
122 122  * High engagement triggers review
123 123  * Community flags issues
124 124  
116 +
125 125  == 6. Fully Automated Structural Tasks ==
126 126  
127 127  These require no human interpretation:
... ... @@ -135,44 +135,40 @@
135 135  * **Layout generation** (diagrams, summaries, UI presentation)
136 136  * **Federation integrity checks** (cross-node data validation)
137 137  
130 +
138 138  == 7. Quality Gates (Automated) ==
139 139  
140 140  Before AI-draft publication (Mode 2), content must pass:
141 141  
142 142  1. **Source Quality Gate**
136 + * Primary sources verified
137 + * Citations complete and accessible
138 + * Source reliability scored
143 143  
144 -* Primary sources verified
145 -* Citations complete and accessible
146 -* Source reliability scored
147 -
148 148  2. **Contradiction Search Gate** (MANDATORY)
141 + * Counter-evidence actively sought
142 + * Reservations and limitations identified
143 + * Bubble detection (echo chambers, conspiracy theories)
144 + * Diverse perspective verification
149 149  
150 -* Counter-evidence actively sought
151 -* Reservations and limitations identified
152 -* Bubble detection (echo chambers, conspiracy theories)
153 -* Diverse perspective verification
154 -
155 155  3. **Uncertainty Quantification Gate**
147 + * Confidence scores calculated
148 + * Limitations stated
149 + * Data gaps disclosed
156 156  
157 -* Confidence scores calculated
158 -* Limitations stated
159 -* Data gaps disclosed
160 -
161 161  4. **Structural Integrity Gate**
152 + * No hallucinations detected
153 + * Logic chain valid
154 + * References verifiable
162 162  
163 -* No hallucinations detected
164 -* Logic chain valid
165 -* References verifiable
156 +See [[AKEL page>>Test.FactHarborV09.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed quality gate specifications.
166 166  
167 -See [[AKEL page>>Test.FactHarbor V0\.9\.23 Lost Data.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed quality gate specifications.
168 168  
169 -
170 170  == 8. Audit System ==
171 171  
172 172  Instead of reviewing all AI output, systematic sampling audits ensure quality:
173 173  
174 174  === 8.1 Stratified Sampling ===
175 -
176 176  * Risk tier (A > B > C sampling rates)
177 177  * Confidence scores (low confidence → more audits)
178 178  * Traffic/engagement (popular content audited more)
... ... @@ -180,9 +180,7 @@
180 180  * User flags and disagreement signals
181 181  
182 182  === 8.2 Continuous Improvement Loop ===
183 -
184 184  Audit findings improve:
185 -
186 186  * Query templates
187 187  * Source reliability weights
188 188  * Contradiction detection algorithms
... ... @@ -190,11 +190,11 @@
190 190  * Bubble detection heuristics
191 191  
192 192  === 8.3 Transparency ===
193 -
194 194  * Audit statistics published
195 195  * Accuracy rates by tier reported
196 196  * System improvements documented
197 197  
183 +
198 198  == 9. Automation Roadmap ==
199 199  
200 200  Automation capabilities increase with system maturity while maintaining quality oversight.
... ... @@ -202,7 +202,6 @@
202 202  === 9.1 POC (Current Focus) ===
203 203  
204 204  **Automated:**
205 -
206 206  * Claim normalization
207 207  * Scenario template generation
208 208  * Evidence metadata extraction
... ... @@ -212,7 +212,6 @@
212 212  * **Risk tier assignment**
213 213  
214 214  **Human:**
215 -
216 216  * High-risk content validation (Tier A)
217 217  * Sampling audits across all tiers
218 218  * Quality standard refinement
... ... @@ -221,7 +221,6 @@
221 221  === 9.2 Beta 0 (Enhanced Automation) ===
222 222  
223 223  **Automated:**
224 -
225 225  * Detailed scenario generation
226 226  * Advanced evidence reliability scoring
227 227  * Cross-scenario comparisons
... ... @@ -230,7 +230,6 @@
230 230  * **Increased AI-draft coverage** (more Tier B content)
231 231  
232 232  **Human:**
233 -
234 234  * Tier A final approval
235 235  * Audit sampling (continued)
236 236  * Expert validation of complex domains
... ... @@ -239,7 +239,6 @@
239 239  === 9.3 Release 1.0 (High Automation) ===
240 240  
241 241  **Automated:**
242 -
243 243  * Full scenario generation (comprehensive)
244 244  * Bayesian verdict scoring across scenarios
245 245  * Multi-scenario summary generation
... ... @@ -248,12 +248,12 @@
248 248  * **Most Tier B and all Tier C** auto-published
249 249  
250 250  **Human:**
251 -
252 252  * Tier A oversight (still required)
253 253  * Strategic audits (lower sampling rates, higher value)
254 254  * Ethical decisions and policy
255 255  * Conflict resolution
256 256  
237 +
257 257  == 10. Automation Levels Diagram ==
258 258  
259 259  {{include reference="Test.FactHarborV09.Specification.Diagrams.Automation Level.WebHome"}}
... ... @@ -275,4 +275,4 @@
275 275  * [[Requirements (Roles)>>Test.FactHarborV09.Specification.Requirements.WebHome]]
276 276  * [[Workflows>>Test.FactHarborV09.Specification.Workflows.WebHome]]
277 277  * [[Governance>>Test.FactHarborV09.Organisation.Governance]]
278 -{{/include}}
259 +