Skip to Content

Changes for page Automation

Last modified by Robert Schaub on 2025/12/24 20:34
From version 7.3
edited by Robert Schaub
on 2025/12/16 20:28
Change comment: Renamed back-links.
To version 7.1
edited by Robert Schaub
on 2025/12/15 16:56
Change comment: Imported from XAR

Summary

Details

Page properties
Content
... ... @@ -3,7 +3,6 @@
3 3  Automation in FactHarbor amplifies human capability while implementing risk-based oversight.
4 4  
5 5  This chapter defines:
6 -
7 7  * Risk-based publication model
8 8  * Quality gates for AI-generated content
9 9  * What must remain human-only
... ... @@ -19,18 +19,14 @@
19 19  
20 20  1. **Input**: User pastes a block of raw text.
21 21  1. **Deep Analysis (Background)**: The system autonomously performs the full pipeline **before** displaying the text:
22 -
23 23  * Extraction & Normalisation
24 24  * Scenario & Sub-query generation
25 25  * Evidence retrieval with **contradiction search**
26 26  * Quality gate validation
27 27  * Verdict computation
28 -
29 29  1. **Visualisation (Extraction & Marking)**: The system displays the text with claims extracted and marked.
30 -
31 31  * **Verdict-Based Coloring**: The extraction highlights (e.g. Orange/Green) are chosen **according to the computed verdict** for each claim.
32 32  * **AI-Generated Label**: Clear indication that content is AI-produced
33 -
34 34  1. **Inspection**: User clicks a highlighted claim to see the **Reasoning Trail**, showing exactly which evidence and sub-queries led to that verdict.
35 35  
36 36  === Technical Scope ===
... ... @@ -49,13 +49,11 @@
49 49  FactHarbor implements a risk-based publication model with three modes:
50 50  
51 51  === Mode 1: Draft-Only ===
52 -
53 53  * Failed quality gates
54 54  * High-risk content pending expert review
55 55  * Internal review queue only
56 56  
57 57  === Mode 2: AI-Generated (Public) ===
58 -
59 59  * Passed all quality gates
60 60  * Risk tier B or C
61 61  * Clear AI-generated labeling
... ... @@ -62,7 +62,6 @@
62 62  * Users can request human review
63 63  
64 64  === Mode 3: Human-Reviewed ===
65 -
66 66  * Validated by human reviewers/experts
67 67  * "Human-Reviewed" status badge
68 68  * Required for Tier A content publication
... ... @@ -74,7 +74,6 @@
74 74  == Risk Tiers and Automation Levels ==
75 75  
76 76  === Tier A (High Risk) ===
77 -
78 78  * **Domains**: Medical, legal, elections, safety, security
79 79  * **Automation**: AI can draft, human review required for "Human-Reviewed" status
80 80  * **AI publication**: Allowed with prominent disclaimers and warnings
... ... @@ -81,7 +81,6 @@
81 81  * **Audit rate**: Recommendation: 30-50%
82 82  
83 83  === Tier B (Medium Risk) ===
84 -
85 85  * **Domains**: Complex policy, science, causality claims
86 86  * **Automation**: AI can draft and publish (Mode 2)
87 87  * **Human review**: Optional, audit-based
... ... @@ -88,7 +88,6 @@
88 88  * **Audit rate**: Recommendation: 10-20%
89 89  
90 90  === Tier C (Low Risk) ===
91 -
92 92  * **Domains**: Definitions, established facts, historical data
93 93  * **Automation**: AI publication default
94 94  * **Human review**: On request or via sampling
... ... @@ -123,7 +123,6 @@
123 123  * **Public summaries** and accessibility text
124 124  
125 125  Most Tier B and C content remains in AI-draft status unless:
126 -
127 127  * Users request human review
128 128  * Audits identify errors
129 129  * High engagement triggers review
... ... @@ -151,30 +151,26 @@
151 151  Before AI-draft publication (Mode 2), content must pass:
152 152  
153 153  1. **Source Quality Gate**
142 + * Primary sources verified
143 + * Citations complete and accessible
144 + * Source reliability scored
154 154  
155 -* Primary sources verified
156 -* Citations complete and accessible
157 -* Source reliability scored
158 -
159 159  2. **Contradiction Search Gate** (MANDATORY)
147 + * Counter-evidence actively sought
148 + * Reservations and limitations identified
149 + * Bubble detection (echo chambers, conspiracy theories)
150 + * Diverse perspective verification
160 160  
161 -* Counter-evidence actively sought
162 -* Reservations and limitations identified
163 -* Bubble detection (echo chambers, conspiracy theories)
164 -* Diverse perspective verification
165 -
166 166  3. **Uncertainty Quantification Gate**
153 + * Confidence scores calculated
154 + * Limitations stated
155 + * Data gaps disclosed
167 167  
168 -* Confidence scores calculated
169 -* Limitations stated
170 -* Data gaps disclosed
171 -
172 172  4. **Structural Integrity Gate**
158 + * No hallucinations detected
159 + * Logic chain valid
160 + * References verifiable
173 173  
174 -* No hallucinations detected
175 -* Logic chain valid
176 -* References verifiable
177 -
178 178  See [[AKEL page>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed quality gate specifications.
179 179  
180 180  ----
... ... @@ -184,7 +184,6 @@
184 184  Instead of reviewing all AI output, systematic sampling audits ensure quality:
185 185  
186 186  === Stratified Sampling ===
187 -
188 188  * Risk tier (A > B > C sampling rates)
189 189  * Confidence scores (low confidence → more audits)
190 190  * Traffic/engagement (popular content audited more)
... ... @@ -192,9 +192,7 @@
192 192  * User flags and disagreement signals
193 193  
194 194  === Continuous Improvement Loop ===
195 -
196 196  Audit findings improve:
197 -
198 198  * Query templates
199 199  * Source reliability weights
200 200  * Contradiction detection algorithms
... ... @@ -202,7 +202,6 @@
202 202  * Bubble detection heuristics
203 203  
204 204  === Transparency ===
205 -
206 206  * Audit statistics published
207 207  * Accuracy rates by tier reported
208 208  * System improvements documented
... ... @@ -216,7 +216,6 @@
216 216  === POC (Current Focus) ===
217 217  
218 218  **Automated:**
219 -
220 220  * Claim normalization
221 221  * Scenario template generation
222 222  * Evidence metadata extraction
... ... @@ -226,7 +226,6 @@
226 226  * **Risk tier assignment**
227 227  
228 228  **Human:**
229 -
230 230  * High-risk content validation (Tier A)
231 231  * Sampling audits across all tiers
232 232  * Quality standard refinement
... ... @@ -235,7 +235,6 @@
235 235  === Beta 0 (Enhanced Automation) ===
236 236  
237 237  **Automated:**
238 -
239 239  * Detailed scenario generation
240 240  * Advanced evidence reliability scoring
241 241  * Cross-scenario comparisons
... ... @@ -244,7 +244,6 @@
244 244  * **Increased AI-draft coverage** (more Tier B content)
245 245  
246 246  **Human:**
247 -
248 248  * Tier A final approval
249 249  * Audit sampling (continued)
250 250  * Expert validation of complex domains
... ... @@ -253,7 +253,6 @@
253 253  === Release 1.0 (High Automation) ===
254 254  
255 255  **Automated:**
256 -
257 257  * Full scenario generation (comprehensive)
258 258  * Bayesian verdict scoring across scenarios
259 259  * Multi-scenario summary generation
... ... @@ -262,7 +262,6 @@
262 262  * **Most Tier B and all Tier C** auto-published
263 263  
264 264  **Human:**
265 -
266 266  * Tier A oversight (still required)
267 267  * Strategic audits (lower sampling rates, higher value)
268 268  * Ethical decisions and policy
... ... @@ -272,13 +272,13 @@
272 272  
273 273  == Automation Levels Diagram ==
274 274  
275 -{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.Automation Level.WebHome"/}}
249 +{{include reference="Test.FactHarborV09.Specification.Diagrams.Automation Level.WebHome"/}}
276 276  
277 277  ----
278 278  
279 279  == Automation Roadmap Diagram ==
280 280  
281 -{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.Automation Roadmap.WebHome"/}}
255 +{{include reference="Test.FactHarborV09.Specification.Diagrams.Automation Roadmap.WebHome"/}}
282 282  
283 283  ----
284 284  
... ... @@ -294,3 +294,4 @@
294 294  * [[Requirements (Roles)>>FactHarbor.Specification.Requirements.WebHome]]
295 295  * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]]
296 296  * [[Governance>>FactHarbor.Organisation.Governance]]
271 +