Skip to Content

Changes for page Automation

Last modified by Robert Schaub on 2025/12/24 20:34
From version 6.1
edited by Robert Schaub
on 2025/12/14 18:59
Change comment: Imported from XAR
To version 7.5
edited by Robert Schaub
on 2025/12/16 21:39
Change comment: Renamed back-links.

Summary

Details

Page properties
Content
... ... @@ -3,6 +3,7 @@
3 3  Automation in FactHarbor amplifies human capability while implementing risk-based oversight.
4 4  
5 5  This chapter defines:
6 +
6 6  * Risk-based publication model
7 7  * Quality gates for AI-generated content
8 8  * What must remain human-only
... ... @@ -18,14 +18,18 @@
18 18  
19 19  1. **Input**: User pastes a block of raw text.
20 20  1. **Deep Analysis (Background)**: The system autonomously performs the full pipeline **before** displaying the text:
22 +
21 21  * Extraction & Normalisation
22 22  * Scenario & Sub-query generation
23 23  * Evidence retrieval with **contradiction search**
24 24  * Quality gate validation
25 25  * Verdict computation
28 +
26 26  1. **Visualisation (Extraction & Marking)**: The system displays the text with claims extracted and marked.
30 +
27 27  * **Verdict-Based Coloring**: The extraction highlights (e.g. Orange/Green) are chosen **according to the computed verdict** for each claim.
28 28  * **AI-Generated Label**: Clear indication that content is AI-produced
33 +
29 29  1. **Inspection**: User clicks a highlighted claim to see the **Reasoning Trail**, showing exactly which evidence and sub-queries led to that verdict.
30 30  
31 31  === Technical Scope ===
... ... @@ -44,11 +44,13 @@
44 44  FactHarbor implements a risk-based publication model with three modes:
45 45  
46 46  === Mode 1: Draft-Only ===
52 +
47 47  * Failed quality gates
48 48  * High-risk content pending expert review
49 49  * Internal review queue only
50 50  
51 51  === Mode 2: AI-Generated (Public) ===
58 +
52 52  * Passed all quality gates
53 53  * Risk tier B or C
54 54  * Clear AI-generated labeling
... ... @@ -55,11 +55,12 @@
55 55  * Users can request human review
56 56  
57 57  === Mode 3: Human-Reviewed ===
65 +
58 58  * Validated by human reviewers/experts
59 59  * "Human-Reviewed" status badge
60 60  * Required for Tier A content publication
61 61  
62 -See [[AKEL page>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed publication mode descriptions.
70 +See [[AKEL page>>FactHarbor.Specification V0\.9\.18.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed publication mode descriptions.
63 63  
64 64  ----
65 65  
... ... @@ -66,6 +66,7 @@
66 66  == Risk Tiers and Automation Levels ==
67 67  
68 68  === Tier A (High Risk) ===
77 +
69 69  * **Domains**: Medical, legal, elections, safety, security
70 70  * **Automation**: AI can draft, human review required for "Human-Reviewed" status
71 71  * **AI publication**: Allowed with prominent disclaimers and warnings
... ... @@ -72,6 +72,7 @@
72 72  * **Audit rate**: Recommendation: 30-50%
73 73  
74 74  === Tier B (Medium Risk) ===
84 +
75 75  * **Domains**: Complex policy, science, causality claims
76 76  * **Automation**: AI can draft and publish (Mode 2)
77 77  * **Human review**: Optional, audit-based
... ... @@ -78,6 +78,7 @@
78 78  * **Audit rate**: Recommendation: 10-20%
79 79  
80 80  === Tier C (Low Risk) ===
91 +
81 81  * **Domains**: Definitions, established facts, historical data
82 82  * **Automation**: AI publication default
83 83  * **Human review**: On request or via sampling
... ... @@ -112,6 +112,7 @@
112 112  * **Public summaries** and accessibility text
113 113  
114 114  Most Tier B and C content remains in AI-draft status unless:
126 +
115 115  * Users request human review
116 116  * Audits identify errors
117 117  * High engagement triggers review
... ... @@ -139,28 +139,32 @@
139 139  Before AI-draft publication (Mode 2), content must pass:
140 140  
141 141  1. **Source Quality Gate**
142 - * Primary sources verified
143 - * Citations complete and accessible
144 - * Source reliability scored
145 145  
155 +* Primary sources verified
156 +* Citations complete and accessible
157 +* Source reliability scored
158 +
146 146  2. **Contradiction Search Gate** (MANDATORY)
147 - * Counter-evidence actively sought
148 - * Reservations and limitations identified
149 - * Bubble detection (echo chambers, conspiracy theories)
150 - * Diverse perspective verification
151 151  
161 +* Counter-evidence actively sought
162 +* Reservations and limitations identified
163 +* Bubble detection (echo chambers, conspiracy theories)
164 +* Diverse perspective verification
165 +
152 152  3. **Uncertainty Quantification Gate**
153 - * Confidence scores calculated
154 - * Limitations stated
155 - * Data gaps disclosed
156 156  
168 +* Confidence scores calculated
169 +* Limitations stated
170 +* Data gaps disclosed
171 +
157 157  4. **Structural Integrity Gate**
158 - * No hallucinations detected
159 - * Logic chain valid
160 - * References verifiable
161 161  
162 -See [[AKEL page>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed quality gate specifications.
174 +* No hallucinations detected
175 +* Logic chain valid
176 +* References verifiable
163 163  
178 +See [[AKEL page>>FactHarbor.Specification V0\.9\.18.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed quality gate specifications.
179 +
164 164  ----
165 165  
166 166  == Audit System ==
... ... @@ -168,6 +168,7 @@
168 168  Instead of reviewing all AI output, systematic sampling audits ensure quality:
169 169  
170 170  === Stratified Sampling ===
187 +
171 171  * Risk tier (A > B > C sampling rates)
172 172  * Confidence scores (low confidence → more audits)
173 173  * Traffic/engagement (popular content audited more)
... ... @@ -175,7 +175,9 @@
175 175  * User flags and disagreement signals
176 176  
177 177  === Continuous Improvement Loop ===
195 +
178 178  Audit findings improve:
197 +
179 179  * Query templates
180 180  * Source reliability weights
181 181  * Contradiction detection algorithms
... ... @@ -183,6 +183,7 @@
183 183  * Bubble detection heuristics
184 184  
185 185  === Transparency ===
205 +
186 186  * Audit statistics published
187 187  * Accuracy rates by tier reported
188 188  * System improvements documented
... ... @@ -196,6 +196,7 @@
196 196  === POC (Current Focus) ===
197 197  
198 198  **Automated:**
219 +
199 199  * Claim normalization
200 200  * Scenario template generation
201 201  * Evidence metadata extraction
... ... @@ -205,6 +205,7 @@
205 205  * **Risk tier assignment**
206 206  
207 207  **Human:**
229 +
208 208  * High-risk content validation (Tier A)
209 209  * Sampling audits across all tiers
210 210  * Quality standard refinement
... ... @@ -213,6 +213,7 @@
213 213  === Beta 0 (Enhanced Automation) ===
214 214  
215 215  **Automated:**
238 +
216 216  * Detailed scenario generation
217 217  * Advanced evidence reliability scoring
218 218  * Cross-scenario comparisons
... ... @@ -221,6 +221,7 @@
221 221  * **Increased AI-draft coverage** (more Tier B content)
222 222  
223 223  **Human:**
247 +
224 224  * Tier A final approval
225 225  * Audit sampling (continued)
226 226  * Expert validation of complex domains
... ... @@ -229,6 +229,7 @@
229 229  === Release 1.0 (High Automation) ===
230 230  
231 231  **Automated:**
256 +
232 232  * Full scenario generation (comprehensive)
233 233  * Bayesian verdict scoring across scenarios
234 234  * Multi-scenario summary generation
... ... @@ -237,6 +237,7 @@
237 237  * **Most Tier B and all Tier C** auto-published
238 238  
239 239  **Human:**
265 +
240 240  * Tier A oversight (still required)
241 241  * Strategic audits (lower sampling rates, higher value)
242 242  * Ethical decisions and policy
... ... @@ -246,26 +246,25 @@
246 246  
247 247  == Automation Levels Diagram ==
248 248  
249 -{{include reference="FactHarbor.Specification.Diagrams.Automation Level.WebHome"/}}
275 +{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.Automation Level.WebHome"/}}
250 250  
251 251  ----
252 252  
253 253  == Automation Roadmap Diagram ==
254 254  
255 -{{include reference="FactHarbor.Specification.Diagrams.Automation Roadmap.WebHome"/}}
281 +{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.Automation Roadmap.WebHome"/}}
256 256  
257 257  ----
258 258  
259 259  == Manual vs Automated Matrix ==
260 260  
261 -{{include reference="FactHarbor.Specification.Diagrams.Manual vs Automated matrix.WebHome"/}}
287 +{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.Manual vs Automated matrix.WebHome"/}}
262 262  
263 263  ----
264 264  
265 265  == Related Pages ==
266 266  
267 -* [[AKEL (AI Knowledge Extraction Layer)>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]]
293 +* [[AKEL (AI Knowledge Extraction Layer)>>FactHarbor.Specification V0\.9\.18.AI Knowledge Extraction Layer (AKEL).WebHome]]
268 268  * [[Requirements (Roles)>>FactHarbor.Specification.Requirements.WebHome]]
269 269  * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]]
270 270  * [[Governance>>FactHarbor.Organisation.Governance]]
271 -