Skip to Content

Changes for page Automation

Last modified by Robert Schaub on 2025/12/24 20:33
From version 1.1
edited by Robert Schaub
on 2025/12/16 20:34
Change comment: Imported from XAR
To version 1.7
edited by Robert Schaub
on 2025/12/16 20:54
Change comment: Renamed back-links.

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -Test.FactHarborV09.Specification.WebHome
1 +FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.WebHome
Content
... ... @@ -3,6 +3,7 @@
3 3  Automation in FactHarbor amplifies human capability while implementing risk-based oversight.
4 4  
5 5  This chapter defines:
6 +
6 6  * Risk-based publication model
7 7  * Quality gates for AI-generated content
8 8  * What must remain human-only
... ... @@ -18,14 +18,18 @@
18 18  
19 19  1. **Input**: User pastes a block of raw text.
20 20  1. **Deep Analysis (Background)**: The system autonomously performs the full pipeline **before** displaying the text:
22 +
21 21  * Extraction & Normalisation
22 22  * Scenario & Sub-query generation
23 23  * Evidence retrieval with **contradiction search**
24 24  * Quality gate validation
25 25  * Verdict computation
28 +
26 26  1. **Visualisation (Extraction & Marking)**: The system displays the text with claims extracted and marked.
30 +
27 27  * **Verdict-Based Coloring**: The extraction highlights (e.g. Orange/Green) are chosen **according to the computed verdict** for each claim.
28 28  * **AI-Generated Label**: Clear indication that content is AI-produced
33 +
29 29  1. **Inspection**: User clicks a highlighted claim to see the **Reasoning Trail**, showing exactly which evidence and sub-queries led to that verdict.
30 30  
31 31  === 1.2 Technical Scope ===
... ... @@ -37,17 +37,18 @@
37 37  * **No Human Approval Gate**: Demonstrates scalable AI publication
38 38  * **Structured Sub-Queries**: Logic generated by decomposing claims into the FactHarbor data model
39 39  
40 -
41 41  == 2. Publication Model ==
42 42  
43 43  FactHarbor implements a risk-based publication model with three modes:
44 44  
45 45  === 2.1 Mode 1: Draft-Only ===
50 +
46 46  * Failed quality gates
47 47  * High-risk content pending expert review
48 48  * Internal review queue only
49 49  
50 50  === 2.2 Mode 2: AI-Generated (Public) ===
56 +
51 51  * Passed all quality gates
52 52  * Risk tier B or C
53 53  * Clear AI-generated labeling
... ... @@ -54,16 +54,18 @@
54 54  * Users can request human review
55 55  
56 56  === 2.3 Mode 3: Human-Reviewed ===
63 +
57 57  * Validated by human reviewers/experts
58 58  * "Human-Reviewed" status badge
59 59  * Required for Tier A content publication
60 60  
61 -See [[AKEL page>>Test.FactHarborV09.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed publication mode descriptions.
68 +See [[AKEL page>>FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed publication mode descriptions.
62 62  
63 63  
64 64  == 3. Risk Tiers and Automation Levels ==
65 65  
66 66  === 3.1 Tier A (High Risk) ===
74 +
67 67  * **Domains**: Medical, legal, elections, safety, security
68 68  * **Automation**: AI can draft, human review required for "Human-Reviewed" status
69 69  * **AI publication**: Allowed with prominent disclaimers and warnings
... ... @@ -70,6 +70,7 @@
70 70  * **Audit rate**: Recommendation: 30-50%
71 71  
72 72  === 3.2 Tier B (Medium Risk) ===
81 +
73 73  * **Domains**: Complex policy, science, causality claims
74 74  * **Automation**: AI can draft and publish (Mode 2)
75 75  * **Human review**: Optional, audit-based
... ... @@ -76,12 +76,12 @@
76 76  * **Audit rate**: Recommendation: 10-20%
77 77  
78 78  === 3.3 Tier C (Low Risk) ===
88 +
79 79  * **Domains**: Definitions, established facts, historical data
80 80  * **Automation**: AI publication default
81 81  * **Human review**: On request or via sampling
82 82  * **Audit rate**: Recommendation: 5-10%
83 83  
84 -
85 85  == 4. Human-Only Tasks ==
86 86  
87 87  These require human judgment and cannot be automated:
... ... @@ -93,7 +93,6 @@
93 93  * **Audit system oversight** and quality standard definition
94 94  * **Risk tier policy** adjustments based on societal context
95 95  
96 -
97 97  == 5. AI-Draft with Audit (Semi-Automated) ==
98 98  
99 99  AKEL drafts these; humans validate via sampling audits:
... ... @@ -108,12 +108,12 @@
108 108  * **Public summaries** and accessibility text
109 109  
110 110  Most Tier B and C content remains in AI-draft status unless:
119 +
111 111  * Users request human review
112 112  * Audits identify errors
113 113  * High engagement triggers review
114 114  * Community flags issues
115 115  
116 -
117 117  == 6. Fully Automated Structural Tasks ==
118 118  
119 119  These require no human interpretation:
... ... @@ -127,40 +127,44 @@
127 127  * **Layout generation** (diagrams, summaries, UI presentation)
128 128  * **Federation integrity checks** (cross-node data validation)
129 129  
130 -
131 131  == 7. Quality Gates (Automated) ==
132 132  
133 133  Before AI-draft publication (Mode 2), content must pass:
134 134  
135 135  1. **Source Quality Gate**
136 - * Primary sources verified
137 - * Citations complete and accessible
138 - * Source reliability scored
139 139  
144 +* Primary sources verified
145 +* Citations complete and accessible
146 +* Source reliability scored
147 +
140 140  2. **Contradiction Search Gate** (MANDATORY)
141 - * Counter-evidence actively sought
142 - * Reservations and limitations identified
143 - * Bubble detection (echo chambers, conspiracy theories)
144 - * Diverse perspective verification
145 145  
150 +* Counter-evidence actively sought
151 +* Reservations and limitations identified
152 +* Bubble detection (echo chambers, conspiracy theories)
153 +* Diverse perspective verification
154 +
146 146  3. **Uncertainty Quantification Gate**
147 - * Confidence scores calculated
148 - * Limitations stated
149 - * Data gaps disclosed
150 150  
157 +* Confidence scores calculated
158 +* Limitations stated
159 +* Data gaps disclosed
160 +
151 151  4. **Structural Integrity Gate**
152 - * No hallucinations detected
153 - * Logic chain valid
154 - * References verifiable
155 155  
156 -See [[AKEL page>>Test.FactHarborV09.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed quality gate specifications.
163 +* No hallucinations detected
164 +* Logic chain valid
165 +* References verifiable
157 157  
167 +See [[AKEL page>>FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed quality gate specifications.
158 158  
169 +
159 159  == 8. Audit System ==
160 160  
161 161  Instead of reviewing all AI output, systematic sampling audits ensure quality:
162 162  
163 163  === 8.1 Stratified Sampling ===
175 +
164 164  * Risk tier (A > B > C sampling rates)
165 165  * Confidence scores (low confidence → more audits)
166 166  * Traffic/engagement (popular content audited more)
... ... @@ -168,7 +168,9 @@
168 168  * User flags and disagreement signals
169 169  
170 170  === 8.2 Continuous Improvement Loop ===
183 +
171 171  Audit findings improve:
185 +
172 172  * Query templates
173 173  * Source reliability weights
174 174  * Contradiction detection algorithms
... ... @@ -176,11 +176,11 @@
176 176  * Bubble detection heuristics
177 177  
178 178  === 8.3 Transparency ===
193 +
179 179  * Audit statistics published
180 180  * Accuracy rates by tier reported
181 181  * System improvements documented
182 182  
183 -
184 184  == 9. Automation Roadmap ==
185 185  
186 186  Automation capabilities increase with system maturity while maintaining quality oversight.
... ... @@ -188,6 +188,7 @@
188 188  === 9.1 POC (Current Focus) ===
189 189  
190 190  **Automated:**
205 +
191 191  * Claim normalization
192 192  * Scenario template generation
193 193  * Evidence metadata extraction
... ... @@ -197,6 +197,7 @@
197 197  * **Risk tier assignment**
198 198  
199 199  **Human:**
215 +
200 200  * High-risk content validation (Tier A)
201 201  * Sampling audits across all tiers
202 202  * Quality standard refinement
... ... @@ -205,6 +205,7 @@
205 205  === 9.2 Beta 0 (Enhanced Automation) ===
206 206  
207 207  **Automated:**
224 +
208 208  * Detailed scenario generation
209 209  * Advanced evidence reliability scoring
210 210  * Cross-scenario comparisons
... ... @@ -213,6 +213,7 @@
213 213  * **Increased AI-draft coverage** (more Tier B content)
214 214  
215 215  **Human:**
233 +
216 216  * Tier A final approval
217 217  * Audit sampling (continued)
218 218  * Expert validation of complex domains
... ... @@ -221,6 +221,7 @@
221 221  === 9.3 Release 1.0 (High Automation) ===
222 222  
223 223  **Automated:**
242 +
224 224  * Full scenario generation (comprehensive)
225 225  * Bayesian verdict scoring across scenarios
226 226  * Multi-scenario summary generation
... ... @@ -229,15 +229,15 @@
229 229  * **Most Tier B and all Tier C** auto-published
230 230  
231 231  **Human:**
251 +
232 232  * Tier A oversight (still required)
233 233  * Strategic audits (lower sampling rates, higher value)
234 234  * Ethical decisions and policy
235 235  * Conflict resolution
236 236  
237 -
238 238  == 10. Automation Levels Diagram ==
239 239  
240 -{{include reference="Test.FactHarborV09.Specification.Diagrams.Automation Level.WebHome"}}
259 +{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.Automation Level.WebHome"}}
241 241  
242 242  
243 243  == 11. Automation Roadmap Diagram ==
... ... @@ -256,4 +256,6 @@
256 256  * [[Requirements (Roles)>>Test.FactHarborV09.Specification.Requirements.WebHome]]
257 257  * [[Workflows>>Test.FactHarborV09.Specification.Workflows.WebHome]]
258 258  * [[Governance>>Test.FactHarborV09.Organisation.Governance]]
259 -
278 +{{/include}}
279 +{{/include}}
280 +{{/include}}