Skip to Content
Last modified by Robert Schaub on 2026/02/08 21:23
From version 1.1
edited by Robert Schaub
on 2025/12/18 12:03
Change comment: Imported from XAR
To version 1.2
edited by Robert Schaub
on 2026/01/20 20:20
Change comment: Renamed back-links.

Summary

Details

Page properties
Content
... ... @@ -1,13 +1,18 @@
1 1  = AKEL — AI Knowledge Extraction Layer =
2 +
2 2  AKEL is FactHarbor's automated intelligence subsystem.
3 3  Its purpose is to reduce human workload, enhance consistency, and enable scalable knowledge processing — **without ever replacing human judgment**.
4 4  AKEL outputs are marked with **AuthorType = AI** and published according to risk-based review policies (see Publication Modes below).
5 5  AKEL operates in two modes:
7 +
6 6  * **Single-node mode** (POC & Beta 0)
7 7  * **Federated multi-node mode** (Release 1.0+)
10 +
8 8  == 1. Purpose and Role ==
12 +
9 9  AKEL transforms unstructured inputs into structured, publication-ready content.
10 10  Core responsibilities:
15 +
11 11  * Claim extraction from arbitrary text
12 12  * Claim classification (domain, type, evaluability, safety, **risk tier**)
13 13  * Scenario generation (definitions, boundaries, assumptions, methodology)
... ... @@ -17,7 +17,9 @@
17 17  * **Bubble detection** (echo chambers, conspiracy theories, isolated sources)
18 18  * Re-evaluation proposal generation
19 19  * Cross-node embedding exchange (Release 1.0+)
25 +
20 20  == 2. Components ==
27 +
21 21  * **AKEL Orchestrator** – central coordinator
22 22  * **Claim Extractor**
23 23  * **Claim Classifier** (with risk tier assignment)
... ... @@ -28,14 +28,19 @@
28 28  * **Audit Sampling Scheduler**
29 29  * **Embedding Handler** (Release 1.0+)
30 30  * **Federation Sync Adapter** (Release 1.0+)
38 +
31 31  == 3. Inputs and Outputs ==
40 +
32 32  === 3.1 Inputs ===
42 +
33 33  * User-submitted claims or evidence
34 34  * Uploaded documents
35 35  * URLs or citations
36 36  * External LLM API (optional)
37 37  * Embeddings (from local or federated peers)
48 +
38 38  === 3.2 Outputs (publication mode varies by risk tier) ===
50 +
39 39  * ClaimVersion (draft or AI-generated)
40 40  * ScenarioVersion (draft or AI-generated)
41 41  * EvidenceVersion (summary + metadata, draft or AI-generated)
... ... @@ -44,17 +44,25 @@
44 44  * Reservation and limitation notices
45 45  * Re-evaluation proposals
46 46  * Updated embeddings
59 +
47 47  == 4. Publication Modes ==
61 +
48 48  AKEL content is published according to three modes:
63 +
49 49  === 4.1 Mode 1: Draft-Only (Never Public) ===
65 +
50 50  **Used for:**
67 +
51 51  * Failed quality gate checks
52 52  * Sensitive topics flagged for expert review
53 53  * Unclear scope or missing critical sources
54 54  * High reputational risk content
55 55  **Visibility:** Internal review queue only
73 +
56 56  === 4.2 Mode 2: Published as AI-Generated (No Prior Human Review) ===
75 +
57 57  **Requirements:**
77 +
58 58  * All automated quality gates passed (see below)
59 59  * Risk tier permits AI-draft publication (Tier B or C)
60 60  * Contradiction search completed successfully
... ... @@ -69,44 +69,62 @@
69 69  * Browse and read content
70 70  * Request human review (escalates to review queue)
71 71  * Flag for expert attention
92 +
72 72  == 5. Risk tiers ==
94 +
73 73  AKEL assigns risk tiers to all content to determine appropriate review requirements:
96 +
74 74  === 5.1 Tier A — High Risk / High Impact ===
98 +
75 75  **Domains:** Medical, legal, elections, safety/security, major reputational harm
76 76  **Publication policy:**
101 +
77 77  * Human review REQUIRED before "AKEL-Generated" status
78 78  * AI-generated content MAY be published but:
79 - ** Clearly flagged as AI-draft with prominent disclaimer
80 - ** May have limited visibility
81 - ** Auto-escalated to expert review queue
82 - ** User warnings displayed
104 +** Clearly flagged as AI-draft with prominent disclaimer
105 +** May have limited visibility
106 +** Auto-escalated to expert review queue
107 +** User warnings displayed
83 83  **Audit rate:** Recommendation: 30-50% of published AI-drafts sampled in first 6 months
109 +
84 84  === 5.2 Tier B — Medium Risk ===
111 +
85 85  **Domains:** Contested public policy, complex science, causality claims, significant financial impact
86 86  **Publication policy:**
114 +
87 87  * AI-draft CAN publish immediately with clear labeling
88 88  * Sampling audits conducted (see Audit System below)
89 89  * High-engagement items auto-escalated to expert review
90 90  * Users can report issue for moderator review
91 91  **Audit rate:** Recommendation: 10-20% of published AI-drafts sampled
120 +
92 92  === 5.3 Tier C — Low Risk ===
122 +
93 93  **Domains:** Definitions, simple factual lookups with strong primary sources, historical facts, established scientific consensus
94 94  **Publication policy:**
125 +
95 95  * AI-draft default publication mode
96 96  * Sampling audits sufficient
97 97  * Community flagging available
98 98  * Human review on request
99 99  **Audit rate:** Recommendation: 5-10% of published AI-drafts sampled
131 +
100 100  == 6. Quality Gates (Mandatory Before AI-Draft Publication) ==
133 +
101 101  All AI-generated content must pass these automated checks before Mode 2 publication:
135 +
102 102  === 6.1 Gate 1: Source Quality ===
137 +
103 103  * Primary sources identified and accessible
104 104  * Source reliability scored against whitelist
105 105  * Citation completeness verified
106 106  * Publication dates checked
107 107  * Author credentials validated (where applicable)
143 +
108 108  === 6.2 Gate 2: Contradiction Search (MANDATORY) ===
145 +
109 109  **The system MUST actively search for:**
147 +
110 110  * **Counter-evidence** – Rebuttals, conflicting results, contradictory studies
111 111  * **Reservations** – Caveats, limitations, boundary conditions, applicability constraints
112 112  * **Alternative interpretations** – Different framings, definitions, contextual variations
... ... @@ -128,13 +128,17 @@
128 128  * **Significant uncertainty detected** → Require uncertainty disclosure in verdict
129 129  * **Bubble indicators present** → Flag for expert review and human validation
130 130  * **Limited perspective diversity** → Expand search or flag for human review
169 +
131 131  === 6.3 Gate 3: Uncertainty Quantification ===
171 +
132 132  * Confidence scores calculated for all claims and verdicts
133 133  * Limitations explicitly stated
134 134  * Data gaps identified and disclosed
135 135  * Strength of evidence assessed
136 136  * Alternative scenarios considered
177 +
137 137  === 6.4 Gate 4: Structural Integrity ===
179 +
138 138  * No hallucinations detected (fact-checking against sources)
139 139  * Logic chain valid and traceable
140 140  * References accessible and verifiable
... ... @@ -145,29 +145,40 @@
145 145  * Failure reason logged
146 146  * Human review required before publication
147 147  * Failure patterns analyzed for system improvement
190 +
148 148  == 7. Audit System (Sampling-Based Quality Assurance) ==
192 +
149 149  Instead of reviewing ALL AI output, FactHarbor implements stratified sampling audits:
194 +
150 150  === 7.1 Sampling Strategy ===
196 +
151 151  Audits prioritize:
198 +
152 152  * **Risk tier** (higher tiers get more frequent audits)
153 153  * **AI confidence score** (low confidence → higher sampling rate)
154 154  * **Traffic and engagement** (high-visibility content audited more)
155 155  * **Novelty** (new claim types, new domains, emerging topics)
156 156  * **Disagreement signals** (user flags, contradiction alerts, community reports)
204 +
157 157  === 7.2 Audit Process ===
206 +
158 158  1. System selects content for audit based on sampling strategy
159 159  2. Human auditor reviews AI-generated content against quality standards
160 160  3. Moderator validates or corrects:
161 - * Claim extraction accuracy
162 - * Scenario appropriateness
163 - * Evidence relevance and interpretation
164 - * Verdict reasoning
165 - * Contradiction search completeness
210 +
211 +* Claim extraction accuracy
212 +* Scenario appropriateness
213 +* Evidence relevance and interpretation
214 +* Verdict reasoning
215 +* Contradiction search completeness
166 166  4. Audit outcome recorded (pass/fail + detailed feedback)
167 167  5. Failed audits trigger immediate content review
168 168  6. Audit results feed back into system improvement
219 +
169 169  === 7.3 Feedback Loop (Continuous Improvement) ===
221 +
170 170  Audit outcomes systematically improve:
223 +
171 171  * **Query templates** – Refined based on missed evidence patterns
172 172  * **Retrieval source weights** – Adjusted for accuracy and reliability
173 173  * **Contradiction detection heuristics** – Enhanced to catch missed counter-evidence
... ... @@ -174,15 +174,22 @@
174 174  * **Model prompts and extraction rules** – Tuned for better claim extraction
175 175  * **Risk tier assignments** – Recalibrated based on error patterns
176 176  * **Bubble detection algorithms** – Improved to identify echo chambers
230 +
177 177  === 7.4 Audit Transparency ===
232 +
178 178  * Audit statistics published regularly
179 179  * Accuracy rates by risk tier tracked and reported
180 180  * System improvements documented
181 181  * Community can view aggregate audit performance
237 +
182 182  == 8. Architecture Overview ==
239 +
183 183  {{include reference="FactHarbor.Specification.Diagrams.AKEL Architecture.WebHome"/}}
241 +
184 184  == 9. AKEL and Federation ==
243 +
185 185  In Release 1.0+, AKEL participates in cross-node knowledge alignment:
245 +
186 186  * Shares embeddings
187 187  * Exchanges canonicalized claim forms
188 188  * Exchanges scenario templates
... ... @@ -194,8 +194,11 @@
194 194  * Trusted nodes: auto-merge embeddings + templates
195 195  * Neutral nodes: require additional verification
196 196  * Untrusted nodes: fully manual import
257 +
197 197  == 10. Human Review Workflow (Mode 3 Publication) ==
259 +
198 198  For content requiring human validation before "AKEL-Generated" status:
261 +
199 199  1. AKEL generates content and publishes as AI-draft (Mode 2) or keeps as draft (Mode 1)
200 200  2. Contributors inspect content in review queue
201 201  3. Contributors validate quality gates were correctly applied
... ... @@ -203,8 +203,11 @@
203 203  5. Moderators finalize "AKEL-Generated" publication
204 204  6. Version numbers increment, full history preserved
205 205  **Note:** Most AI-generated content (Tier B and C) can remain in Mode 2 (AI-Generated) indefinitely. Human review is optional for these tiers unless users or audits flag issues.
269 +
206 206  == 11. POC v1 Behavior ==
271 +
207 207  The POC explicitly demonstrates AI-generated content publication:
273 +
208 208  * Produces public AI-generated output (Mode 2)
209 209  * No human data sources required
210 210  * No human approval gate
... ... @@ -212,8 +212,10 @@
212 212  * All quality gates active (including contradiction search)
213 213  * Users understand this demonstrates AI reasoning capabilities
214 214  * Risk tier classification shown (demo purposes)
281 +
215 215  == 12. Related Pages ==
283 +
216 216  * [[Automation>>FactHarbor.Specification.Automation.WebHome]]
217 217  * [[Requirements (Roles)>>FactHarbor.Specification.Requirements.WebHome]]
218 218  * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]]
219 -* [[Governance>>FactHarbor.Organisation.Governance.WebHome]]
287 +* [[Governance>>Archive.FactHarbor.Organisation.Governance.WebHome]]