Wiki source code of AI Knowledge Extraction Layer (AKEL)
Version 5.1 by Robert Schaub on 2025/12/14 18:59
Hide last authors
| author | version | line-number | content |
|---|---|---|---|
 |
1.1 | 1 | = AKEL — AI Knowledge Extraction Layer = |
| 2 | |||
| |
5.1 | 3 | AKEL is FactHarbor's automated intelligence subsystem. |
| |
1.1 | 4 | Its purpose is to reduce human workload, enhance consistency, and enable scalable knowledge processing — **without ever replacing human judgment**. |
| 5 | |||
| |
5.1 | 6 | AKEL outputs are marked with **AuthorType = AI** and published according to risk-based review policies (see Publication Modes below). |
| |
1.1 | 7 | |
| 8 | AKEL operates in two modes: | ||
| 9 | |||
| 10 | * **Single-node mode** (POC & Beta 0) | ||
| 11 | * **Federated multi-node mode** (Release 1.0+) | ||
| 12 | |||
| |
5.1 | 13 | Human reviewers, experts, and moderators always retain final authority over content marked as "Human-Reviewed." |
| |
1.1 | 14 | |
| |
4.1 | 15 | ---- |
| 16 | |||
| |
1.1 | 17 | == Purpose and Role == |
| 18 | |||
| |
5.1 | 19 | AKEL transforms unstructured inputs into structured, publication-ready content. |
| |
1.1 | 20 | |
| 21 | Core responsibilities: | ||
| 22 | |||
| 23 | * Claim extraction from arbitrary text | ||
| |
5.1 | 24 | * Claim classification (domain, type, evaluability, safety, **risk tier**) |
| |
1.1 | 25 | * Scenario generation (definitions, boundaries, assumptions, methodology) |
| 26 | * Evidence summarization and metadata extraction | ||
| |
5.1 | 27 | * **Contradiction detection and counter-evidence search** |
| 28 | * **Reservation and limitation identification** | ||
| 29 | * **Bubble detection** (echo chambers, conspiracy theories, isolated sources) | ||
| |
1.1 | 30 | * Re-evaluation proposal generation |
| 31 | * Cross-node embedding exchange (Release 1.0+) | ||
| 32 | |||
| |
4.1 | 33 | ---- |
| 34 | |||
| |
1.1 | 35 | == Components == |
| 36 | |||
| 37 | * **AKEL Orchestrator** – central coordinator | ||
| 38 | * **Claim Extractor** | ||
| |
5.1 | 39 | * **Claim Classifier** (with risk tier assignment) |
| |
1.1 | 40 | * **Scenario Generator** |
| 41 | * **Evidence Summarizer** | ||
| |
5.1 | 42 | * **Contradiction Detector** (enhanced with counter-evidence search) |
| 43 | * **Quality Gate Validator** | ||
| 44 | * **Audit Sampling Scheduler** | ||
| |
1.1 | 45 | * **Embedding Handler** (Release 1.0+) |
| 46 | * **Federation Sync Adapter** (Release 1.0+) | ||
| 47 | |||
| |
4.1 | 48 | ---- |
| 49 | |||
| |
1.1 | 50 | == Inputs and Outputs == |
| 51 | |||
| 52 | === Inputs === | ||
| |
4.1 | 53 | * User-submitted claims or evidence |
| 54 | * Uploaded documents | ||
| 55 | * URLs or citations | ||
| 56 | * External LLM API (optional) | ||
| |
1.1 | 57 | * Embeddings (from local or federated peers) |
| 58 | |||
| |
5.1 | 59 | === Outputs (publication mode varies by risk tier) === |
| 60 | * ClaimVersion (draft or AI-generated) | ||
| 61 | * ScenarioVersion (draft or AI-generated) | ||
| 62 | * EvidenceVersion (summary + metadata, draft or AI-generated) | ||
| 63 | * VerdictVersion (draft, AI-generated, or human-reviewed) | ||
| |
4.1 | 64 | * Contradiction alerts |
| |
5.1 | 65 | * Reservation and limitation notices |
| |
4.1 | 66 | * Re-evaluation proposals |
| |
1.1 | 67 | * Updated embeddings |
| 68 | |||
| |
4.1 | 69 | ---- |
| 70 | |||
| |
5.1 | 71 | == Publication Modes == |
| 72 | |||
| 73 | AKEL content is published according to three modes: | ||
| 74 | |||
| 75 | === Mode 1: Draft-Only (Never Public) === | ||
| 76 | |||
| 77 | **Used for:** | ||
| 78 | * Failed quality gate checks | ||
| 79 | * Sensitive topics flagged for expert review | ||
| 80 | * Unclear scope or missing critical sources | ||
| 81 | * High reputational risk content | ||
| 82 | |||
| 83 | **Visibility:** Internal review queue only | ||
| 84 | |||
| 85 | === Mode 2: Published as AI-Generated (No Prior Human Review) === | ||
| 86 | |||
| 87 | **Requirements:** | ||
| 88 | * All automated quality gates passed (see below) | ||
| 89 | * Risk tier permits AI-draft publication (Tier B or C) | ||
| 90 | * Contradiction search completed successfully | ||
| 91 | * Clear labeling as "AI-Generated, Awaiting Human Review" | ||
| 92 | |||
| 93 | **Label shown to users:** | ||
| 94 | ``` | ||
| 95 | [AI-Generated] This content was produced by AI and has not yet been human-reviewed. | ||
| 96 | Source: AI | Review Status: Pending | Risk Tier: [B/C] | ||
| 97 | Contradiction Search: Completed | Last Updated: [timestamp] | ||
| 98 | ``` | ||
| 99 | |||
| 100 | **User actions:** | ||
| 101 | * Browse and read content | ||
| 102 | * Request human review (escalates to review queue) | ||
| 103 | * Flag for expert attention | ||
| 104 | |||
| 105 | === Mode 3: Published as Human-Reviewed === | ||
| 106 | |||
| 107 | **Requirements:** | ||
| 108 | * Human reviewer or domain expert validated | ||
| 109 | * All quality gates passed | ||
| 110 | * Visible "Human-Reviewed" mark with reviewer role and timestamp | ||
| 111 | |||
| 112 | **Label shown to users:** | ||
| 113 | ``` | ||
| 114 | [Human-Reviewed] This content has been validated by human reviewers. | ||
| 115 | Source: AI+Human | Review Status: Approved | Reviewed by: [Role] on [timestamp] | ||
| 116 | Risk Tier: [A/B/C] | Contradiction Search: Completed | ||
| 117 | ``` | ||
| 118 | |||
| 119 | ---- | ||
| 120 | |||
| 121 | == Risk Tiers == | ||
| 122 | |||
| 123 | AKEL assigns risk tiers to all content to determine appropriate review requirements: | ||
| 124 | |||
| 125 | === Tier A — High Risk / High Impact === | ||
| 126 | |||
| 127 | **Domains:** Medical, legal, elections, safety/security, major reputational harm | ||
| 128 | |||
| 129 | **Publication policy:** | ||
| 130 | * Human review REQUIRED before "Human-Reviewed" status | ||
| 131 | * AI-generated content MAY be published but: | ||
| 132 | ** Clearly flagged as AI-draft with prominent disclaimer | ||
| 133 | ** May have limited visibility | ||
| 134 | ** Auto-escalated to expert review queue | ||
| 135 | ** User warnings displayed | ||
| 136 | |||
| 137 | **Audit rate:** Recommendation: 30-50% of published AI-drafts sampled in first 6 months | ||
| 138 | |||
| 139 | === Tier B — Medium Risk === | ||
| 140 | |||
| 141 | **Domains:** Contested public policy, complex science, causality claims, significant financial impact | ||
| 142 | |||
| 143 | **Publication policy:** | ||
| 144 | * AI-draft CAN publish immediately with clear labeling | ||
| 145 | * Sampling audits conducted (see Audit System below) | ||
| 146 | * High-engagement items auto-escalated to expert review | ||
| 147 | * Users can request human review | ||
| 148 | |||
| 149 | **Audit rate:** Recommendation: 10-20% of published AI-drafts sampled | ||
| 150 | |||
| 151 | === Tier C — Low Risk === | ||
| 152 | |||
| 153 | **Domains:** Definitions, simple factual lookups with strong primary sources, historical facts, established scientific consensus | ||
| 154 | |||
| 155 | **Publication policy:** | ||
| 156 | * AI-draft default publication mode | ||
| 157 | * Sampling audits sufficient | ||
| 158 | * Community flagging available | ||
| 159 | * Human review on request | ||
| 160 | |||
| 161 | **Audit rate:** Recommendation: 5-10% of published AI-drafts sampled | ||
| 162 | |||
| 163 | ---- | ||
| 164 | |||
| 165 | == Quality Gates (Mandatory Before AI-Draft Publication) == | ||
| 166 | |||
| 167 | All AI-generated content must pass these automated checks before Mode 2 publication: | ||
| 168 | |||
| 169 | === Gate 1: Source Quality === | ||
| 170 | * Primary sources identified and accessible | ||
| 171 | * Source reliability scored against whitelist | ||
| 172 | * Citation completeness verified | ||
| 173 | * Publication dates checked | ||
| 174 | * Author credentials validated (where applicable) | ||
| 175 | |||
| 176 | === Gate 2: Contradiction Search (MANDATORY) === | ||
| 177 | |||
| 178 | **The system MUST actively search for:** | ||
| 179 | |||
| 180 | * **Counter-evidence** – Rebuttals, conflicting results, contradictory studies | ||
| 181 | * **Reservations** – Caveats, limitations, boundary conditions, applicability constraints | ||
| 182 | * **Alternative interpretations** – Different framings, definitions, contextual variations | ||
| 183 | * **Bubble detection** – Conspiracy theories, echo chambers, ideologically isolated sources | ||
| 184 | |||
| 185 | **Search coverage requirements:** | ||
| 186 | * Academic literature (BOTH supporting AND opposing views) | ||
| 187 | * Reputable media across diverse political/ideological perspectives | ||
| 188 | * Official contradictions (retractions, corrections, updates, amendments) | ||
| 189 | * Domain-specific skeptics, critics, and alternative expert opinions | ||
| 190 | * Cross-cultural and international perspectives | ||
| 191 | |||
| 192 | **Search must actively avoid algorithmic bubbles:** | ||
| 193 | * Deliberately seek opposing viewpoints | ||
| 194 | * Check for echo chamber patterns in source clusters | ||
| 195 | * Identify tribal or ideological source clustering | ||
| 196 | * Flag when search space appears artificially constrained | ||
| 197 | * Verify diversity of perspectives represented | ||
| 198 | |||
| 199 | **Outcomes:** | ||
| 200 | * **Strong counter-evidence found** → Auto-escalate to Tier B or draft-only mode | ||
| 201 | * **Significant uncertainty detected** → Require uncertainty disclosure in verdict | ||
| 202 | * **Bubble indicators present** → Flag for expert review and human validation | ||
| 203 | * **Limited perspective diversity** → Expand search or flag for human review | ||
| 204 | |||
| 205 | === Gate 3: Uncertainty Quantification === | ||
| 206 | * Confidence scores calculated for all claims and verdicts | ||
| 207 | * Limitations explicitly stated | ||
| 208 | * Data gaps identified and disclosed | ||
| 209 | * Strength of evidence assessed | ||
| 210 | * Alternative scenarios considered | ||
| 211 | |||
| 212 | === Gate 4: Structural Integrity === | ||
| 213 | * No hallucinations detected (fact-checking against sources) | ||
| 214 | * Logic chain valid and traceable | ||
| 215 | * References accessible and verifiable | ||
| 216 | * No circular reasoning | ||
| 217 | * Premises clearly stated | ||
| 218 | |||
| 219 | **If any gate fails:** | ||
| 220 | * Content remains in draft-only mode | ||
| 221 | * Failure reason logged | ||
| 222 | * Human review required before publication | ||
| 223 | * Failure patterns analyzed for system improvement | ||
| 224 | |||
| 225 | ---- | ||
| 226 | |||
| 227 | == Audit System (Sampling-Based Quality Assurance) == | ||
| 228 | |||
| 229 | Instead of reviewing ALL AI output, FactHarbor implements stratified sampling audits: | ||
| 230 | |||
| 231 | === Sampling Strategy === | ||
| 232 | |||
| 233 | Audits prioritize: | ||
| 234 | * **Risk tier** (higher tiers get more frequent audits) | ||
| 235 | * **AI confidence score** (low confidence → higher sampling rate) | ||
| 236 | * **Traffic and engagement** (high-visibility content audited more) | ||
| 237 | * **Novelty** (new claim types, new domains, emerging topics) | ||
| 238 | * **Disagreement signals** (user flags, contradiction alerts, community reports) | ||
| 239 | |||
| 240 | === Audit Process === | ||
| 241 | |||
| 242 | 1. System selects content for audit based on sampling strategy | ||
| 243 | 2. Human auditor reviews AI-generated content against quality standards | ||
| 244 | 3. Auditor validates or corrects: | ||
| 245 | * Claim extraction accuracy | ||
| 246 | * Scenario appropriateness | ||
| 247 | * Evidence relevance and interpretation | ||
| 248 | * Verdict reasoning | ||
| 249 | * Contradiction search completeness | ||
| 250 | 4. Audit outcome recorded (pass/fail + detailed feedback) | ||
| 251 | 5. Failed audits trigger immediate content review | ||
| 252 | 6. Audit results feed back into system improvement | ||
| 253 | |||
| 254 | === Feedback Loop (Continuous Improvement) === | ||
| 255 | |||
| 256 | Audit outcomes systematically improve: | ||
| 257 | * **Query templates** – Refined based on missed evidence patterns | ||
| 258 | * **Retrieval source weights** – Adjusted for accuracy and reliability | ||
| 259 | * **Contradiction detection heuristics** – Enhanced to catch missed counter-evidence | ||
| 260 | * **Model prompts and extraction rules** – Tuned for better claim extraction | ||
| 261 | * **Risk tier assignments** – Recalibrated based on error patterns | ||
| 262 | * **Bubble detection algorithms** – Improved to identify echo chambers | ||
| 263 | |||
| 264 | === Audit Transparency === | ||
| 265 | |||
| 266 | * Audit statistics published regularly | ||
| 267 | * Accuracy rates by risk tier tracked and reported | ||
| 268 | * System improvements documented | ||
| 269 | * Community can view aggregate audit performance | ||
| 270 | |||
| 271 | ---- | ||
| 272 | |||
| |
1.1 | 273 | == Architecture Overview == |
| 274 | |||
| |
2.1 | 275 | {{include reference="FactHarbor.Specification.Diagrams.AKEL Architecture.WebHome"/}} |
| |
1.1 | 276 | |
| |
4.1 | 277 | ---- |
| 278 | |||
| |
1.1 | 279 | == AKEL and Federation == |
| 280 | |||
| 281 | In Release 1.0+, AKEL participates in cross-node knowledge alignment: | ||
| 282 | |||
| |
4.1 | 283 | * Shares embeddings |
| 284 | * Exchanges canonicalized claim forms | ||
| 285 | * Exchanges scenario templates | ||
| 286 | * Sends + receives contradiction alerts | ||
| |
5.1 | 287 | * Shares audit findings (with privacy controls) |
| |
4.1 | 288 | * Never shares model weights |
| |
1.1 | 289 | * Never overrides local governance |
| 290 | |||
| 291 | Nodes may choose trust levels for AKEL-related data: | ||
| 292 | |||
| |
4.1 | 293 | * Trusted nodes: auto-merge embeddings + templates |
| 294 | * Neutral nodes: require reviewer approval | ||
| |
1.1 | 295 | * Untrusted nodes: fully manual import |
| 296 | |||
| |
4.1 | 297 | ---- |
| 298 | |||
| |
5.1 | 299 | == Human Review Workflow (Mode 3 Publication) == |
| |
1.1 | 300 | |
| |
5.1 | 301 | For content requiring human validation before "Human-Reviewed" status: |
| |
1.1 | 302 | |
| |
5.1 | 303 | 1. AKEL generates content and publishes as AI-draft (Mode 2) or keeps as draft (Mode 1) |
| 304 | 2. Reviewers inspect content in review queue | ||
| 305 | 3. Reviewers validate quality gates were correctly applied | ||
| 306 | 4. Experts validate high-risk (Tier A) or domain-specific outputs | ||
| 307 | 5. Moderators finalize "Human-Reviewed" publication | ||
| 308 | 6. Version numbers increment, full history preserved | ||
| |
4.1 | 309 | |
| |
5.1 | 310 | **Note:** Most AI-generated content (Tier B and C) can remain in Mode 2 (AI-Generated) indefinitely. Human review is optional for these tiers unless users or audits flag issues. |
| 311 | |||
| |
4.1 | 312 | ---- |
| |
5.1 | 313 | |
| 314 | == POC v1 Behavior == | ||
| 315 | |||
| 316 | The POC explicitly demonstrates AI-generated content publication: | ||
| 317 | |||
| 318 | * Produces public AI-generated output (Mode 2) | ||
| 319 | * No human data sources required | ||
| 320 | * No human approval gate | ||
| 321 | * Clear "AI-Generated - POC/Demo" labeling | ||
| 322 | * All quality gates active (including contradiction search) | ||
| 323 | * Users understand this demonstrates AI reasoning capabilities | ||
| 324 | * Risk tier classification shown (demo purposes) | ||
| 325 | |||
| 326 | ---- | ||
| 327 | |||
| 328 | == Related Pages == | ||
| 329 | |||
| 330 | * [[Automation>>FactHarbor.Specification.Automation.WebHome]] | ||
| 331 | * [[Requirements (Roles)>>FactHarbor.Specification.Requirements.WebHome]] | ||
| 332 | * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] | ||
| 333 | * [[Governance>>FactHarbor.Organisation.Governance]] | ||
| 334 |