Skip to Content
Last modified by Robert Schaub on 2025/12/24 20:33
From version 1.2
edited by Robert Schaub
on 2025/12/11 21:34
Change comment: Renamed back-links.
To version 6.5
edited by Robert Schaub
on 2025/12/16 21:40
Change comment: Renamed back-links.

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -FactHarbor.Specification.WebHome
1 +FactHarbor.Specification V0\.9\.18.WebHome
Content
... ... @@ -1,9 +1,9 @@
1 1  = AKEL — AI Knowledge Extraction Layer =
2 2  
3 -AKEL is FactHarbor’s automated intelligence subsystem.
3 +AKEL is FactHarbor's automated intelligence subsystem.
4 4  Its purpose is to reduce human workload, enhance consistency, and enable scalable knowledge processing — **without ever replacing human judgment**.
5 5  
6 -All AKEL outputs are marked with **AuthorType = AI** and require human approval before publication.
6 +AKEL outputs are marked with **AuthorType = AI** and published according to risk-based review policies (see Publication Modes below).
7 7  
8 8  AKEL operates in two modes:
9 9  
... ... @@ -10,21 +10,23 @@
10 10  * **Single-node mode** (POC & Beta 0)
11 11  * **Federated multi-node mode** (Release 1.0+)
12 12  
13 -Human reviewers, experts, and moderators always retain final authority.
13 +Human reviewers, experts, and moderators always retain final authority over content marked as "Human-Reviewed."
14 14  
15 15  ----
16 16  
17 17  == Purpose and Role ==
18 18  
19 -AKEL transforms unstructured inputs into structured, review-ready drafts.
19 +AKEL transforms unstructured inputs into structured, publication-ready content.
20 20  
21 21  Core responsibilities:
22 22  
23 23  * Claim extraction from arbitrary text
24 -* Claim classification (domain, type, evaluability, safety)
24 +* Claim classification (domain, type, evaluability, safety, **risk tier**)
25 25  * Scenario generation (definitions, boundaries, assumptions, methodology)
26 26  * Evidence summarization and metadata extraction
27 -* Contradiction detection
27 +* **Contradiction detection and counter-evidence search**
28 +* **Reservation and limitation identification**
29 +* **Bubble detection** (echo chambers, conspiracy theories, isolated sources)
28 28  * Re-evaluation proposal generation
29 29  * Cross-node embedding exchange (Release 1.0+)
30 30  
... ... @@ -34,10 +34,12 @@
34 34  
35 35  * **AKEL Orchestrator** – central coordinator
36 36  * **Claim Extractor**
37 -* **Claim Classifier**
39 +* **Claim Classifier** (with risk tier assignment)
38 38  * **Scenario Generator**
39 39  * **Evidence Summarizer**
40 -* **Contradiction Detector**
42 +* **Contradiction Detector** (enhanced with counter-evidence search)
43 +* **Quality Gate Validator**
44 +* **Audit Sampling Scheduler**
41 41  * **Embedding Handler** (Release 1.0+)
42 42  * **Federation Sync Adapter** (Release 1.0+)
43 43  
... ... @@ -53,21 +53,241 @@
53 53  * External LLM API (optional)
54 54  * Embeddings (from local or federated peers)
55 55  
56 -=== Outputs (all require human approval) ===
60 +=== Outputs (publication mode varies by risk tier) ===
57 57  
58 -* ClaimVersion (draft)
59 -* ScenarioVersion (draft)
60 -* EvidenceVersion (summary + metadata draft)
61 -* VerdictVersion (draft; internal only)
62 +* ClaimVersion (draft or AI-generated)
63 +* ScenarioVersion (draft or AI-generated)
64 +* EvidenceVersion (summary + metadata, draft or AI-generated)
65 +* VerdictVersion (draft, AI-generated, or human-reviewed)
62 62  * Contradiction alerts
67 +* Reservation and limitation notices
63 63  * Re-evaluation proposals
64 64  * Updated embeddings
65 65  
66 66  ----
67 67  
73 +== Publication Modes ==
74 +
75 +AKEL content is published according to three modes:
76 +
77 +=== Mode 1: Draft-Only (Never Public) ===
78 +
79 +**Used for:**
80 +
81 +* Failed quality gate checks
82 +* Sensitive topics flagged for expert review
83 +* Unclear scope or missing critical sources
84 +* High reputational risk content
85 +
86 +**Visibility:** Internal review queue only
87 +
88 +=== Mode 2: Published as AI-Generated (No Prior Human Review) ===
89 +
90 +**Requirements:**
91 +
92 +* All automated quality gates passed (see below)
93 +* Risk tier permits AI-draft publication (Tier B or C)
94 +* Contradiction search completed successfully
95 +* Clear labeling as "AI-Generated, Awaiting Human Review"
96 +
97 +**Label shown to users:**
98 +```
99 +[AI-Generated] This content was produced by AI and has not yet been human-reviewed.
100 +Source: AI | Review Status: Pending | Risk Tier: [B/C]
101 +Contradiction Search: Completed | Last Updated: [timestamp]
102 +```
103 +
104 +**User actions:**
105 +
106 +* Browse and read content
107 +* Request human review (escalates to review queue)
108 +* Flag for expert attention
109 +
110 +=== Mode 3: Published as Human-Reviewed ===
111 +
112 +**Requirements:**
113 +
114 +* Human reviewer or domain expert validated
115 +* All quality gates passed
116 +* Visible "Human-Reviewed" mark with reviewer role and timestamp
117 +
118 +**Label shown to users:**
119 +```
120 +[Human-Reviewed] This content has been validated by human reviewers.
121 +Source: AI+Human | Review Status: Approved | Reviewed by: [Role] on [timestamp]
122 +Risk Tier: [A/B/C] | Contradiction Search: Completed
123 +```
124 +
125 +----
126 +
127 +== Risk Tiers ==
128 +
129 +AKEL assigns risk tiers to all content to determine appropriate review requirements:
130 +
131 +=== Tier A — High Risk / High Impact ===
132 +
133 +**Domains:** Medical, legal, elections, safety/security, major reputational harm
134 +
135 +**Publication policy:**
136 +
137 +* Human review REQUIRED before "Human-Reviewed" status
138 +* AI-generated content MAY be published but:
139 +** Clearly flagged as AI-draft with prominent disclaimer
140 +** May have limited visibility
141 +** Auto-escalated to expert review queue
142 +** User warnings displayed
143 +
144 +**Audit rate:** Recommendation: 30-50% of published AI-drafts sampled in first 6 months
145 +
146 +=== Tier B — Medium Risk ===
147 +
148 +**Domains:** Contested public policy, complex science, causality claims, significant financial impact
149 +
150 +**Publication policy:**
151 +
152 +* AI-draft CAN publish immediately with clear labeling
153 +* Sampling audits conducted (see Audit System below)
154 +* High-engagement items auto-escalated to expert review
155 +* Users can request human review
156 +
157 +**Audit rate:** Recommendation: 10-20% of published AI-drafts sampled
158 +
159 +=== Tier C — Low Risk ===
160 +
161 +**Domains:** Definitions, simple factual lookups with strong primary sources, historical facts, established scientific consensus
162 +
163 +**Publication policy:**
164 +
165 +* AI-draft default publication mode
166 +* Sampling audits sufficient
167 +* Community flagging available
168 +* Human review on request
169 +
170 +**Audit rate:** Recommendation: 5-10% of published AI-drafts sampled
171 +
172 +----
173 +
174 +== Quality Gates (Mandatory Before AI-Draft Publication) ==
175 +
176 +All AI-generated content must pass these automated checks before Mode 2 publication:
177 +
178 +=== Gate 1: Source Quality ===
179 +
180 +* Primary sources identified and accessible
181 +* Source reliability scored against whitelist
182 +* Citation completeness verified
183 +* Publication dates checked
184 +* Author credentials validated (where applicable)
185 +
186 +=== Gate 2: Contradiction Search (MANDATORY) ===
187 +
188 +**The system MUST actively search for:**
189 +
190 +* **Counter-evidence** – Rebuttals, conflicting results, contradictory studies
191 +* **Reservations** – Caveats, limitations, boundary conditions, applicability constraints
192 +* **Alternative interpretations** – Different framings, definitions, contextual variations
193 +* **Bubble detection** – Conspiracy theories, echo chambers, ideologically isolated sources
194 +
195 +**Search coverage requirements:**
196 +
197 +* Academic literature (BOTH supporting AND opposing views)
198 +* Reputable media across diverse political/ideological perspectives
199 +* Official contradictions (retractions, corrections, updates, amendments)
200 +* Domain-specific skeptics, critics, and alternative expert opinions
201 +* Cross-cultural and international perspectives
202 +
203 +**Search must actively avoid algorithmic bubbles:**
204 +
205 +* Deliberately seek opposing viewpoints
206 +* Check for echo chamber patterns in source clusters
207 +* Identify tribal or ideological source clustering
208 +* Flag when search space appears artificially constrained
209 +* Verify diversity of perspectives represented
210 +
211 +**Outcomes:**
212 +
213 +* **Strong counter-evidence found** → Auto-escalate to Tier B or draft-only mode
214 +* **Significant uncertainty detected** → Require uncertainty disclosure in verdict
215 +* **Bubble indicators present** → Flag for expert review and human validation
216 +* **Limited perspective diversity** → Expand search or flag for human review
217 +
218 +=== Gate 3: Uncertainty Quantification ===
219 +
220 +* Confidence scores calculated for all claims and verdicts
221 +* Limitations explicitly stated
222 +* Data gaps identified and disclosed
223 +* Strength of evidence assessed
224 +* Alternative scenarios considered
225 +
226 +=== Gate 4: Structural Integrity ===
227 +
228 +* No hallucinations detected (fact-checking against sources)
229 +* Logic chain valid and traceable
230 +* References accessible and verifiable
231 +* No circular reasoning
232 +* Premises clearly stated
233 +
234 +**If any gate fails:**
235 +
236 +* Content remains in draft-only mode
237 +* Failure reason logged
238 +* Human review required before publication
239 +* Failure patterns analyzed for system improvement
240 +
241 +----
242 +
243 +== Audit System (Sampling-Based Quality Assurance) ==
244 +
245 +Instead of reviewing ALL AI output, FactHarbor implements stratified sampling audits:
246 +
247 +=== Sampling Strategy ===
248 +
249 +Audits prioritize:
250 +
251 +* **Risk tier** (higher tiers get more frequent audits)
252 +* **AI confidence score** (low confidence → higher sampling rate)
253 +* **Traffic and engagement** (high-visibility content audited more)
254 +* **Novelty** (new claim types, new domains, emerging topics)
255 +* **Disagreement signals** (user flags, contradiction alerts, community reports)
256 +
257 +=== Audit Process ===
258 +
259 +1. System selects content for audit based on sampling strategy
260 +2. Human auditor reviews AI-generated content against quality standards
261 +3. Auditor validates or corrects:
262 +
263 +* Claim extraction accuracy
264 +* Scenario appropriateness
265 +* Evidence relevance and interpretation
266 +* Verdict reasoning
267 +* Contradiction search completeness
268 +4. Audit outcome recorded (pass/fail + detailed feedback)
269 +5. Failed audits trigger immediate content review
270 +6. Audit results feed back into system improvement
271 +
272 +=== Feedback Loop (Continuous Improvement) ===
273 +
274 +Audit outcomes systematically improve:
275 +
276 +* **Query templates** – Refined based on missed evidence patterns
277 +* **Retrieval source weights** – Adjusted for accuracy and reliability
278 +* **Contradiction detection heuristics** – Enhanced to catch missed counter-evidence
279 +* **Model prompts and extraction rules** – Tuned for better claim extraction
280 +* **Risk tier assignments** – Recalibrated based on error patterns
281 +* **Bubble detection algorithms** – Improved to identify echo chambers
282 +
283 +=== Audit Transparency ===
284 +
285 +* Audit statistics published regularly
286 +* Accuracy rates by risk tier tracked and reported
287 +* System improvements documented
288 +* Community can view aggregate audit performance
289 +
290 +----
291 +
68 68  == Architecture Overview ==
69 69  
70 -{{include reference="FactHarbor.Archive.Diagrams v0\.8q.AKEL Architecture.WebHome"/}}
294 +{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.AKEL Architecture.WebHome"/}}
71 71  
72 72  ----
73 73  
... ... @@ -79,6 +79,7 @@
79 79  * Exchanges canonicalized claim forms
80 80  * Exchanges scenario templates
81 81  * Sends + receives contradiction alerts
306 +* Shares audit findings (with privacy controls)
82 82  * Never shares model weights
83 83  * Never overrides local governance
84 84  
... ... @@ -90,14 +90,38 @@
90 90  
91 91  ----
92 92  
93 -== Human Approval Workflow ==
318 +== Human Review Workflow (Mode 3 Publication) ==
94 94  
95 -1. AKEL generates draft outputs (AuthorType = AI)
96 -2. Reviewers inspect and approve/moderate the drafts
97 -3. Experts validate high-risk or domain-specific outputs
98 -4. Moderators finalize publication
99 -5. Version numbers increment, history preserved
320 +For content requiring human validation before "Human-Reviewed" status:
100 100  
101 -No AKEL output is ever published automatically.
322 +1. AKEL generates content and publishes as AI-draft (Mode 2) or keeps as draft (Mode 1)
323 +2. Reviewers inspect content in review queue
324 +3. Reviewers validate quality gates were correctly applied
325 +4. Experts validate high-risk (Tier A) or domain-specific outputs
326 +5. Moderators finalize "Human-Reviewed" publication
327 +6. Version numbers increment, full history preserved
102 102  
329 +**Note:** Most AI-generated content (Tier B and C) can remain in Mode 2 (AI-Generated) indefinitely. Human review is optional for these tiers unless users or audits flag issues.
330 +
103 103  ----
332 +
333 +== POC v1 Behavior ==
334 +
335 +The POC explicitly demonstrates AI-generated content publication:
336 +
337 +* Produces public AI-generated output (Mode 2)
338 +* No human data sources required
339 +* No human approval gate
340 +* Clear "AI-Generated - POC/Demo" labeling
341 +* All quality gates active (including contradiction search)
342 +* Users understand this demonstrates AI reasoning capabilities
343 +* Risk tier classification shown (demo purposes)
344 +
345 +----
346 +
347 +== Related Pages ==
348 +
349 +* [[Automation>>FactHarbor.Specification V0\.9\.18.Automation.WebHome]]
350 +* [[Requirements (Roles)>>FactHarbor.Specification V0\.9\.18.Requirements.WebHome]]
351 +* [[Workflows>>FactHarbor.Specification.Workflows.WebHome]]
352 +* [[Governance>>FactHarbor.Organisation.Governance]]