Changes for page AI Knowledge Extraction Layer (AKEL)

Last modified by Robert Schaub on 2025/12/24 20:33

From 4.1 to 3.1 From 6.4 to 6.3

From version 6.3

edited by Robert Schaub
on 2025/12/16 20:26

Change comment: Update document after refactoring.

To version 4.1

edited by Robert Schaub
on 2025/12/12 15:41

Change comment: Imported from XAR

Raw
Rendered

Summary

Page properties (2 modified, 0 added, 0 removed)

Details

Page properties

Parent

@@ -1,1 +1,1 @@
--FactHarbor.Archive.FactHarbor V0\.9\.18.Specification.WebHome
++FactHarbor.Specification.WebHome

Content

@@ -1,9 +1,9 @@
  = AKEL — AI Knowledge Extraction Layer =
--AKEL is FactHarbor's automated intelligence subsystem.
++AKEL is FactHarbor’s automated intelligence subsystem.
  Its purpose is to reduce human workload, enhance consistency, and enable scalable knowledge processing — **without ever replacing human judgment**.
--AKEL outputs are marked with **AuthorType = AI** and published according to risk-based review policies (see Publication Modes below).
++All AKEL outputs are marked with **AuthorType = AI** and require human approval before publication.
  AKEL operates in two modes:
@@ -10,23 +10,21 @@
  * **Single-node mode** (POC & Beta 0)
  * **Federated multi-node mode** (Release 1.0+)
--Human reviewers, experts, and moderators always retain final authority over content marked as "Human-Reviewed."
++Human reviewers, experts, and moderators always retain final authority.
  ----
  == Purpose and Role ==
--AKEL transforms unstructured inputs into structured, publication-ready content.
++AKEL transforms unstructured inputs into structured, review-ready drafts.
  Core responsibilities:
  * Claim extraction from arbitrary text
--* Claim classification (domain, type, evaluability, safety, **risk tier**)
++* Claim classification (domain, type, evaluability, safety)
  * Scenario generation (definitions, boundaries, assumptions, methodology)
  * Evidence summarization and metadata extraction
--* **Contradiction detection and counter-evidence search**
--* **Reservation and limitation identification**
--* **Bubble detection** (echo chambers, conspiracy theories, isolated sources)
++* Contradiction detection
  * Re-evaluation proposal generation
  * Cross-node embedding exchange (Release 1.0+)
@@ -36,12 +36,10 @@
  * **AKEL Orchestrator** – central coordinator
  * **Claim Extractor**
--* **Claim Classifier** (with risk tier assignment)
++* **Claim Classifier**
  * **Scenario Generator**
  * **Evidence Summarizer**
--* **Contradiction Detector** (enhanced with counter-evidence search)
--* **Quality Gate Validator**
--* **Audit Sampling Scheduler**
++* **Contradiction Detector**
  * **Embedding Handler** (Release 1.0+)
  * **Federation Sync Adapter** (Release 1.0+)
@@ -56,223 +56,20 @@
  * External LLM API (optional)
  * Embeddings (from local or federated peers)
--=== Outputs (publication mode varies by risk tier) ===
--* ClaimVersion (draft or AI-generated)
--* ScenarioVersion (draft or AI-generated)
--* EvidenceVersion (summary + metadata, draft or AI-generated)
--* VerdictVersion (draft, AI-generated, or human-reviewed)
++=== Outputs (all require human approval) ===
++* ClaimVersion (draft)
++* ScenarioVersion (draft)
++* EvidenceVersion (summary + metadata draft)
++* VerdictVersion (draft; internal only)
  * Contradiction alerts
--* Reservation and limitation notices
  * Re-evaluation proposals
  * Updated embeddings
  ----
--== Publication Modes ==
--
--AKEL content is published according to three modes:
--
--=== Mode 1: Draft-Only (Never Public) ===
--
--**Used for:**
--* Failed quality gate checks
--* Sensitive topics flagged for expert review
--* Unclear scope or missing critical sources
--* High reputational risk content
--
--**Visibility:** Internal review queue only
--
--=== Mode 2: Published as AI-Generated (No Prior Human Review) ===
--
--**Requirements:**
--* All automated quality gates passed (see below)
--* Risk tier permits AI-draft publication (Tier B or C)
--* Contradiction search completed successfully
--* Clear labeling as "AI-Generated, Awaiting Human Review"
--
--**Label shown to users:**
--```
--[AI-Generated] This content was produced by AI and has not yet been human-reviewed.
--Source: AI | Review Status: Pending | Risk Tier: [B/C]
--Contradiction Search: Completed | Last Updated: [timestamp]
--```
--
--**User actions:**
--* Browse and read content
--* Request human review (escalates to review queue)
--* Flag for expert attention
--
--=== Mode 3: Published as Human-Reviewed ===
--
--**Requirements:**
--* Human reviewer or domain expert validated
--* All quality gates passed
--* Visible "Human-Reviewed" mark with reviewer role and timestamp
--
--**Label shown to users:**
--```
--[Human-Reviewed] This content has been validated by human reviewers.
--Source: AI+Human | Review Status: Approved | Reviewed by: [Role] on [timestamp]
--Risk Tier: [A/B/C] | Contradiction Search: Completed
--```
--
------
--
--== Risk Tiers ==
--
--AKEL assigns risk tiers to all content to determine appropriate review requirements:
--
--=== Tier A — High Risk / High Impact ===
--
--**Domains:** Medical, legal, elections, safety/security, major reputational harm
--
--**Publication policy:**
--* Human review REQUIRED before "Human-Reviewed" status
--* AI-generated content MAY be published but:
--  ** Clearly flagged as AI-draft with prominent disclaimer
--  ** May have limited visibility
--  ** Auto-escalated to expert review queue
--  ** User warnings displayed
--
--**Audit rate:** Recommendation: 30-50% of published AI-drafts sampled in first 6 months
--
--=== Tier B — Medium Risk ===
--
--**Domains:** Contested public policy, complex science, causality claims, significant financial impact
--
--**Publication policy:**
--* AI-draft CAN publish immediately with clear labeling
--* Sampling audits conducted (see Audit System below)
--* High-engagement items auto-escalated to expert review
--* Users can request human review
--
--**Audit rate:** Recommendation: 10-20% of published AI-drafts sampled
--
--=== Tier C — Low Risk ===
--
--**Domains:** Definitions, simple factual lookups with strong primary sources, historical facts, established scientific consensus
--
--**Publication policy:**
--* AI-draft default publication mode
--* Sampling audits sufficient
--* Community flagging available
--* Human review on request
--
--**Audit rate:** Recommendation: 5-10% of published AI-drafts sampled
--
------
--
--== Quality Gates (Mandatory Before AI-Draft Publication) ==
--
--All AI-generated content must pass these automated checks before Mode 2 publication:
--
--=== Gate 1: Source Quality ===
--* Primary sources identified and accessible
--* Source reliability scored against whitelist
--* Citation completeness verified
--* Publication dates checked
--* Author credentials validated (where applicable)
--
--=== Gate 2: Contradiction Search (MANDATORY) ===
--
--**The system MUST actively search for:**
--
--* **Counter-evidence** – Rebuttals, conflicting results, contradictory studies
--* **Reservations** – Caveats, limitations, boundary conditions, applicability constraints
--* **Alternative interpretations** – Different framings, definitions, contextual variations
--* **Bubble detection** – Conspiracy theories, echo chambers, ideologically isolated sources
--
--**Search coverage requirements:**
--* Academic literature (BOTH supporting AND opposing views)
--* Reputable media across diverse political/ideological perspectives
--* Official contradictions (retractions, corrections, updates, amendments)
--* Domain-specific skeptics, critics, and alternative expert opinions
--* Cross-cultural and international perspectives
--
--**Search must actively avoid algorithmic bubbles:**
--* Deliberately seek opposing viewpoints
--* Check for echo chamber patterns in source clusters
--* Identify tribal or ideological source clustering
--* Flag when search space appears artificially constrained
--* Verify diversity of perspectives represented
--
--**Outcomes:**
--* **Strong counter-evidence found** → Auto-escalate to Tier B or draft-only mode
--* **Significant uncertainty detected** → Require uncertainty disclosure in verdict
--* **Bubble indicators present** → Flag for expert review and human validation
--* **Limited perspective diversity** → Expand search or flag for human review
--
--=== Gate 3: Uncertainty Quantification ===
--* Confidence scores calculated for all claims and verdicts
--* Limitations explicitly stated
--* Data gaps identified and disclosed
--* Strength of evidence assessed
--* Alternative scenarios considered
--
--=== Gate 4: Structural Integrity ===
--* No hallucinations detected (fact-checking against sources)
--* Logic chain valid and traceable
--* References accessible and verifiable
--* No circular reasoning
--* Premises clearly stated
--
--**If any gate fails:**
--* Content remains in draft-only mode
--* Failure reason logged
--* Human review required before publication
--* Failure patterns analyzed for system improvement
--
------
--
--== Audit System (Sampling-Based Quality Assurance) ==
--
--Instead of reviewing ALL AI output, FactHarbor implements stratified sampling audits:
--
--=== Sampling Strategy ===
--
--Audits prioritize:
--* **Risk tier** (higher tiers get more frequent audits)
--* **AI confidence score** (low confidence → higher sampling rate)
--* **Traffic and engagement** (high-visibility content audited more)
--* **Novelty** (new claim types, new domains, emerging topics)
--* **Disagreement signals** (user flags, contradiction alerts, community reports)
--
--=== Audit Process ===
--
--1. System selects content for audit based on sampling strategy
--2. Human auditor reviews AI-generated content against quality standards
--3. Auditor validates or corrects:
--   * Claim extraction accuracy
--   * Scenario appropriateness
--   * Evidence relevance and interpretation
--   * Verdict reasoning
--   * Contradiction search completeness
--4. Audit outcome recorded (pass/fail + detailed feedback)
--5. Failed audits trigger immediate content review
--6. Audit results feed back into system improvement
--
--=== Feedback Loop (Continuous Improvement) ===
--
--Audit outcomes systematically improve:
--* **Query templates** – Refined based on missed evidence patterns
--* **Retrieval source weights** – Adjusted for accuracy and reliability
--* **Contradiction detection heuristics** – Enhanced to catch missed counter-evidence
--* **Model prompts and extraction rules** – Tuned for better claim extraction
--* **Risk tier assignments** – Recalibrated based on error patterns
--* **Bubble detection algorithms** – Improved to identify echo chambers
--
--=== Audit Transparency ===
--
--* Audit statistics published regularly
--* Accuracy rates by risk tier tracked and reported
--* System improvements documented
--* Community can view aggregate audit performance
--
------
--
  == Architecture Overview ==
--{{include reference="Test.FactHarborV09.Specification.Diagrams.AKEL Architecture.WebHome"/}}
++{{include reference="FactHarbor.Specification.Diagrams.AKEL Architecture.WebHome"/}}
  ----
@@ -284,7 +284,6 @@
  * Exchanges canonicalized claim forms
  * Exchanges scenario templates
  * Sends + receives contradiction alerts
--* Shares audit findings (with privacy controls)
  * Never shares model weights
  * Never overrides local governance
@@ -296,39 +296,14 @@
  ----
--== Human Review Workflow (Mode 3 Publication) ==
++== Human Approval Workflow ==
--For content requiring human validation before "Human-Reviewed" status:
++1. AKEL generates draft outputs (AuthorType = AI)
++2. Reviewers inspect and approve/moderate the drafts
++3. Experts validate high-risk or domain-specific outputs
++4. Moderators finalize publication
++5. Version numbers increment, history preserved
--1. AKEL generates content and publishes as AI-draft (Mode 2) or keeps as draft (Mode 1)
--2. Reviewers inspect content in review queue
--3. Reviewers validate quality gates were correctly applied
--4. Experts validate high-risk (Tier A) or domain-specific outputs
--5. Moderators finalize "Human-Reviewed" publication
--6. Version numbers increment, full history preserved
++No AKEL output is ever published automatically.
--**Note:** Most AI-generated content (Tier B and C) can remain in Mode 2 (AI-Generated) indefinitely. Human review is optional for these tiers unless users or audits flag issues.
--
  ----
--
--== POC v1 Behavior ==
--
--The POC explicitly demonstrates AI-generated content publication:
--
--* Produces public AI-generated output (Mode 2)
--* No human data sources required
--* No human approval gate
--* Clear "AI-Generated - POC/Demo" labeling
--* All quality gates active (including contradiction search)
--* Users understand this demonstrates AI reasoning capabilities
--* Risk tier classification shown (demo purposes)
--
------
--
--== Related Pages ==
--
--* [[Automation>>FactHarbor.Specification.Automation.WebHome]]
--* [[Requirements (Roles)>>FactHarbor.Specification.Requirements.WebHome]]
--* [[Workflows>>FactHarbor.Specification.Workflows.WebHome]]
--* [[Governance>>FactHarbor.Organisation.Governance]]
--

Changes for page AI Knowledge Extraction Layer (AKEL)

Summary

Details

Applications

Navigation

Need help?