Changes for page Automation

Last modified by Robert Schaub on 2025/12/24 20:34

From 1.3 to 1.2 From 7.4 to 7.3

From version 7.3

edited by Robert Schaub
on 2025/12/16 20:26

Change comment: Update document after refactoring.

To version 1.3

edited by Robert Schaub
on 2025/12/11 21:34

Change comment: Renamed back-links.

Raw
Rendered

Summary

Page properties (2 modified, 0 added, 0 removed)

Details

Page properties

Parent

@@ -1,1 +1,1 @@
--FactHarbor.Archive.FactHarbor V0\.9\.18.Specification.WebHome
++FactHarbor.Specification.WebHome

Content

@@ -1,271 +1,167 @@
  = Automation =
--Automation in FactHarbor amplifies human capability while implementing risk-based oversight.
++Automation in FactHarbor amplifies human capability but never replaces human oversight.
++All automated outputs require human review before publication.
  This chapter defines:
--* Risk-based publication model
--* Quality gates for AI-generated content
++
  * What must remain human-only
--* What AI (AKEL) can draft and publish
++* What AI (AKEL) can draft
  * What can be fully automated
  * How automation evolves through POC → Beta 0 → Release 1.0
--== POC v1 (AI-Generated Publication Demonstration) ==
--
--The goal of POC v1 is to validate the automated reasoning capabilities and demonstrate AI-generated content publication.
--
--=== Workflow ===
--
--1. **Input**: User pastes a block of raw text.
--1. **Deep Analysis (Background)**: The system autonomously performs the full pipeline **before** displaying the text:
--* Extraction & Normalisation
--* Scenario & Sub-query generation
--* Evidence retrieval with **contradiction search**
--* Quality gate validation
--* Verdict computation
--1. **Visualisation (Extraction & Marking)**: The system displays the text with claims extracted and marked.
--* **Verdict-Based Coloring**: The extraction highlights (e.g. Orange/Green) are chosen **according to the computed verdict** for each claim.
--* **AI-Generated Label**: Clear indication that content is AI-produced
--1. **Inspection**: User clicks a highlighted claim to see the **Reasoning Trail**, showing exactly which evidence and sub-queries led to that verdict.
--
--=== Technical Scope ===
--
--* **AI-Generated Publication**: Content published as Mode 2 (AI-Generated, no prior human review)
--* **Quality Gates Active**: All automated quality checks enforced
--* **Contradiction Search Demonstrated**: Shows counter-evidence and reservation detection
--* **Risk Tier Classification**: POC shows tier assignment (demo purposes)
--* **No Human Approval Gate**: Demonstrates scalable AI publication
--* **Structured Sub-Queries**: Logic generated by decomposing claims into the FactHarbor data model
--
  ----
--== Publication Model ==
++= Manual vs Automated Responsibilities =
--FactHarbor implements a risk-based publication model with three modes:
--
--=== Mode 1: Draft-Only ===
--* Failed quality gates
--* High-risk content pending expert review
--* Internal review queue only
--
--=== Mode 2: AI-Generated (Public) ===
--* Passed all quality gates
--* Risk tier B or C
--* Clear AI-generated labeling
--* Users can request human review
--
--=== Mode 3: Human-Reviewed ===
--* Validated by human reviewers/experts
--* "Human-Reviewed" status badge
--* Required for Tier A content publication
--
--See [[AKEL page>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed publication mode descriptions.
--
------
--
--== Risk Tiers and Automation Levels ==
--
--=== Tier A (High Risk) ===
--* **Domains**: Medical, legal, elections, safety, security
--* **Automation**: AI can draft, human review required for "Human-Reviewed" status
--* **AI publication**: Allowed with prominent disclaimers and warnings
--* **Audit rate**: Recommendation: 30-50%
--
--=== Tier B (Medium Risk) ===
--* **Domains**: Complex policy, science, causality claims
--* **Automation**: AI can draft and publish (Mode 2)
--* **Human review**: Optional, audit-based
--* **Audit rate**: Recommendation: 10-20%
--
--=== Tier C (Low Risk) ===
--* **Domains**: Definitions, established facts, historical data
--* **Automation**: AI publication default
--* **Human review**: On request or via sampling
--* **Audit rate**: Recommendation: 5-10%
--
------
--
  == Human-Only Tasks ==
--These require human judgment and cannot be automated:
++These require human judgment, ethics, or contextual interpretation:
--* **Ethical boundary decisions** (especially medical, political, psychological harm assessment)
--* **Dispute resolution** between conflicting expert opinions
--* **Governance policy** setting and enforcement
--* **Final authority** on Tier A "Human-Reviewed" status
--* **Audit system oversight** and quality standard definition
--* **Risk tier policy** adjustments based on societal context
++* Definition of key terms in claims
++* Approval or rejection of scenarios
++* Interpretation of evidence in context
++* Final verdict approval
++* Governance decisions and dispute resolution
++* High-risk domain oversight
++* Ethical boundary decisions (especially medical, political, psychological)
------
++== Semi-Automated (AI Draft → Human Review) ==
--== AI-Draft with Audit (Semi-Automated) ==
++AKEL can draft these, but humans must refine/approve:
--AKEL drafts these; humans validate via sampling audits:
++* Scenario structures (definitions, assumptions, context)
++* Evaluation methods
++* Evidence relevance suggestions
++* Reliability hints
++* Verdict reasoning chains
++* Uncertainty and limitations
++* Scenario comparison explanations
++* Suggestions for merging or splitting scenarios
++* Draft public summaries
--* **Scenario structures** (definitions, assumptions, context)
--* **Evaluation methods** and reasoning chains
--* **Evidence relevance** assessment and ranking
--* **Reliability scoring** and source evaluation
--* **Verdict reasoning** with uncertainty quantification
--* **Contradiction and reservation** identification
--* **Scenario comparison** explanations
--* **Public summaries** and accessibility text
--
--Most Tier B and C content remains in AI-draft status unless:
--* Users request human review
--* Audits identify errors
--* High engagement triggers review
--* Community flags issues
--
------
--
  == Fully Automated Structural Tasks ==
  These require no human interpretation:
--* **Claim normalization** (canonical form generation)
--* **Duplicate detection** (vector embeddings, clustering)
--* **Evidence metadata extraction** (dates, authors, publication info)
--* **Basic reliability heuristics** (source reputation scoring)
--* **Contradiction detection** (conflicting statements across sources)
--* **Re-evaluation triggers** (new evidence, source updates)
--* **Layout generation** (diagrams, summaries, UI presentation)
--* **Federation integrity checks** (cross-node data validation)
++* Claim normalization
++* Duplicate & cluster detection (vector embeddings)
++* Evidence metadata extraction
++* Basic reliability heuristics
++* Contradiction detection
++* Re-evaluation triggers
++* Batch layout generation (diagrams, summaries)
++* Federation integrity checks
  ----
--== Quality Gates (Automated) ==
++= Automation Roadmap =
--Before AI-draft publication (Mode 2), content must pass:
++Automation increases with maturity.
--1. **Source Quality Gate**
--   * Primary sources verified
--   * Citations complete and accessible
--   * Source reliability scored
++== POC (Low Automation) ==
--2. **Contradiction Search Gate** (MANDATORY)
--   * Counter-evidence actively sought
--   * Reservations and limitations identified
--   * Bubble detection (echo chambers, conspiracy theories)
--   * Diverse perspective verification
++=== Automated ===
--3. **Uncertainty Quantification Gate**
--   * Confidence scores calculated
--   * Limitations stated
--   * Data gaps disclosed
++* Claim normalization
++* Light scenario templates
++* Evidence metadata extraction
++* Simple verdict drafts (internal only)
--4. **Structural Integrity Gate**
--   * No hallucinations detected
--   * Logic chain valid
--   * References verifiable
++=== Human ===
--See [[AKEL page>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] for detailed quality gate specifications.
++* All scenario definitions
++* Evidence interpretation
++* Verdict creation
++* Governance
------
++== Beta 0 (Medium Automation) ==
--== Audit System ==
++=== Automated ===
--Instead of reviewing all AI output, systematic sampling audits ensure quality:
++* Detailed scenario drafts
++* Evidence reliability scoring
++* Cross-scenario comparisons
++* Contradiction detection (local + remote nodes)
++* Internal Truth Landscape drafts
--=== Stratified Sampling ===
--* Risk tier (A > B > C sampling rates)
--* Confidence scores (low confidence → more audits)
--* Traffic/engagement (popular content audited more)
--* Novelty (new topics/claim types prioritized)
--* User flags and disagreement signals
++=== Human ===
--=== Continuous Improvement Loop ===
--Audit findings improve:
--* Query templates
--* Source reliability weights
--* Contradiction detection algorithms
--* Risk tier assignment rules
--* Bubble detection heuristics
++* Scenario approval
++* Final verdict validation
--=== Transparency ===
--* Audit statistics published
--* Accuracy rates by tier reported
--* System improvements documented
++== Release 1.0 (High Automation) ==
------
++=== Automated ===
--== Automation Roadmap ==
++* Full scenario generation (definitions, assumptions, boundaries)
++* Evidence relevance scoring and ranking
++* Bayesian verdict scoring across scenario sets
++* Multi-scenario summary generation
++* Anomaly detection across nodes
++* AKEL-assisted federated synchronization
--Automation capabilities increase with system maturity while maintaining quality oversight.
++=== Human ===
--=== POC (Current Focus) ===
++* Final approval of all scenarios and verdicts
++* Ethical decisions
++* Oversight and conflict resolution
--**Automated:**
--* Claim normalization
--* Scenario template generation
--* Evidence metadata extraction
--* Simple verdict drafts
--* **AI-generated publication** (Mode 2, with quality gates)
--* **Contradiction search**
--* **Risk tier assignment**
++----
--**Human:**
--* High-risk content validation (Tier A)
--* Sampling audits across all tiers
--* Quality standard refinement
--* Governance decisions
++= Automation Levels =
--=== Beta 0 (Enhanced Automation) ===
++== Level 0 — Human-Centric (POC) ==
--**Automated:**
--* Detailed scenario generation
--* Advanced evidence reliability scoring
--* Cross-scenario comparisons
--* Multi-source contradiction detection
--* Internal Truth Landscape generation
--* **Increased AI-draft coverage** (more Tier B content)
++AI is purely advisory, nothing auto-published.
--**Human:**
--* Tier A final approval
--* Audit sampling (continued)
--* Expert validation of complex domains
--* Quality improvement oversight
++== Level 1 — Assisted (Beta 0) ==
--=== Release 1.0 (High Automation) ===
++AI drafts structures; humans approve each part.
--**Automated:**
--* Full scenario generation (comprehensive)
--* Bayesian verdict scoring across scenarios
--* Multi-scenario summary generation
--* Anomaly detection across federated nodes
--* AKEL-assisted cross-node synchronization
--* **Most Tier B and all Tier C** auto-published
++== Level 2 — Structured (Release 1.0) ==
--**Human:**
--* Tier A oversight (still required)
--* Strategic audits (lower sampling rates, higher value)
--* Ethical decisions and policy
--* Conflict resolution
++AI produces near-complete drafts; humans refine.
------
++== Level 3 — Distributed Intelligence (Future) ==
--== Automation Levels Diagram ==
++Nodes exchange embeddings, contradiction alerts, and scenario templates.
++Humans still approve everything.
--{{include reference="Test.FactHarborV09.Specification.Diagrams.Automation Level.WebHome"/}}
--
  ----
--== Automation Roadmap Diagram ==
++= Automation Matrix =
--{{include reference="Test.FactHarborV09.Specification.Diagrams.Automation Roadmap.WebHome"/}}
++== Always Human ==
------
++* Final verdict approval
++* Scenario validity
++* Ethical decisions
++* Dispute resolution
--== Manual vs Automated Matrix ==
++== Mostly AI (Human Validation Needed) ==
--{{include reference="Test.FactHarborV09.Specification.Diagrams.Manual vs Automated matrix.WebHome"/}}
++* Claim normalization
++* Clustering
++* Evidence metadata
++* Reliability heuristics
++* Scenario drafts
++* Contradiction detection
++== Mixed ==
++
++* Definitions of ambiguous terms
++* Boundary choices
++* Assumption evaluation
++* Evidence selection
++* Verdict reasoning
++
  ----
--== Related Pages ==
++= Diagram References =
--* [[AKEL (AI Knowledge Extraction Layer)>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]]
--* [[Requirements (Roles)>>FactHarbor.Specification.Requirements.WebHome]]
--* [[Workflows>>FactHarbor.Specification.Workflows.WebHome]]
--* [[Governance>>FactHarbor.Organisation.Governance]]
++{{include reference="FactHarbor.Archive.Diagrams v0\.8q.Automation Roadmap.WebHome"/}}
++{{include reference="FactHarbor.Archive.Diagrams v0\.8q.Automation Level.WebHome"/}}
++
++{{include reference="FactHarbor.Specification.Diagrams.Manual vs Automated matrix.WebHome"/}}

Changes for page Automation

Summary

Details

Applications

Navigation

Need help?