Changes for page Workflows

Last modified by Robert Schaub on 2025/12/24 20:34

From 5.1 to 4.1 From 7.3 to 7.2

From version 7.2

edited by Robert Schaub
on 2025/12/16 20:28

Change comment: Renamed back-links.

To version 5.1

edited by Robert Schaub
on 2025/12/12 21:50

Change comment: Rollback to version 3.1

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -1,410 +1,127 @@
  = Workflows =
--This page describes the core workflows for content creation, review, and publication in FactHarbor.
++This chapter defines the core workflows used across the FactHarbor system.
--== Overview ==
++Each workflow describes:
++* Purpose
++* Participants
++* Steps
++* Automation vs. manual work
--FactHarbor workflows support three publication modes with risk-based review:
++== 1. Claim Workflow ==
--* **Mode 1 (Draft)**: Internal only, failed quality gates or pending review
--* **Mode 2 (AI-Generated)**: Public with AI-generated label, passed quality gates
--* **Mode 3 (Human-Reviewed)**: Public with human-reviewed status, highest trust
++**Purpose:** Transform raw text or input material into a normalized, classified, deduplicated, and versioned claim.
--Workflows vary by **Risk Tier** (A/B/C) and **Content Type** (Claim, Scenario, Evidence, Verdict).
++**Participants:**
++* Contributor
++* AKEL
++* Reviewer
------
++**Steps:**
++1. **Ingestion**: User submits text/URL; AKEL extracts claims.
++1. **Normalization**: Standardize wording, reduce ambiguity.
++1. **Classification**: Domain, Evaluability, Safety (AKEL draft → Human confirm).
++1. **Duplicate Detection**: Check embeddings for existing claims.
++1. **Version Creation**: Store new ClaimVersion.
++1. **Cluster Assignment**: Assign to Claim Cluster.
++1. **Scenario Linking**: Connect to existing or draft new scenarios.
++1. **Publication**: Make visible.
--== Claim Submission & Publication Workflow ==
++**Flow:** Ingest → Normalize → Classify → Deduplicate → Cluster → Version → Publish
--=== Step 1: Claim Submission ===
++== 2. Scenario Workflow ==
--**Actor**: Contributor or AKEL
++**Purpose:** Define the specific analytic contexts needed to evaluate each claim.
--**Actions**:
++**Steps:**
++1. **Scenario Proposal**: Drafted by contributor or AKEL.
++1. **Required Fields**: Definitions, Assumptions, ContextBoundary, EvaluationMethod, SafetyClass.
++1. **Safety Interception**: AKEL flags non-falsifiable or unsafe content.
++1. **Conflict Check**: Merge similar scenarios, flag contradictions.
++1. **Reviewer Validation**: Ensure clarity and validity.
++1. **Expert Approval**: Mandatory for high-risk domains.
++1. **Version Storage**: Save ScenarioVersion.
--* Submit claim text
--* Provide initial sources (optional for human contributors, mandatory for AKEL)
--* System assigns initial AuthorType (Human or AI)
++**Flow:** Draft → Validate → Safety Check → Review → Expert → Version → Activate
--**Output**: Claim draft created
++== 3. Evidence Workflow ==
--=== Step 2: AKEL Processing ===
++**Purpose:** Structure, classify, validate, version, and link evidence to scenarios.
--**Automated Steps**:
++**Steps:**
++1. **Submission**: File, URL, or text.
++1. **Metadata Extraction**: Type, Category, Provenance, ReliabilityHints.
++1. **Relevance Check**: Verify applicability to scenario.
++1. **Reliability Assessment**: Score reliability (Reviewer + Expert).
++1. **Link Creation**: Create ScenarioEvidenceLink with relevance score.
++1. **Versioning**: Update EvidenceVersion.
--1. Claim extraction and normalization
--2. Classification (domain, type, evaluability)
--3. Risk tier assignment (A/B/C suggested)
--4. Initial scenario generation
--5. Evidence search
--6. **Contradiction search** (mandatory)
--7. Quality gate validation
++**Flow:** Submit → Extract → Relevance → Reliability → Link → Version
--**Output**: Processed claim with risk tier and quality gate results
++== 4. Verdict Workflow ==
--=== Step 3: Quality Gate Checkpoint ===
++**Purpose:** Generate likelihood estimates **per scenario** based on evidence.
--**Gates Evaluated**:
++**Steps:**
++1. **Aggregation**: Collect linked evidence for a specific scenario.
++1. **Draft Verdict**: AKEL proposes likelihood and uncertainty for that scenario.
++1. **Reasoning**: AKEL drafts explanation chain.
++1. **Validation**: Reviewer checks logic and hallucinations.
++1. **Expert Review**: Required for sensitive topics.
++1. **Storage**: Save VerdictVersion.
--* Source quality
--* Contradiction search completion
--* Uncertainty quantification
--* Structural integrity
++**Flow:** Aggregate → Draft → Reasoning → Review → Expert → Version
--**Outcomes**:
++== 5. Re-evaluation Workflow ==
--* **All gates pass** → Proceed to Mode 2 publication (if Tier B or C)
--* **Any gate fails** → Mode 1 (Draft), flag for human review
--* **Tier A** → Mode 2 with warnings + auto-escalate to expert queue
++**Purpose:** Keep verdicts current when inputs change.
--=== Step 4: Publication (Risk-Tier Dependent) ===
++**Steps:**
++1. **Trigger**: Evidence update, Scenario change, or Contradiction.
++1. **Impact Analysis**: Identify affected nodes.
++1. **Re-calculation**: AKEL proposes new likelihoods.
++1. **Validation**: Human review.
++1. **Storage**: New version.
--**Tier C (Low Risk)**:
++**Flow:** Trigger → Analyze → Recompute → Review → Version
--* **Direct to Mode 2**: AI-generated, public, clearly labeled
--* User can request human review
--* Sampling audit applies
++== 6. Federation Synchronization Workflow ==
--**Tier B (Medium Risk)**:
++**Purpose:** Exchange structured data between nodes.
--* **Direct to Mode 2**: AI-generated, public, clearly labeled
--* Higher audit sampling rate
--* High-engagement content may auto-escalate
++**Steps:**
++1. Detect Version Changes.
++1. Build Signed Bundle (Merkle tree).
++1. Push/Pull to Peers.
++1. Validate Signatures & Lineage.
++1. Resolve Conflicts (Merge/Fork).
++1. Trigger Re-evaluation.
--**Tier A (High Risk)**:
++== 7. User Role & Review Workflow ==
--* **Mode 2 with warnings**: AI-generated, public, prominent disclaimers
--* **Auto-escalated** to expert review queue
--* User warnings displayed
--* Highest audit sampling rate
++**Purpose:** Ensure correctness and safety.
--=== Step 5: Human Review (Optional for B/C, Escalated for A) ===
++**Steps:**
++1. Submission.
++1. Auto-check (AKEL).
++1. Reviewer Validation.
++1. Expert Validation (if needed).
++1. Moderator Oversight (if flagged).
--**Triggers**:
++== 8. AKEL Workflow ==
--* User requests review
--* Audit flags issues
--* High engagement (Tier B)
--* Automatic (Tier A)
++**Stages:**
++* Input Understanding
++* Scenario Drafting
++* Evidence Processing
++* Verdict Drafting
++* Safety & Integrity
++* Human Approval
--**Process**:
++== 9. Global Trigger Flow (Cascade) ==
--1. Reviewer/Expert examines claim
--2. Validates quality gates
--3. Checks contradiction search results
--4. Assesses risk tier appropriateness
--5. Decision: Approve, Request Changes, or Reject
++**Sources:** Claim/Scenario/Evidence change, Verdict contradiction, Federation update.
--**Outcomes**:
++**Flow:** Trigger → Dependency Graph → Re-evaluation → Updated Verdicts
--* **Approved** → Mode 3 (Human-Reviewed)
--* **Changes Requested** → Back to contributor or AKEL for revision
--* **Rejected** → Rejected status with reasoning
--
------
--
--== Scenario Creation Workflow ==
--
--=== Step 1: Scenario Generation ===
--
--**Automated (AKEL)**:
--
--* Generate scenarios for claim
--* Define boundaries, assumptions, context
--* Identify evaluation methods
--
--**Manual (Expert/Reviewer)**:
--
--* Create custom scenarios
--* Refine AKEL-generated scenarios
--* Add domain-specific nuances
--
--=== Step 2: Scenario Validation ===
--
--**Quality Checks**:
--
--* Completeness (definitions, boundaries, assumptions clear)
--* Relevance to claim
--* Evaluability
--* No circular logic
--
--**Risk Tier Assignment**:
--
--* Inherits from parent claim
--* Can be overridden by expert if scenario increases/decreases risk
--
--=== Step 3: Scenario Publication ===
--
--**Mode 2 (AI-Generated)**:
--
--* Tier B/C scenarios can publish immediately
--* Subject to sampling audits
--
--**Mode 1 (Draft)**:
--
--* Tier A scenarios default to draft
--* Require expert validation for Mode 2 or Mode 3
--
------
--
--== Evidence Evaluation Workflow ==
--
--=== Step 1: Evidence Search & Retrieval ===
--
--**AKEL Actions**:
--
--* Search academic databases, reputable media
--* **Mandatory contradiction search** (counter-evidence, reservations)
--* Extract metadata (author, date, publication, methodology)
--* Assess source reliability
--
--**Quality Requirements**:
--
--* Primary sources preferred
--* Diverse perspectives included
--* Echo chambers flagged
--* Conflicting evidence acknowledged
--
--=== Step 2: Evidence Summarization ===
--
--**AKEL Generates**:
--
--* Summary of evidence
--* Relevance assessment
--* Reliability score
--* Limitations and caveats
--* Conflicting evidence summary
--
--**Quality Gate**: Structural integrity, source quality
--
--=== Step 3: Evidence Review ===
--
--**Reviewer/Expert Validates**:
--
--* Accuracy of summaries
--* Appropriateness of sources
--* Completeness of contradiction search
--* Reliability assessments
--
--**Outcomes**:
--
--* **Mode 2**: Evidence summaries published as AI-generated
--* **Mode 3**: After human validation
--* **Mode 1**: Failed quality checks or pending expert review
--
------
--
--== Verdict Generation Workflow ==
--
--=== Step 1: Verdict Computation ===
--
--**AKEL Computes**:
--
--* Verdict across scenarios
--* Confidence scores
--* Uncertainty quantification
--* Key assumptions
--* Limitations
--
--**Inputs**:
--
--* Claim text
--* Scenario definitions
--* Evidence assessments
--* Contradiction search results
--
--=== Step 2: Verdict Validation ===
--
--**Quality Gates**:
--
--* All four gates apply (source, contradiction, uncertainty, structure)
--* Reasoning chain must be traceable
--* Assumptions must be explicit
--
--**Risk Tier Check**:
--
--* Tier A: Always requires expert validation for Mode 3
--* Tier B: Mode 2 allowed, audit sampling
--* Tier C: Mode 2 default
--
--=== Step 3: Verdict Publication ===
--
--**Mode 2 (AI-Generated Verdict)**:
--
--* Clear labeling with confidence scores
--* Uncertainty disclosure
--* Links to reasoning trail
--* User can request expert review
--
--**Mode 3 (Expert-Validated Verdict)**:
--
--* Human reviewer/expert stamp
--* Additional commentary (optional)
--* Highest trust level
--
------
--
--== Audit Workflow ==
--
--=== Step 1: Audit Sampling Selection ===
--
--**Stratified Sampling**:
--
--* Risk tier priority (A > B > C)
--* Low confidence scores
--* High traffic content
--* Novel topics
--* User flags
--
--**Sampling Rates** (Recommendations):
--
--* Tier A: 30-50%
--* Tier B: 10-20%
--* Tier C: 5-10%
--
--=== Step 2: Audit Execution ===
--
--**Auditor Actions**:
--
--1. Review sampled AI-generated content
--2. Validate quality gates were properly applied
--3. Check contradiction search completeness
--4. Assess reasoning quality
--5. Identify errors or hallucinations
--
--**Audit Outcome**:
--
--* **Pass**: Content remains in Mode 2, logged as validated
--* **Fail**: Content flagged for review, system improvement triggered
--
--=== Step 3: Feedback Loop ===
--
--**System Improvements**:
--
--* Failed audits analyzed for patterns
--* AKEL parameters adjusted
--* Quality gates refined
--* Risk tier assignments recalibrated
--
--**Transparency**:
--
--* Audit statistics published periodically
--* Patterns shared with community
--* System improvements documented
--
------
--
--== Mode Transition Workflow ==
--
--=== Mode 1 → Mode 2 ===
--
--**Requirements**:
--
--* All quality gates pass
--* Risk tier B or C (or A with warnings)
--* Contradiction search completed
--
--**Trigger**: Automatic upon quality gate validation
--
--=== Mode 2 → Mode 3 ===
--
--**Requirements**:
--
--* Human reviewer/expert validation
--* Quality standards confirmed
--* For Tier A: Expert approval required
--* For Tier B/C: Reviewer approval sufficient
--
--**Trigger**: Human review completion
--
--=== Mode 3 → Mode 1 (Demotion) ===
--
--**Rare - Only if**:
--
--* New evidence contradicts verdict
--* Error discovered in reasoning
--* Source retraction
--
--**Process**:
--
--1. Content flagged for re-evaluation
--2. Moved to draft (Mode 1)
--3. Re-processed through workflow
--4. Reason for demotion documented
--
------
--
--== User Actions Across Modes ==
--
--=== On Mode 1 (Draft) Content ===
--
--**Contributors**:
--
--* Edit their own drafts
--* Submit for review
--
--**Reviewers/Experts**:
--
--* View and comment
--* Request changes
--* Approve for Mode 2 or Mode 3
--
--=== On Mode 2 (AI-Generated) Content ===
--
--**All Users**:
--
--* Read and use content
--* Request human review
--* Flag for expert attention
--* Provide feedback
--
--**Reviewers/Experts**:
--
--* Validate for Mode 3 transition
--* Edit and refine
--* Adjust risk tier if needed
--
--=== On Mode 3 (Human-Reviewed) Content ===
--
--**All Users**:
--
--* Read with highest confidence
--* Still can flag if new evidence emerges
--
--**Reviewers/Experts**:
--
--* Update if needed
--* Trigger re-evaluation if new evidence
--
------
--
--== Diagram References ==
--
--=== Claim and Scenario Lifecycle (Overview) ===
--
--{{include reference="FactHarbor.Archive.FactHarbor V0\.9\.23 Lost Data.Organisation.Diagrams.Claim and Scenario Lifecycle (Overview).WebHome"/}}
--
--=== Claim and Scenario Workflow ===
--
--{{include reference="Test.FactHarborV09.Specification.Diagrams.Claim and Scenario Workflow.WebHome"/}}
--
--=== Evidence and Verdict Workflow ===
--
--{{include reference="Test.FactHarborV09.Specification.Diagrams.Evidence and Verdict Workflow.WebHome"/}}
--
--=== Quality and Audit Workflow ===
--
--{{include reference="Test.FactHarborV09.Specification.Diagrams.Quality and Audit Workflow.WebHome"/}}
--
--
--
--{{include reference="Test.FactHarborV09.Specification.Diagrams.Manual vs Automated matrix.WebHome"/}}
--
------
--
--== Related Pages ==
--
--* [[AKEL (AI Knowledge Extraction Layer)>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]]
--* [[Automation>>FactHarbor.Specification.Automation.WebHome]]
--* [[Requirements (Roles)>>FactHarbor.Specification.Requirements.WebHome]]
--* [[Governance>>FactHarbor.Organisation.Governance]]
++{{include reference="FactHarbor.Specification.Diagrams.Global Trigger Cascade.WebHome"/}}

Changes for page Workflows

Summary

Details

Applications

Navigation

Need help?