Changes for page Workflows

Last modified by Robert Schaub on 2025/12/24 20:34

From 2.1 to 3.1 From 6.1 to 7.1

From version 3.1

edited by Robert Schaub
on 2025/12/12 09:32

Change comment: Imported from XAR

To version 6.1

edited by Robert Schaub
on 2025/12/14 18:59

Change comment: Imported from XAR

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -1,127 +1,352 @@
  = Workflows =
--This chapter defines the core workflows used across the FactHarbor system.
++This page describes the core workflows for content creation, review, and publication in FactHarbor.
--Each workflow describes:
--* Purpose
--* Participants
--* Steps
--* Automation vs. manual work
++== Overview ==
--== 1. Claim Workflow ==
++FactHarbor workflows support three publication modes with risk-based review:
--**Purpose:** Transform raw text or input material into a normalized, classified, deduplicated, and versioned claim.
++* **Mode 1 (Draft)**: Internal only, failed quality gates or pending review
++* **Mode 2 (AI-Generated)**: Public with AI-generated label, passed quality gates
++* **Mode 3 (Human-Reviewed)**: Public with human-reviewed status, highest trust
--**Participants:**
--* Contributor
--* AKEL
--* Reviewer
++Workflows vary by **Risk Tier** (A/B/C) and **Content Type** (Claim, Scenario, Evidence, Verdict).
--**Steps:**
--1. **Ingestion**: User submits text/URL; AKEL extracts claims.
--1. **Normalization**: Standardize wording, reduce ambiguity.
--1. **Classification**: Domain, Evaluability, Safety (AKEL draft → Human confirm).
--1. **Duplicate Detection**: Check embeddings for existing claims.
--1. **Version Creation**: Store new ClaimVersion.
--1. **Cluster Assignment**: Assign to Claim Cluster.
--1. **Scenario Linking**: Connect to existing or draft new scenarios.
--1. **Publication**: Make visible.
++----
--**Flow:** Ingest → Normalize → Classify → Deduplicate → Cluster → Version → Publish
++== Claim Submission & Publication Workflow ==
--== 2. Scenario Workflow ==
++=== Step 1: Claim Submission ===
--**Purpose:** Define the specific analytic contexts needed to evaluate each claim.
++**Actor**: Contributor or AKEL
--**Steps:**
--1. **Scenario Proposal**: Drafted by contributor or AKEL.
--1. **Required Fields**: Definitions, Assumptions, ContextBoundary, EvaluationMethod, SafetyClass.
--1. **Safety Interception**: AKEL flags non-falsifiable or unsafe content.
--1. **Conflict Check**: Merge similar scenarios, flag contradictions.
--1. **Reviewer Validation**: Ensure clarity and validity.
--1. **Expert Approval**: Mandatory for high-risk domains.
--1. **Version Storage**: Save ScenarioVersion.
++**Actions**:
++* Submit claim text
++* Provide initial sources (optional for human contributors, mandatory for AKEL)
++* System assigns initial AuthorType (Human or AI)
--**Flow:** Draft → Validate → Safety Check → Review → Expert → Version → Activate
++**Output**: Claim draft created
--== 3. Evidence Workflow ==
++=== Step 2: AKEL Processing ===
--**Purpose:** Structure, classify, validate, version, and link evidence to scenarios.
++**Automated Steps**:
++1. Claim extraction and normalization
++2. Classification (domain, type, evaluability)
++3. Risk tier assignment (A/B/C suggested)
++4. Initial scenario generation
++5. Evidence search
++6. **Contradiction search** (mandatory)
++7. Quality gate validation
--**Steps:**
--1. **Submission**: File, URL, or text.
--1. **Metadata Extraction**: Type, Category, Provenance, ReliabilityHints.
--1. **Relevance Check**: Verify applicability to scenario.
--1. **Reliability Assessment**: Score reliability (Reviewer + Expert).
--1. **Link Creation**: Create ScenarioEvidenceLink with relevance score.
--1. **Versioning**: Update EvidenceVersion.
++**Output**: Processed claim with risk tier and quality gate results
--**Flow:** Submit → Extract → Relevance → Reliability → Link → Version
++=== Step 3: Quality Gate Checkpoint ===
--== 4. Verdict Workflow ==
++**Gates Evaluated**:
++* Source quality
++* Contradiction search completion
++* Uncertainty quantification
++* Structural integrity
--**Purpose:** Generate likelihood estimates **per scenario** based on evidence.
++**Outcomes**:
++* **All gates pass** → Proceed to Mode 2 publication (if Tier B or C)
++* **Any gate fails** → Mode 1 (Draft), flag for human review
++* **Tier A** → Mode 2 with warnings + auto-escalate to expert queue
--**Steps:**
--1. **Aggregation**: Collect linked evidence for a specific scenario.
--1. **Draft Verdict**: AKEL proposes likelihood and uncertainty for that scenario.
--1. **Reasoning**: AKEL drafts explanation chain.
--1. **Validation**: Reviewer checks logic and hallucinations.
--1. **Expert Review**: Required for sensitive topics.
--1. **Storage**: Save VerdictVersion.
++=== Step 4: Publication (Risk-Tier Dependent) ===
--**Flow:** Aggregate → Draft → Reasoning → Review → Expert → Version
++**Tier C (Low Risk)**:
++* **Direct to Mode 2**: AI-generated, public, clearly labeled
++* User can request human review
++* Sampling audit applies
--== 5. Re-evaluation Workflow ==
++**Tier B (Medium Risk)**:
++* **Direct to Mode 2**: AI-generated, public, clearly labeled
++* Higher audit sampling rate
++* High-engagement content may auto-escalate
--**Purpose:** Keep verdicts current when inputs change.
++**Tier A (High Risk)**:
++* **Mode 2 with warnings**: AI-generated, public, prominent disclaimers
++* **Auto-escalated** to expert review queue
++* User warnings displayed
++* Highest audit sampling rate
--**Steps:**
--1. **Trigger**: Evidence update, Scenario change, or Contradiction.
--1. **Impact Analysis**: Identify affected nodes.
--1. **Re-calculation**: AKEL proposes new likelihoods.
--1. **Validation**: Human review.
--1. **Storage**: New version.
++=== Step 5: Human Review (Optional for B/C, Escalated for A) ===
--**Flow:** Trigger → Analyze → Recompute → Review → Version
++**Triggers**:
++* User requests review
++* Audit flags issues
++* High engagement (Tier B)
++* Automatic (Tier A)
--== 6. Federation Synchronization Workflow ==
++**Process**:
++1. Reviewer/Expert examines claim
++2. Validates quality gates
++3. Checks contradiction search results
++4. Assesses risk tier appropriateness
++5. Decision: Approve, Request Changes, or Reject
--**Purpose:** Exchange structured data between nodes.
++**Outcomes**:
++* **Approved** → Mode 3 (Human-Reviewed)
++* **Changes Requested** → Back to contributor or AKEL for revision
++* **Rejected** → Rejected status with reasoning
--**Steps:**
--1. Detect Version Changes.
--1. Build Signed Bundle (Merkle tree).
--1. Push/Pull to Peers.
--1. Validate Signatures & Lineage.
--1. Resolve Conflicts (Merge/Fork).
--1. Trigger Re-evaluation.
++----
--== 7. User Role & Review Workflow ==
++== Scenario Creation Workflow ==
--**Purpose:** Ensure correctness and safety.
++=== Step 1: Scenario Generation ===
--**Steps:**
--1. Submission.
--1. Auto-check (AKEL).
--1. Reviewer Validation.
--1. Expert Validation (if needed).
--1. Moderator Oversight (if flagged).
++**Automated (AKEL)**:
++* Generate scenarios for claim
++* Define boundaries, assumptions, context
++* Identify evaluation methods
--== 8. AKEL Workflow ==
++**Manual (Expert/Reviewer)**:
++* Create custom scenarios
++* Refine AKEL-generated scenarios
++* Add domain-specific nuances
--**Stages:**
--* Input Understanding
--* Scenario Drafting
--* Evidence Processing
--* Verdict Drafting
--* Safety & Integrity
--* Human Approval
++=== Step 2: Scenario Validation ===
--== 9. Global Trigger Flow (Cascade) ==
++**Quality Checks**:
++* Completeness (definitions, boundaries, assumptions clear)
++* Relevance to claim
++* Evaluability
++* No circular logic
--**Sources:** Claim/Scenario/Evidence change, Verdict contradiction, Federation update.
++**Risk Tier Assignment**:
++* Inherits from parent claim
++* Can be overridden by expert if scenario increases/decreases risk
--**Flow:** Trigger → Dependency Graph → Re-evaluation → Updated Verdicts
++=== Step 3: Scenario Publication ===
--{{include reference="FactHarbor.Specification.Diagrams.Global Trigger Cascade.WebHome"/}}
++**Mode 2 (AI-Generated)**:
++* Tier B/C scenarios can publish immediately
++* Subject to sampling audits
++
++**Mode 1 (Draft)**:
++* Tier A scenarios default to draft
++* Require expert validation for Mode 2 or Mode 3
++
++----
++
++== Evidence Evaluation Workflow ==
++
++=== Step 1: Evidence Search & Retrieval ===
++
++**AKEL Actions**:
++* Search academic databases, reputable media
++* **Mandatory contradiction search** (counter-evidence, reservations)
++* Extract metadata (author, date, publication, methodology)
++* Assess source reliability
++
++**Quality Requirements**:
++* Primary sources preferred
++* Diverse perspectives included
++* Echo chambers flagged
++* Conflicting evidence acknowledged
++
++=== Step 2: Evidence Summarization ===
++
++**AKEL Generates**:
++* Summary of evidence
++* Relevance assessment
++* Reliability score
++* Limitations and caveats
++* Conflicting evidence summary
++
++**Quality Gate**: Structural integrity, source quality
++
++=== Step 3: Evidence Review ===
++
++**Reviewer/Expert Validates**:
++* Accuracy of summaries
++* Appropriateness of sources
++* Completeness of contradiction search
++* Reliability assessments
++
++**Outcomes**:
++* **Mode 2**: Evidence summaries published as AI-generated
++* **Mode 3**: After human validation
++* **Mode 1**: Failed quality checks or pending expert review
++
++----
++
++== Verdict Generation Workflow ==
++
++=== Step 1: Verdict Computation ===
++
++**AKEL Computes**:
++* Verdict across scenarios
++* Confidence scores
++* Uncertainty quantification
++* Key assumptions
++* Limitations
++
++**Inputs**:
++* Claim text
++* Scenario definitions
++* Evidence assessments
++* Contradiction search results
++
++=== Step 2: Verdict Validation ===
++
++**Quality Gates**:
++* All four gates apply (source, contradiction, uncertainty, structure)
++* Reasoning chain must be traceable
++* Assumptions must be explicit
++
++**Risk Tier Check**:
++* Tier A: Always requires expert validation for Mode 3
++* Tier B: Mode 2 allowed, audit sampling
++* Tier C: Mode 2 default
++
++=== Step 3: Verdict Publication ===
++
++**Mode 2 (AI-Generated Verdict)**:
++* Clear labeling with confidence scores
++* Uncertainty disclosure
++* Links to reasoning trail
++* User can request expert review
++
++**Mode 3 (Expert-Validated Verdict)**:
++* Human reviewer/expert stamp
++* Additional commentary (optional)
++* Highest trust level
++
++----
++
++== Audit Workflow ==
++
++=== Step 1: Audit Sampling Selection ===
++
++**Stratified Sampling**:
++* Risk tier priority (A > B > C)
++* Low confidence scores
++* High traffic content
++* Novel topics
++* User flags
++
++**Sampling Rates** (Recommendations):
++* Tier A: 30-50%
++* Tier B: 10-20%
++* Tier C: 5-10%
++
++=== Step 2: Audit Execution ===
++
++**Auditor Actions**:
++1. Review sampled AI-generated content
++2. Validate quality gates were properly applied
++3. Check contradiction search completeness
++4. Assess reasoning quality
++5. Identify errors or hallucinations
++
++**Audit Outcome**:
++* **Pass**: Content remains in Mode 2, logged as validated
++* **Fail**: Content flagged for review, system improvement triggered
++
++=== Step 3: Feedback Loop ===
++
++**System Improvements**:
++* Failed audits analyzed for patterns
++* AKEL parameters adjusted
++* Quality gates refined
++* Risk tier assignments recalibrated
++
++**Transparency**:
++* Audit statistics published periodically
++* Patterns shared with community
++* System improvements documented
++
++----
++
++== Mode Transition Workflow ==
++
++=== Mode 1 → Mode 2 ===
++
++**Requirements**:
++* All quality gates pass
++* Risk tier B or C (or A with warnings)
++* Contradiction search completed
++
++**Trigger**: Automatic upon quality gate validation
++
++=== Mode 2 → Mode 3 ===
++
++**Requirements**:
++* Human reviewer/expert validation
++* Quality standards confirmed
++* For Tier A: Expert approval required
++* For Tier B/C: Reviewer approval sufficient
++
++**Trigger**: Human review completion
++
++=== Mode 3 → Mode 1 (Demotion) ===
++
++**Rare - Only if**:
++* New evidence contradicts verdict
++* Error discovered in reasoning
++* Source retraction
++
++**Process**:
++1. Content flagged for re-evaluation
++2. Moved to draft (Mode 1)
++3. Re-processed through workflow
++4. Reason for demotion documented
++
++----
++
++== User Actions Across Modes ==
++
++=== On Mode 1 (Draft) Content ===
++
++**Contributors**:
++* Edit their own drafts
++* Submit for review
++
++**Reviewers/Experts**:
++* View and comment
++* Request changes
++* Approve for Mode 2 or Mode 3
++
++=== On Mode 2 (AI-Generated) Content ===
++
++**All Users**:
++* Read and use content
++* Request human review
++* Flag for expert attention
++* Provide feedback
++
++**Reviewers/Experts**:
++* Validate for Mode 3 transition
++* Edit and refine
++* Adjust risk tier if needed
++
++=== On Mode 3 (Human-Reviewed) Content ===
++
++**All Users**:
++* Read with highest confidence
++* Still can flag if new evidence emerges
++
++**Reviewers/Experts**:
++* Update if needed
++* Trigger re-evaluation if new evidence
++
++----
++
++== Diagram References ==
++
++{{include reference="FactHarbor.Organisation.Diagrams.Claim-Scenario-Lifecycle"/}}
++
++{{include reference="FactHarbor.Specification.Diagrams.Manual vs Automated matrix.WebHome"/}}
++
++----
++
++== Related Pages ==
++
++* [[AKEL (AI Knowledge Extraction Layer)>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]]
++* [[Automation>>FactHarbor.Specification.Automation.WebHome]]
++* [[Requirements (Roles)>>FactHarbor.Specification.Requirements.WebHome]]
++* [[Governance>>FactHarbor.Organisation.Governance]]
++

Changes for page Workflows

Summary

Details

Applications

Navigation

Need help?