Changes for page Data Model (From Specification Chat)

Last modified by Robert Schaub on 2025/12/24 20:35

From 3.1 to 4.1 From 8.1 to 8.2

From version 4.1

edited by Robert Schaub
on 2025/11/27 12:11

Change comment: There is no comment for this version

To version 8.1

edited by Robert Schaub
on 2025/11/27 12:55

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -1,3 +1,162 @@
++(((
++
++)))
++
++= 5. Data Model =
++
++The FactHarbor data model centers on four fully versioned, immutable entities:
++
++* **Claim**
++* **Scenario**
++* **Evidence**
++* **Verdict**
++
++These entities form the structured **“truth landscape”** for each claim.
++The model is explicitly **versioned**, **traceable**, and **federation-ready**.
++
++To keep the system auditable and explainable, FactHarbor uses a consistent
++**identity vs. version** pattern:
++
++* Identity entities (e.g. {{code}}CLAIM{{/code}}, {{code}}SCENARIO{{/code}})
++  define *what* something is in a stable sense.
++* Version entities (e.g. {{code}}CLAIM_VERSION{{/code}}, {{code}}SCENARIO_VERSION{{/code}})
++  define *how that thing looked at a given point in time*.
++
++All reasoning (e.g. verdicts, review actions) is attached to **versions**, never to
++mutable identities.
++
++----
++
++= 5.1 Core entities and versioning pattern =
++
++(% class="wikitable" %)
++| **Logical concept** | **Identity entity** | **Version entity** | **Notes**
++| Claim (what people argue about) | {{code}}CLAIM{{/code}} | {{code}}CLAIM_VERSION{{/code}} | Claim text, phrasing, and metadata live in {{code}}CLAIM_VERSION{{/code}}. The identity {{code}}CLAIM{{/code}} stays stable across rephrasings.
++| Scenario (interpretive frame) | {{code}}SCENARIO{{/code}} | {{code}}SCENARIO_VERSION{{/code}} | A SCENARIO belongs to a CLAIM. Its versions capture evolving definitions, assumptions, and boundaries.
++| Evidence (source / datapoint) | {{code}}EVIDENCE{{/code}} | {{code}}EVIDENCE_VERSION{{/code}} | Identity of a source vs. specific extractions / updates over time.
++| Verdict (assessment) | {{code}}VERDICT{{/code}} | {{code}}VERDICT_VERSION{{/code}} | A VERDICT is defined per SCENARIO; VERDICT_VERSION captures the history of assessments.
++| Scenario–Evidence link | {{code}}SCENARIO_EVIDENCE_LINK{{/code}} | {{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}} | Links bind scenario versions to evidence versions with relevance & direction.
++| Claim cluster (semantic group) | {{code}}CLAIM_CLUSTER{{/code}} | – | Groups semantically related claims; mainly for discovery and navigation.
++
++Key design decisions:
++
++* A {{code}}CLAIM{{/code}} belongs to exactly one {{code}}CLAIM_CLUSTER{{/code}}.
++* A {{code}}SCENARIO{{/code}} belongs to exactly one {{code}}CLAIM{{/code}}
++  (scenarios live at the *claim* level, not per individual phrasing).
++* Verdicts and Scenario–Evidence links are always attached to **versions**:
++* {{code}}SCENARIO_VERSION{{/code}} +
++{{code}}EVIDENCE_VERSION{{/code}} →
++{{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
++* {{code}}SCENARIO_VERSION{{/code}} →
++{{code}}VERDICT_VERSION{{/code}}
++
++This ensures that when a Scenario or Evidence changes, old verdicts and links
++remain intact as historical records and can be revisited.
++
++----
++
++= 5.2 Core Data Model ERD (expanded, versioned) =
++
++The following Mermaid ER diagram shows the main entities and their relationships.
++The convention is that fields ending in {{code}}Id{{/code}} are primary keys,
++and fields with {{code}}...IdFk{{/code}} are foreign keys.
++
++{{comment}} Core Data Model ERD (Mermaid, from /Specification/Diagrams/Data Model) {{/comment}}
++{{include document="FactHarbor.Playground.Core Data Model ERD Page (from Specification chat).WebHome" reference="FactHarbor.Playground.data.Core Data Model ERD Page (from Specification chat).WebHome"/}}
++
++**Important points:**
++
++* Scenarios and Evidence are **linked via their versions**
++  ({{code}}SCENARIO_VERSION{{/code}} and {{code}}EVIDENCE_VERSION{{/code}}).
++* Verdicts are **per ScenarioVersion** and stored in {{code}}VERDICT_VERSION{{/code}}.
++* {{code}}CLAIM_CLUSTER{{/code}} is shared across diagrams; it is shown here and in the Data Use / Review model.
++
++All version entities are immutable: once created, they are never changed, only
++superseded by newer versions.
++
++----
++
++= 5.3 Data Use & Review ERD =
++
++The **Data Use** model captures who does what with which versioned data:
++
++* Users (including technical users)
++* Roles and role assignments
++* Review actions on versioned entities
++
++{{comment}} Data Use ERD (Mermaid, from /Specification/Diagrams/Data Use ERD) {{/comment}}
++{{include document="FactHarbor.Playground.Data Use ERD Page (from Specification chat).WebHome" reference="FactHarbor.Playground.data.Data Use ERD Page (from Specification chat).WebHome"/}}
++
++
++Notes:
++
++* Most roles (READER, CONTRIBUTOR, TRUSTED_CONTRIBUTOR, REVIEWER, MODERATOR,
++  SYSTEM_ADMIN, FEDERATION_OPERATOR, FEDERATION_ADMIN, …) are represented as rows
++  in {{code}}ROLE{{/code}}.
++* {{code}}TECHNICAL_USER{{/code}} captures strictly technical accounts (API keys,
++  node-to-node federation agents, batch jobs). All other roles can, in principle,
++  be held by both human and technical users where appropriate.
++* A {{code}}READER{{/code}} normally does **not** perform REVIEW_ACTIONs, while
++  roles like REVIEWER, TRUSTED_CONTRIBUTOR, MODERATOR, and some federation roles
++  do.
++
++----
++
++= 5.4 Versioning and re-evaluation behavior =
++
++This section ties the data model to the re-evaluation logic
++(described in more detail in the Versioning and Automation chapters).
++
++* When a new {{code}}EVIDENCE_VERSION{{/code}} is created:
++* All related {{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}} entries referencing
++    that evidence version are candidates for re-assessment.
++* Related {{code}}VERDICT_VERSION{{/code}} entries may become **outdated** and
++    are queued for re-evaluation.
++
++* When a new {{code}}SCENARIO_VERSION{{/code}} is created:
++* It may inherit some links from earlier scenarios, or start empty depending
++    on the change classification (cosmetic vs. conceptual).
++* All verdicts for that scenario are recalculated and stored as new
++{{code}}VERDICT_VERSION{{/code}} entries.
++
++* REVIEW_ACTIONs are always attached to the **exact version** that was seen by
++  the reviewer. This preserves a faithful audit trail if data later changes.
++
++* In a federated environment, nodes can choose:
++* which identity entities to replicate (CLAIM, SCENARIO, EVIDENCE, VERDICT)
++* which versioned entities to replicate (e.g. only accepted VERDICT_VERSIONs,
++    only EVIDENCE_VERSIONs above a reliability threshold, etc.)
++
++----
++
++= 5.5 Behavioral Notes =
++
++== 5.5.1 Late-Arriving Evidence ==
++
++New evidence versions can make existing verdicts **outdated** and may trigger
++re-evaluation cascades. This is handled by the global trigger and automation
++architecture (see the Versioning & Automation chapters).
++
++== 5.5.2 Scenario Evolution ==
++
++Scenario changes create new SCENARIO_VERSIONs; dependent verdicts and
++Scenario–Evidence links are re-assessed. Old versions remain available for
++historical comparison and reproducibility.
++
++== 5.5.3 Federation ==
++
++Federated nodes can replicate subsets of the graph, including:
++
++* Claims and Scenarios of local interest
++* Evidence metadata (without full content)
++* Verdict lineages used for local decision-making
++
++Federation-specific entities (such as {{code}}FEDERATION_NODE{{/code}},
++replication logs, and trust rules) are described in the Federation &
++Decentralization chapter and build on top of the core data model defined here.
++
++----
++
  == 1. Overall analysis & review of the data model ==
  === 1.1 Strengths of the current design ===
@@ -165,385 +165,3 @@
  )))
  * That’s fine for now; I’ll just clarify that those belong to a “Processing / AKEL” submodel, not the core logical data model.
  )))
--
--= 5. Data Model =
--
--The FactHarbor data model centers on four fully versioned, immutable entities:
--
--* **Claim**
--* **Scenario**
--* **Evidence**
--* **Verdict**
--
--These entities form the structured **“truth landscape”** for each claim.
--The model is explicitly **versioned**, **traceable**, and **federation-ready**.
--
--To keep the system auditable and explainable, FactHarbor uses a consistent
--**identity vs. version** pattern:
--
--* Identity entities (e.g. {{code}}CLAIM{{/code}}, {{code}}SCENARIO{{/code}})
--  define *what* something is in a stable sense.
--* Version entities (e.g. {{code}}CLAIM_VERSION{{/code}}, {{code}}SCENARIO_VERSION{{/code}})
--  define *how that thing looked at a given point in time*.
--
--All reasoning (e.g. verdicts, review actions) is attached to **versions**, never to
--mutable identities.
--
------
--
--= 5.1 Core entities and versioning pattern =
--
--(% class="wikitable" %)
--| **Logical concept** | **Identity entity** | **Version entity** | **Notes**
--| Claim (what people argue about) | {{code}}CLAIM{{/code}} | {{code}}CLAIM_VERSION{{/code}} | Claim text, phrasing, and metadata live in {{code}}CLAIM_VERSION{{/code}}. The identity {{code}}CLAIM{{/code}} stays stable across rephrasings.
--| Scenario (interpretive frame) | {{code}}SCENARIO{{/code}} | {{code}}SCENARIO_VERSION{{/code}} | A SCENARIO belongs to a CLAIM. Its versions capture evolving definitions, assumptions, and boundaries.
--| Evidence (source / datapoint) | {{code}}EVIDENCE{{/code}} | {{code}}EVIDENCE_VERSION{{/code}} | Identity of a source vs. specific extractions / updates over time.
--| Verdict (assessment) | {{code}}VERDICT{{/code}} | {{code}}VERDICT_VERSION{{/code}} | A VERDICT is defined per SCENARIO; VERDICT_VERSION captures the history of assessments.
--| Scenario–Evidence link | {{code}}SCENARIO_EVIDENCE_LINK{{/code}} | {{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}} | Links bind scenario versions to evidence versions with relevance & direction.
--| Claim cluster (semantic group) | {{code}}CLAIM_CLUSTER{{/code}} | – | Groups semantically related claims; mainly for discovery and navigation.
--
--Key design decisions:
--
--* A {{code}}CLAIM{{/code}} belongs to exactly one {{code}}CLAIM_CLUSTER{{/code}}.
--* A {{code}}SCENARIO{{/code}} belongs to exactly one {{code}}CLAIM{{/code}}
--  (scenarios live at the *claim* level, not per individual phrasing).
--* Verdicts and Scenario–Evidence links are always attached to **versions**:
--* {{code}}SCENARIO_VERSION{{/code}} +
--{{code}}EVIDENCE_VERSION{{/code}} →
--{{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
--* {{code}}SCENARIO_VERSION{{/code}} →
--{{code}}VERDICT_VERSION{{/code}}
--
--This ensures that when a Scenario or Evidence changes, old verdicts and links
--remain intact as historical records and can be revisited.
--
------
--
--= 5.2 Core Data Model ERD (expanded, versioned) =
--
--The following Mermaid ER diagram shows the main entities and their relationships.
--The convention is that fields ending in {{code}}Id{{/code}} are primary keys,
--and fields with {{code}}...IdFk{{/code}} are foreign keys.
--
--{{mermaid}}
--erDiagram
--    CLAIM_CLUSTER {
--        string ClusterID PK
--        string EmbeddingVectorRef
--        string Theme
--    }
--
--    CLAIM {
--        string ClaimID PK
--        string ClusterID FK
--        string Status
--        datetime CreatedAt
--    }
--
--    CLAIM_VERSION {
--        string ClaimVersionID PK
--        string ClaimID FK
--        string Text
--        string ClaimType
--        string Domain
--        datetime CreatedAt
--    }
--
--    SCENARIO {
--        string ScenarioID PK
--        string ClaimID FK
--        string Name
--        datetime CreatedAt
--    }
--
--    SCENARIO_VERSION {
--        string ScenarioVersionID PK
--        string ScenarioID FK
--        string Definitions
--        string Assumptions
--        string Boundaries
--        datetime CreatedAt
--    }
--
--    EVIDENCE {
--        string EvidenceID PK
--        string SourceType
--        string URL
--        float ReliabilityScore
--    }
--
--    EVIDENCE_VERSION {
--        string EvidenceVersionID PK
--        string EvidenceID FK
--        string Summary
--        float ReliabilityScore
--        datetime CreatedAt
--    }
--
--    SCENARIO_EVIDENCE_LINK {
--        string LinkID PK
--        string ScenarioVersionID FK
--        string EvidenceVersionID FK
--        float Relevance
--        string Direction
--    }
--
--    VERDICT {
--        string VerdictID PK
--        string ScenarioID FK
--    }
--
--    VERDICT_VERSION {
--        string VerdictVersionID PK
--        string VerdictID FK
--        float Verdict
--        float Confidence
--        string Reasoning
--        datetime CreatedAt
--    }
--
--    CLAIM_CLUSTER ||--o{ CLAIM : contains
--    CLAIM ||--o{ CLAIM_VERSION : versions
--
--    CLAIM ||--o{ SCENARIO : has
--    SCENARIO ||--o{ SCENARIO_VERSION : versions
--
--    EVIDENCE ||--o{ EVIDENCE_VERSION : versions
--
--    SCENARIO_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : links
--    EVIDENCE_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : linked
--
--    SCENARIO ||--o{ VERDICT : assessed
--    VERDICT ||--o{ VERDICT_VERSION : versions
--
--{{/mermaid}}
--
--**Important points:**
--
--* Scenarios and Evidence are **linked via their versions**
--  ({{code}}SCENARIO_VERSION{{/code}} and {{code}}EVIDENCE_VERSION{{/code}}).
--* Verdicts are **per ScenarioVersion** and stored in {{code}}VERDICT_VERSION{{/code}}.
--* {{code}}CLAIM_CLUSTER{{/code}} is shared across diagrams; it is shown here and in the Data Use / Review model.
--
--All version entities are immutable: once created, they are never changed, only
--superseded by newer versions.
--
------
--
--= 5.3 Data Use & Review ERD (expanded, versioned) =
--
--The **Data Use** model captures who does what with which versioned data:
--
--* Users (including technical users)
--* Roles and role assignments
--* Review actions on versioned entities
--
--{{mermaid}}
--erDiagram
--    %% Core clusters shown for context
--    CLAIM_CLUSTER {
--        string ClusterID PK
--        string EmbeddingVectorRef
--        string Theme
--    }
--
--    CLAIM {
--        string ClaimID PK
--        string ClusterID FK
--        string Status
--        datetime CreatedAt
--    }
--
--    CLAIM_VERSION {
--        string ClaimVersionID PK
--        string ClaimID FK
--        string Text
--        string ClaimType
--        string Domain
--        datetime CreatedAt
--    }
--
--    SCENARIO {
--        string ScenarioID PK
--        string ClaimID FK
--        string Name
--        datetime CreatedAt
--    }
--
--    SCENARIO_VERSION {
--        string ScenarioVersionID PK
--        string ScenarioID FK
--        string Definitions
--        string Assumptions
--        string Boundaries
--        datetime CreatedAt
--    }
--
--    EVIDENCE {
--        string EvidenceID PK
--        string SourceType
--        string URL
--        float ReliabilityScore
--    }
--
--    EVIDENCE_VERSION {
--        string EvidenceVersionID PK
--        string EvidenceID FK
--        string Summary
--        float ReliabilityScore
--        datetime CreatedAt
--    }
--
--    VERDICT {
--        string VerdictID PK
--        string ScenarioID FK
--    }
--
--    VERDICT_VERSION {
--        string VerdictVersionID PK
--        string VerdictID FK
--        float Verdict
--        float Confidence
--        string Reasoning
--        datetime CreatedAt
--    }
--
--    %% Users and roles
--    USER {
--        string UserID PK
--        string Handle
--        string Email
--    }
--
--    TECHNICAL_USER {
--        string UserID PK
--        string SystemName
--    }
--
--    CONTRIBUTING_USER {
--        string UserID PK
--        string DisplayName
--    }
--
--    TRUSTED_CONTRIBUTOR {
--        string UserID PK
--        string TrustLevel
--    }
--
--    REVIEWER {
--        string UserID PK
--        string Domain
--    }
--
--    EXPERT {
--        string UserID PK
--        string ExpertiseArea
--    }
--
--    FEDERATION_NODE {
--        string NodeID PK
--        string Region
--    }
--
--    FEDERATION_ADMIN {
--        string UserID PK
--        string Permissions
--    }
--
--    REVIEW_ACTION {
--        string ReviewActionID PK
--        string UserID FK
--        string TargetEntityType
--        string TargetEntityVersionID
--        string ActionType
--        string Comment
--        datetime Timestamp
--    }
--
--    %% Inheritance / specialization (modelled as relationships)
--    USER ||--o{ TECHNICAL_USER : "is a"
--    USER ||--o{ CONTRIBUTING_USER : "is a"
--
--    CONTRIBUTING_USER ||--o{ TRUSTED_CONTRIBUTOR : "subset"
--    CONTRIBUTING_USER ||--o{ REVIEWER : "subset"
--    CONTRIBUTING_USER ||--o{ EXPERT : "subset"
--
--    TECHNICAL_USER ||--o{ FEDERATION_NODE : "operates"
--    TECHNICAL_USER ||--o{ FEDERATION_ADMIN : "administers"
--
--    %% Review actions on versioned entities
--    USER ||--o{ REVIEW_ACTION : performs
--
--    REVIEW_ACTION }o--|| CLAIM_VERSION : reviews
--    REVIEW_ACTION }o--|| SCENARIO_VERSION : reviews
--    REVIEW_ACTION }o--|| EVIDENCE_VERSION : reviews
--    REVIEW_ACTION }o--|| VERDICT_VERSION : reviews
--
--{{/mermaid}}
--
--Notes:
--
--* Most roles (READER, CONTRIBUTOR, TRUSTED_CONTRIBUTOR, REVIEWER, MODERATOR,
--  SYSTEM_ADMIN, FEDERATION_OPERATOR, FEDERATION_ADMIN, …) are represented as rows
--  in {{code}}ROLE{{/code}}.
--* {{code}}TECHNICAL_USER{{/code}} captures strictly technical accounts (API keys,
--  node-to-node federation agents, batch jobs). All other roles can, in principle,
--  be held by both human and technical users where appropriate.
--* A {{code}}READER{{/code}} normally does **not** perform REVIEW_ACTIONs, while
--  roles like REVIEWER, TRUSTED_CONTRIBUTOR, MODERATOR, and some federation roles
--  do.
--
------
--
--= 5.4 Versioning and re-evaluation behavior =
--
--This section ties the data model to the re-evaluation logic
--(described in more detail in the Versioning and Automation chapters).
--
--* When a new {{code}}EVIDENCE_VERSION{{/code}} is created:
--* All related {{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}} entries referencing
--    that evidence version are candidates for re-assessment.
--* Related {{code}}VERDICT_VERSION{{/code}} entries may become **outdated** and
--    are queued for re-evaluation.
--
--* When a new {{code}}SCENARIO_VERSION{{/code}} is created:
--* It may inherit some links from earlier scenarios, or start empty depending
--    on the change classification (cosmetic vs. conceptual).
--* All verdicts for that scenario are recalculated and stored as new
--{{code}}VERDICT_VERSION{{/code}} entries.
--
--* REVIEW_ACTIONs are always attached to the **exact version** that was seen by
--  the reviewer. This preserves a faithful audit trail if data later changes.
--
--* In a federated environment, nodes can choose:
--* which identity entities to replicate (CLAIM, SCENARIO, EVIDENCE, VERDICT)
--* which versioned entities to replicate (e.g. only accepted VERDICT_VERSIONs,
--    only EVIDENCE_VERSIONs above a reliability threshold, etc.)
--
------
--
--= 5.5 Behavioral Notes =
--
--== 5.5.1 Late-Arriving Evidence ==
--
--New evidence versions can make existing verdicts **outdated** and may trigger
--re-evaluation cascades. This is handled by the global trigger and automation
--architecture (see the Versioning & Automation chapters).
--
--== 5.5.2 Scenario Evolution ==
--
--Scenario changes create new SCENARIO_VERSIONs; dependent verdicts and
--Scenario–Evidence links are re-assessed. Old versions remain available for
--historical comparison and reproducibility.
--
--== 5.5.3 Federation ==
--
--Federated nodes can replicate subsets of the graph, including:
--
--* Claims and Scenarios of local interest
--* Evidence metadata (without full content)
--* Verdict lineages used for local decision-making
--
--Federation-specific entities (such as {{code}}FEDERATION_NODE{{/code}},
--replication logs, and trust rules) are described in the Federation &
--Decentralization chapter and build on top of the core data model defined here.

Changes for page Data Model (From Specification Chat)

Summary

Details

Applications

Navigation

Need help?