Changes for page Data Model (From Specification Chat)

Last modified by Robert Schaub on 2025/12/24 20:35

From 1.2 to 1.1 From 9.2 to 9.1

From version 9.1

edited by Robert Schaub
on 2025/11/27 23:17

Change comment: There is no comment for this version

To version 1.2

edited by Robert Schaub
on 2025/11/27 12:03

Change comment: Update document after refactoring.

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -1,7 +5,3 @@
--(((
--
--)))
--
  = 5. Data Model =
  The FactHarbor data model centers on four fully versioned, immutable entities:
@@ -45,10 +45,10 @@
    (scenarios live at the *claim* level, not per individual phrasing).
  * Verdicts and Scenario–Evidence links are always attached to **versions**:
  * {{code}}SCENARIO_VERSION{{/code}} +
--{{code}}EVIDENCE_VERSION{{/code}} →
--{{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
++   {{code}}EVIDENCE_VERSION{{/code}} →
++   {{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
  * {{code}}SCENARIO_VERSION{{/code}} →
--{{code}}VERDICT_VERSION{{/code}}
++   {{code}}VERDICT_VERSION{{/code}}
  This ensures that when a Scenario or Evidence changes, old verdicts and links
  remain intact as historical records and can be revisited.
@@ -61,111 +61,128 @@
  The convention is that fields ending in {{code}}Id{{/code}} are primary keys,
  and fields with {{code}}...IdFk{{/code}} are foreign keys.
--{{comment}} Core Data Model ERD (Mermaid, from /Specification/Diagrams/Data Model) {{/comment}}
--{{include document="FactHarbor.Playground.Core Data Model ERD Page (from Specification chat).WebHome" reference="FactHarbor.Playground.data.Core Data Model ERD Page (from Specification chat).WebHome"/}}
--
--= Core Data Model ERD (Versioned) =
--
--This diagram shows the full core data model with all versioned entities.
--
  {{mermaid}}
  erDiagram
++
      CLAIM_CLUSTER {
--        string ClusterID PK
--        string EmbeddingVectorRef
--        string Theme
++        string claimClusterId
++        string theme
++        string embeddingVectorRef
++        string language
++        datetime createdAt
      }
      CLAIM {
--        string ClaimID PK
--        string ClusterID FK
--        string Status
--        datetime CreatedAt
++        string claimId
++        string claimClusterIdFk
++        string status
++        datetime createdAt
      }
      CLAIM_VERSION {
--        string ClaimVersionID PK
--        string ClaimID FK
--        string Text
--        string ClaimType
--        string Domain
--        datetime CreatedAt
++        string claimVersionId
++        string claimIdFk
++        string text
++        string language
++        string claimType
++        string domain
++        string authorType
++        datetime createdAt
      }
      SCENARIO {
--        string ScenarioID PK
--        string ClaimID FK
--        string Name
--        datetime CreatedAt
++        string scenarioId
++        string claimIdFk
++        string key
++        string title
++        boolean isDeprecated
      }
      SCENARIO_VERSION {
--        string ScenarioVersionID PK
--        string ScenarioID FK
--        string Definitions
--        string Assumptions
--        string Boundaries
--        datetime CreatedAt
++        string scenarioVersionId
++        string scenarioIdFk
++        string versionTag
++        string definitionsJson
++        string assumptionsJson
++        string boundariesJson
++        string notes
++        datetime createdAt
      }
      EVIDENCE {
--        string EvidenceID PK
--        string SourceType
--        string URL
--        float ReliabilityScore
++        string evidenceId
++        string canonicalSourceId
++        string mainUrl
++        string evidenceType
++        string language
      }
      EVIDENCE_VERSION {
--        string EvidenceVersionID PK
--        string EvidenceID FK
--        string Summary
--        float ReliabilityScore
--        datetime CreatedAt
++        string evidenceVersionId
++        string evidenceIdFk
++        string snapshotLocation
++        string extractionSummary
++        string reliabilityModel
++        datetime collectedAt
++        datetime createdAt
      }
      SCENARIO_EVIDENCE_LINK {
--        string LinkID PK
--        string ScenarioVersionID FK
--        string EvidenceVersionID FK
--        float Relevance
--        string Direction
++        string scenarioEvidenceLinkId
++        string scenarioIdFk
++        string evidenceIdFk
      }
++    SCENARIO_EVIDENCE_LINK_VERSION {
++        string scenarioEvidenceLinkVersionId
++        string scenarioEvidenceLinkIdFk
++        string scenarioVersionIdFk
++        string evidenceVersionIdFk
++        float relevance
++        string direction   %% SUPPORTS / CONTRADICTS / MIXED / CONTEXT
++        string rationale
++        datetime createdAt
++    }
++
      VERDICT {
--        string VerdictID PK
--        string ScenarioID FK
++        string verdictId
++        string scenarioIdFk
++        string verdictType    %% e.g. likelihood, classification
      }
      VERDICT_VERSION {
--        string VerdictVersionID PK
--        string VerdictID FK
--        float Verdict
--        float Confidence
--        string Reasoning
--        datetime CreatedAt
++        string verdictVersionId
++        string verdictIdFk
++        string scenarioVersionIdFk
++        float probability
++        float confidence
++        string reasoningSummary
++        string uncertaintyFactorsJson
++        datetime createdAt
      }
++    %% Relationships
++
      CLAIM_CLUSTER ||--o{ CLAIM : contains
--    CLAIM ||--o{ CLAIM_VERSION : versions
++    CLAIM ||--o{ CLAIM_VERSION : has_versions
++    CLAIM ||--o{ SCENARIO : has_scenarios
++    SCENARIO ||--o{ SCENARIO_VERSION : has_versions
--    CLAIM ||--o{ SCENARIO : has
--    SCENARIO ||--o{ SCENARIO_VERSION : versions
++    EVIDENCE ||--o{ EVIDENCE_VERSION : has_versions
--    EVIDENCE ||--o{ EVIDENCE_VERSION : versions
++    SCENARIO ||--o{ SCENARIO_EVIDENCE_LINK : may_link
++    EVIDENCE ||--o{ SCENARIO_EVIDENCE_LINK : may_link
--    SCENARIO_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : links
--    EVIDENCE_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : linked
++    SCENARIO_EVIDENCE_LINK ||--o{ SCENARIO_EVIDENCE_LINK_VERSION : has_versions
--    SCENARIO ||--o{ VERDICT : assessed
--    VERDICT ||--o{ VERDICT_VERSION : versions
++    SCENARIO_VERSION ||--o{ SCENARIO_EVIDENCE_LINK_VERSION : uses_evidence
++    EVIDENCE_VERSION ||--o{ SCENARIO_EVIDENCE_LINK_VERSION : is_used_in
++
++    SCENARIO ||--o{ VERDICT : has_verdicts
++    VERDICT ||--o{ VERDICT_VERSION : has_versions
++    SCENARIO_VERSION ||--o{ VERDICT_VERSION : assessed_in
  {{/mermaid}}
--{{info}}
--All key entities are explicitly versioned here (…VERSION tables).
--This reflects the versioning requirements in the textual Data Model chapter.
--{{/info}}
--
--
  **Important points:**
  * Scenarios and Evidence are **linked via their versions**
@@ -178,7 +178,7 @@
  ----
--= 5.3 Data Use & Review ERD =
++= 5.3 Data Use & Review ERD (expanded, versioned) =
  The **Data Use** model captures who does what with which versioned data:
@@ -186,165 +186,87 @@
  * Roles and role assignments
  * Review actions on versioned entities
--{{comment}} Data Use ERD (Mermaid, from /Specification/Diagrams/Data Use ERD) {{/comment}}
--{{include document="FactHarbor.Playground.Data Use ERD Page (from Specification chat).WebHome" reference="FactHarbor.Playground.data.Data Use ERD Page (from Specification chat).WebHome"/}}
--
--= Data Use ERD (Roles, Review & Versioned Entities) =
--
--This diagram shows how users, roles, and review actions relate to the
--versioned core entities.
--
  {{mermaid}}
  erDiagram
--    %% Core clusters shown for context
--    CLAIM_CLUSTER {
--        string ClusterID PK
--        string EmbeddingVectorRef
--        string Theme
--    }
--    CLAIM {
--        string ClaimID PK
--        string ClusterID FK
--        string Status
--        datetime CreatedAt
++    USER {
++        string userId
++        string displayName
++        string email
++        string userType      %% "human" or "technical"
++        datetime createdAt
      }
--    CLAIM_VERSION {
--        string ClaimVersionID PK
--        string ClaimID FK
--        string Text
--        string ClaimType
--        string Domain
--        datetime CreatedAt
++    TECHNICAL_USER {
++        string technicalUserId
++        string userIdFk
++        string description
++        string systemIdentifier
      }
--    SCENARIO {
--        string ScenarioID PK
--        string ClaimID FK
--        string Name
--        datetime CreatedAt
++    ROLE {
++        string roleId
++        string code          %% e.g. READER, CONTRIBUTOR, REVIEWER, TRUSTED_CONTRIBUTOR, MODERATOR, SYSTEM_ADMIN, FEDERATION_OPERATOR, FEDERATION_ADMIN
++        string description
++        boolean isFederationRole
      }
--    SCENARIO_VERSION {
--        string ScenarioVersionID PK
--        string ScenarioID FK
--        string Definitions
--        string Assumptions
--        string Boundaries
--        datetime CreatedAt
++    USER_ROLE_MEMBERSHIP {
++        string membershipId
++        string userIdFk
++        string roleIdFk
++        datetime grantedAt
++        string grantedByUserIdFk
      }
--    EVIDENCE {
--        string EvidenceID PK
--        string SourceType
--        string URL
--        float ReliabilityScore
++    REVIEW_ACTION {
++        string reviewActionId
++        string subjectType         %% e.g. CLAIM_VERSION, SCENARIO_VERSION...
++        string subjectVersionId
++        string actionType          %% APPROVE, REJECT, FLAG, COMMENT, REQUEST_CHANGES...
++        string outcome             %% ACCEPTED, REJECTED, ESCALATED...
++        string comment
++        string createdByUserIdFk
++        datetime createdAt
      }
--    EVIDENCE_VERSION {
--        string EvidenceVersionID PK
--        string EvidenceID FK
--        string Summary
--        float ReliabilityScore
--        datetime CreatedAt
--    }
++    %% Versioned data entities (references from the core model)
--    VERDICT {
--        string VerdictID PK
--        string ScenarioID FK
++    CLAIM_VERSION {
++        string claimVersionId
      }
--    VERDICT_VERSION {
--        string VerdictVersionID PK
--        string VerdictID FK
--        float Verdict
--        float Confidence
--        string Reasoning
--        datetime CreatedAt
++    SCENARIO_VERSION {
++        string scenarioVersionId
      }
--    %% Users and roles
--    USER {
--        string UserID PK
--        string Handle
--        string Email
++    EVIDENCE_VERSION {
++        string evidenceVersionId
      }
--    TECHNICAL_USER {
--        string UserID PK
--        string SystemName
++    SCENARIO_EVIDENCE_LINK_VERSION {
++        string scenarioEvidenceLinkVersionId
      }
--    CONTRIBUTING_USER {
--        string UserID PK
--        string DisplayName
++    VERDICT_VERSION {
++        string verdictVersionId
      }
--    TRUSTED_CONTRIBUTOR {
--        string UserID PK
--        string TrustLevel
--    }
++    %% Relationships
--    REVIEWER {
--        string UserID PK
--        string Domain
--    }
++    USER ||--o{ TECHNICAL_USER : may_be
++    USER ||--o{ USER_ROLE_MEMBERSHIP : has_role
++    ROLE ||--o{ USER_ROLE_MEMBERSHIP : assigned_to
--    EXPERT {
--        string UserID PK
--        string ExpertiseArea
--    }
--
--    FEDERATION_NODE {
--        string NodeID PK
--        string Region
--    }
--
--    FEDERATION_ADMIN {
--        string UserID PK
--        string Permissions
--    }
--
--    REVIEW_ACTION {
--        string ReviewActionID PK
--        string UserID FK
--        string TargetEntityType
--        string TargetEntityVersionID
--        string ActionType
--        string Comment
--        datetime Timestamp
--    }
--
--    %% Inheritance / specialization (modelled as relationships)
--    USER ||--o{ TECHNICAL_USER : "is a"
--    USER ||--o{ CONTRIBUTING_USER : "is a"
--
--    CONTRIBUTING_USER ||--o{ TRUSTED_CONTRIBUTOR : "subset"
--    CONTRIBUTING_USER ||--o{ REVIEWER : "subset"
--    CONTRIBUTING_USER ||--o{ EXPERT : "subset"
--
--    TECHNICAL_USER ||--o{ FEDERATION_NODE : "operates"
--    TECHNICAL_USER ||--o{ FEDERATION_ADMIN : "administers"
--
--    %% Review actions on versioned entities
      USER ||--o{ REVIEW_ACTION : performs
--    REVIEW_ACTION }o--|| CLAIM_VERSION : reviews
--    REVIEW_ACTION }o--|| SCENARIO_VERSION : reviews
--    REVIEW_ACTION }o--|| EVIDENCE_VERSION : reviews
--    REVIEW_ACTION }o--|| VERDICT_VERSION : reviews
++    CLAIM_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
++    SCENARIO_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
++    EVIDENCE_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
++    SCENARIO_EVIDENCE_LINK_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
++    VERDICT_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
  {{/mermaid}}
--{{info}}
--This diagram focuses on *who* uses and reviews *which* versioned entities.
--USER is the base type; TECHNICAL_USER and CONTRIBUTING_USER are specializations.
--Other roles (REVIEWER, EXPERT, TRUSTED_CONTRIBUTOR, FEDERATION_ADMIN, FEDERATION_NODE)
--are modelled as specializations or technical subtypes.
--{{/info}}
--
--
--
  Notes:
  * Most roles (READER, CONTRIBUTOR, TRUSTED_CONTRIBUTOR, REVIEWER, MODERATOR,
@@ -374,7 +374,7 @@
  * It may inherit some links from earlier scenarios, or start empty depending
      on the change classification (cosmetic vs. conceptual).
  * All verdicts for that scenario are recalculated and stored as new
--{{code}}VERDICT_VERSION{{/code}} entries.
++   {{code}}VERDICT_VERSION{{/code}} entries.
  * REVIEW_ACTIONs are always attached to the **exact version** that was seen by
    the reviewer. This preserves a faithful audit trail if data later changes.
@@ -411,192 +411,3 @@
  Federation-specific entities (such as {{code}}FEDERATION_NODE{{/code}},
  replication logs, and trust rules) are described in the Federation &
  Decentralization chapter and build on top of the core data model defined here.
--
--
--
--
--USER
--├── TECHNICAL_USER
--│     ├── FEDERATION_ADMIN
--│     └── AKEL_AGENT (optional future)
--
--READER
--└── CONTRIBUTING_USER
--      ├── TRUSTED_CONTRIBUTOR
--      ├── REVIEWER
--      ├── EXPERT
--      ├── MODERATOR
--
--
--ADMIN
--
--FEDERATION_ADMIN (administrative, but human)
--
--
--== 1. Overall analysis & review of the data model ==
--
--=== 1.1 Strengths of the current design ===
--
--* (((
--**Identity vs. version pattern**
--Using base entities plus version entities (CLAIM + CLAIM_VERSION, SCENARIO + SCENARIO_VERSION, etc.) is exactly how modern knowledge systems handle:
--
--* auditability
--* time evolution
--* re-evaluation triggers
--* federation and partial replication
--)))
--* (((
--**Scenario-centric reasoning**
--Separating //Claim// (what people argue about) from //Scenario// (interpretive frame) is very aligned with “truth landscape” style systems:
--
--* Scenarios explain //why people disagree//.
--* Verdicts are tied to specific scenario versions → avoids mixing incompatible assumptions.
--)))
--* **Evidence and verdicts as first-class entities**
--Evidence is explicit, linked to scenarios, and verdicts are per scenario. This matches good practice from fact-checking, scientific assessment panels, and trust graphs.
--* (((
--**Cluster level (CLAIM_CLUSTER)**
--Grouping related claims avoids duplication and lets you:
--
--* reuse scenarios across paraphrases
--* share embeddings / semantic search
--* keep the system scalable as the corpus grows.
--)))
--* (((
--**Explicit review layer (REVIEW_ACTION, roles, etc.)**
--Separating “data” from “who reviewed what” keeps the model clean, and is exactly what you want for:
--
--* governance
--* permissions
--* audit trails
--* future trust scoring per user / role.
--)))
--
------
--
--=== 1.2 Design decisions I’m locking in (based on our discussions) ===
--
--To make the model consistent and “state-of-the-art”, I will assume the following as //current intended design//:
--
--1. (((
--**Claims vs Scenarios**
--
--* CLAIM is the stable identity for “what people argue about”.
--* CLAIM_VERSION are individual phrasings / formulations / metadata.
--* (((
--SCENARIO belongs to a **CLAIM**, not to a specific CLAIM_VERSION.
--Rationale:
--
--* Many different phrasings share the //same// scenario.
--* You avoid duplicating scenarios per wording.
--)))
--* SCENARIO_VERSION holds detailed definitions, assumptions, boundaries, etc.
--)))
--1. (((
--**Version-specific reasoning**
--
--* **Verdicts** are always attached to SCENARIO_VERSION (not base SCENARIO).
--* **Evidence links** are between SCENARIO_VERSION and EVIDENCE_VERSION.
--→ This is what we agreed when we said //“SCENARIO_EVIDENCE_LINK should link the respective versions instead”//.
--)))
--1. (((
--**Clusters**
--
--* CLAIM_CLUSTER groups Claims (semantically close claims).
--* It is visible in **both diagrams** (Core Data Model and Data Use).
--)))
--1. (((
--**Review vs data**
--
--* (((
--All review happens **on versioned entities**:
--
--* CLAIM_VERSION
--* SCENARIO_VERSION
--* EVIDENCE_VERSION
--* SCENARIO_EVIDENCE_LINK_VERSION
--* VERDICT_VERSION
--)))
--* REVIEW_ACTION is the generic log of //who// did //what// on //which version//.
--)))
--1. (((
--**Users & roles**
--
--* USER has an attribute (or a linked entity) that distinguishes **technical users** from normal accounts.
--* We //keep// TECHNICAL_USER as a specialisation of USER (strictly technical accounts).
--* All human & technical accounts can hold roles via USER_ROLE_MEMBERSHIP.
--* (((
--Roles include:
--
--* READER
--* CONTRIBUTOR
--* TRUSTED_CONTRIBUTOR
--* REVIEWER
--* MODERATOR
--* SYSTEM_ADMIN / MAINTAINER
--* FEDERATION_OPERATOR
--* FEDERATION_ADMIN
--(all present in the Data Use ERD, but as rows of ROLE rather than separate entities).
--)))
--)))
--
------
--
--=== 1.3 Gaps / potential problems ===
--
--These are the main issues & missing areas I see:
--
--1. (((
--**Versioning text in chapter 5 is currently too thin (‘…’ placeholders)**
--
--* (((
--The spec does not yet //verbally// spell out:
--
--* the identity vs version pattern, systematically
--* how re-evaluation triggers are derived from version changes
--* how this aligns with federation (which versions are replicated where).
--)))
--)))
--1. (((
--**No explicit “provenance granularity” in the model**
--
--* (((
--EVIDENCE is a single entity. For more advanced use cases, you may later want:
--
--* EVIDENCE_SOURCE (the whole article/report/video)
--* EVIDENCE_FRAGMENT (specific paragraph/clip with its own reliability, quote, etc.)
--)))
--* For now, I’ll keep EVIDENCE/EVIDENCE_VERSION as is, but I’ll mention this as a possible extension.
--)))
--1. (((
--**Review target polymorphism**
--
--* (((
--REVIEW_ACTION can apply to multiple entity types. In the diagram this shows as multiple relationships:
--
--* CLAIM_VERSION → REVIEW_ACTION
--* SCENARIO_VERSION → REVIEW_ACTION
--* etc.
--)))
--* A more “pure” relational modeling would use a generic “subjectType + subjectId” or an intermediate “REVIEW_TARGET” table.
--* For readability, I’ll keep the simpler multi-edge representation and mention the polymorphism in text.
--)))
--1. (((
--**Federation details missing from core ERD**
--
--* There is no explicit FEDERATION_NODE / REPLICATION_LOG in the Data Model chapter.
--* This is ok for “core logical data model”, but I’ll add a short note that federation metadata is handled in the Federation chapter and via additional entities.
--)))
--1. (((
--**Automation / AKEL artifacts left implicit**
--
--* (((
--The Data Model chapter currently doesn’t describe:
--
--* AKEL task queues
--* extraction runs
--* model versions
--)))
--* That’s fine for now; I’ll just clarify that those belong to a “Processing / AKEL” submodel, not the core logical data model.
--)))

Changes for page Data Model (From Specification Chat)

Summary

Details

Applications

Navigation

Need help?