Wiki source code of Data Model
Last modified by Robert Schaub on 2025/12/24 20:34
Hide last authors
| author | version | line-number | content |
|---|---|---|---|
 |
1.1 | 1 | = Data Model = |
| 2 | |||
| |
6.1 | 3 | This page describes the current data model for FactHarbor v0.9.1. |
| |
1.1 | 4 | |
| |
5.1 | 5 | == Versioning Strategy == |
| 6 | |||
| 7 | Every entity in FactHarbor has a full immutable version history. This ensures: | ||
| |
6.4 | 8 | |
| |
5.1 | 9 | * Complete auditability |
| 10 | * Ability to reconstruct historical state | ||
| 11 | * Federation-compatible lineage tracking | ||
| 12 | * Transparent evolution of claims, scenarios, and verdicts | ||
| 13 | |||
| 14 | === Core Versioning Principles === | ||
| 15 | |||
| 16 | **Immutability**: | ||
| |
6.4 | 17 | |
| |
5.1 | 18 | * Each version is stored independently |
| 19 | * Versions cannot be deleted, only superseded | ||
| 20 | * Historical versions remain accessible | ||
| 21 | |||
| 22 | **Lineage**: | ||
| |
6.4 | 23 | |
| |
5.1 | 24 | * Each version links to its parent via `ParentVersionID` |
| 25 | * Forms directed acyclic graph (DAG) of changes | ||
| 26 | * Supports branching in federated environments | ||
| 27 | |||
| 28 | **Provenance**: | ||
| |
6.4 | 29 | |
| |
5.1 | 30 | * Every version timestamped (`CreatedAt`) |
| 31 | * Author type recorded (`AuthorType`: Human, AI, ExternalNode) | ||
| 32 | * Justification captured (`JustificationText`) | ||
| 33 | * Digital signatures for integrity (`SignatureHash` in Release 1.0) | ||
| 34 | |||
| 35 | **Federation Support**: | ||
| |
6.4 | 36 | |
| |
5.1 | 37 | * Versions can originate from remote nodes |
| 38 | * Conflict detection via lineage comparison | ||
| 39 | * Parallel version trees for branching scenarios | ||
| 40 | * Cross-node version synchronization | ||
| 41 | |||
| 42 | === Common Version Fields === | ||
| 43 | |||
| 44 | All versioned entities include: | ||
| 45 | |||
| 46 | * **VersionID**: Unique identifier for this specific version | ||
| 47 | * **ParentVersionID**: Link to previous version (null for first version) | ||
| 48 | * **CreatedAt**: Timestamp (ISO 8601, UTC) | ||
| 49 | * **AuthorType**: Human | AI | ExternalNode | ||
| |
6.1 | 50 | * **CreatedBy**: Foreign key to User or TechnicalUser |
| |
5.1 | 51 | * **JustificationText**: Brief explanation of changes |
| |
6.1 | 52 | * **PublicationMode**: Mode1 (draft) | Mode2 (AI-published) | Mode3 (human-reviewed) |
| 53 | * **ReviewStatus**: Workflow state (draft|in_review|approved|rejected) | ||
| 54 | * **NodeOrigin**: Node ID where version was created (for federation) | ||
| |
5.1 | 55 | * **SignatureHash**: Cryptographic signature (Release 1.0) |
| 56 | |||
| 57 | ---- | ||
| 58 | |||
| |
6.1 | 59 | == Core Entity Definitions == |
| |
2.1 | 60 | |
| |
6.1 | 61 | === User Entities === |
| 62 | |||
| 63 | **USER** (base user table): | ||
| |
6.4 | 64 | |
| |
6.1 | 65 | * ``UserID`` (PK) |
| 66 | * ``UserType`` (Reader|Contributor|Reviewer|Auditor|Expert|Moderator|Maintainer) | ||
| 67 | * ``DisplayName`` | ||
| 68 | * ``Email`` (for Contributors and above) | ||
| 69 | * ``RegisteredAt`` | ||
| 70 | * ``LastActive`` | ||
| 71 | * ``Status`` (active|suspended|banned) | ||
| 72 | |||
| 73 | **TECHNICAL_USER** (system processes): | ||
| |
6.4 | 74 | |
| |
6.1 | 75 | * ``SystemID`` (PK) |
| 76 | * ``SystemName`` | ||
| 77 | * ``Purpose`` (AKEL|FederationSync|BackupService|Monitor|Audit) | ||
| 78 | * ``CreatedBy`` (FK to Maintainer who created this system user) | ||
| 79 | * ``CreatedAt`` | ||
| 80 | * ``Status`` (active|paused|deprecated) | ||
| 81 | * ``ApiKey`` (encrypted) | ||
| 82 | * ``Permissions`` (JSON - authorized operations) | ||
| 83 | |||
| 84 | **Examples of Technical Users**: | ||
| |
6.4 | 85 | |
| |
6.1 | 86 | * AKEL instances (AI processing) |
| 87 | * Federation sync bots | ||
| 88 | * Scheduled audit tasks | ||
| 89 | * Backup services | ||
| 90 | * Monitoring systems | ||
| 91 | * External API integrations | ||
| 92 | |||
| 93 | ---- | ||
| 94 | |||
| 95 | === Content Entities === | ||
| 96 | |||
| |
2.1 | 97 | The system relies on the following versioned core entities: |
| 98 | |||
| |
6.1 | 99 | **CLAIM_CLUSTER**: |
| |
6.4 | 100 | |
| |
6.1 | 101 | * ``ClusterID`` (PK) |
| 102 | * ``EmbeddingVectorRef`` | ||
| 103 | * ``Theme`` | ||
| 104 | * Groups related claims into topical clusters | ||
| 105 | * One Cluster has many Claims | ||
| 106 | * A Claim belongs to exactly one primary cluster | ||
| |
2.1 | 107 | |
| |
6.1 | 108 | **CLAIM / CLAIM_VERSION**: |
| |
6.4 | 109 | |
| |
6.1 | 110 | * ``CLAIM`` is the long-lived anchor for a real-world claim |
| 111 | * ``CLAIM_VERSION`` is an immutable snapshot that includes: | ||
| |
6.4 | 112 | * ``VersionID`` (PK) |
| 113 | * ``ClaimID`` (FK to CLAIM) | ||
| 114 | * ``ParentVersionID`` (FK to prior version, nullable) | ||
| 115 | * ``Text`` | ||
| 116 | * ``Domain`` | ||
| 117 | * ``ClaimType`` (literal|metaphorical|rhetorical|supernatural) | ||
| 118 | * ``Evaluability`` (empirical|subjective|non-falsifiable) | ||
| 119 | * ``RiskTier`` (A|B|C) - replaced SafetyCategory for consistency | ||
| 120 | * ``PublicationMode`` (Mode1|Mode2|Mode3) | ||
| 121 | * ``ReviewStatus`` (draft|in_review|approved|rejected) | ||
| 122 | * ``CreatedAt``, ``AuthorType``, ``CreatedBy``, ``JustificationText`` | ||
| 123 | * ``NodeOrigin``, ``SignatureHash`` | ||
| 124 | * ``Status`` (active|superseded|merged) | ||
| |
2.1 | 125 | |
| |
6.1 | 126 | **SCENARIO / SCENARIO_VERSION**: |
| |
6.4 | 127 | |
| |
6.1 | 128 | * ``SCENARIO`` is the anchor for a scenario across time |
| 129 | * ``SCENARIO_VERSION`` is an immutable snapshot: | ||
| |
6.4 | 130 | * ``VersionID`` (PK) |
| 131 | * ``ScenarioID`` (FK to SCENARIO) | ||
| 132 | * ``ParentVersionID`` | ||
| 133 | * ``ClaimID`` (FK to CLAIM) | ||
| 134 | * ``Definitions`` (JSON) | ||
| 135 | * ``Boundaries`` (JSON) | ||
| 136 | * ``Assumptions`` (JSON) | ||
| 137 | * ``Context`` (text) | ||
| 138 | * ``EvaluationMethod`` (text) | ||
| 139 | * ``PublicationMode`` (Mode1|Mode2|Mode3) | ||
| 140 | * ``ReviewStatus`` (draft|in_review|approved|rejected) | ||
| 141 | * ``CreatedAt``, ``AuthorType``, ``CreatedBy``, ``JustificationText`` | ||
| 142 | * ``NodeOrigin``, ``SignatureHash`` | ||
| 143 | * ``Status`` (active|superseded|deprecated) | ||
| |
2.1 | 144 | |
| |
6.1 | 145 | **Note**: SafetyClass removed from Scenario - risk tier is at claim level |
| |
2.1 | 146 | |
| |
6.1 | 147 | **EVIDENCE / EVIDENCE_VERSION**: |
| |
6.4 | 148 | |
| |
6.1 | 149 | * ``EVIDENCE`` is the anchor |
| 150 | * ``EVIDENCE_VERSION`` is the versioned snapshot: | ||
| |
6.4 | 151 | * ``VersionID`` (PK) |
| 152 | * ``EvidenceID`` (FK to EVIDENCE) | ||
| 153 | * ``ParentVersionID`` | ||
| 154 | * ``Type`` (paper|dataset|report|transcript|expert|media) | ||
| 155 | * ``Category`` (empirical|historical|rhetorical|dataset|meta-analysis) | ||
| 156 | * ``Reliability`` (low|medium|high) | ||
| 157 | * ``Provenance`` (URL, DOI, source metadata) | ||
| 158 | * ``ExtractionMethod`` (manual|OCR|API|AKEL) | ||
| 159 | * ``ContentHash`` (SHA256 of evidence content) | ||
| 160 | * ``PublicationMode`` (Mode1|Mode2|Mode3) | ||
| 161 | * ``ReviewStatus`` (draft|verified|disputed|retracted) | ||
| 162 | * ``CreatedAt``, ``AuthorType``, ``CreatedBy``, ``JustificationText`` | ||
| 163 | * ``NodeOrigin``, ``SignatureHash`` | ||
| 164 | * ``Status`` (active|superseded) | ||
| |
2.1 | 165 | |
| |
6.1 | 166 | **VERDICT / VERDICT_VERSION**: |
| |
6.4 | 167 | |
| |
6.1 | 168 | * ``VERDICT`` is the anchor |
| 169 | * ``VERDICT_VERSION`` is the snapshot: | ||
| |
6.4 | 170 | * ``VersionID`` (PK) |
| 171 | * ``VerdictID`` (FK to VERDICT) | ||
| 172 | * ``ParentVersionID`` | ||
| 173 | * ``ClaimID`` (FK to CLAIM) | ||
| 174 | * ``ScenarioVersionID`` (FK to specific SCENARIO_VERSION) | ||
| 175 | * ``EvidenceVersionSet`` (JSON array of Evidence VersionIDs used) | ||
| 176 | * ``LikelihoodRange`` (0–1, with uncertainty bounds) | ||
| 177 | * ``ExplanationChain`` (JSON) | ||
| 178 | * ``UncertaintyFactors`` (JSON) | ||
| 179 | * ``PublicationMode`` (Mode1|Mode2|Mode3) | ||
| 180 | * ``ReviewStatus`` (draft|in_review|approved|retracted) | ||
| 181 | * ``CreatedAt``, ``AuthorType``, ``CreatedBy``, ``JustificationText`` | ||
| 182 | * ``NodeOrigin``, ``SignatureHash`` | ||
| 183 | * ``Status`` (current|outdated|superseded|retracted) | ||
| |
6.1 | 184 | |
| |
5.1 | 185 | ---- |
| |
2.1 | 186 | |
| |
5.1 | 187 | == Many-to-Many Linking Tables == |
| |
1.1 | 188 | |
| |
6.1 | 189 | **ScenarioEvidenceLink**: |
| |
6.4 | 190 | |
| |
6.1 | 191 | * Links scenario versions to evidence versions with relevance scoring |
| 192 | * ``ScenarioID``, ``ScenarioVersionID`` | ||
| 193 | * ``EvidenceID``, ``EvidenceVersionID`` | ||
| |
5.1 | 194 | * ``RelevanceScore`` (0–1) - How relevant this evidence is to this scenario |
| 195 | * ``LinkJustification`` - Brief explanation of relevance | ||
| 196 | |||
| 197 | **Purpose**: | ||
| |
6.4 | 198 | |
| |
5.1 | 199 | * Evidence can be used by multiple scenarios |
| 200 | * Scenarios can draw from multiple pieces of evidence | ||
| 201 | * Relevance scoring helps prioritize evidence | ||
| 202 | * Version-specific linking preserves historical accuracy | ||
| 203 | |||
| |
6.1 | 204 | **ClaimCluster**: |
| |
6.4 | 205 | |
| |
6.1 | 206 | * Semantic clustering of similar claims |
| |
5.1 | 207 | * ``ClusterID`` (PK) |
| 208 | * ``EmbeddingVector`` - Vector representation for semantic search | ||
| 209 | * ``MemberList`` - List of ClaimIDs in this cluster | ||
| 210 | * ``Theme`` - Human-readable theme description | ||
| 211 | |||
| |
6.1 | 212 | ---- |
| |
5.1 | 213 | |
| |
6.1 | 214 | == Key Changes in v0.9.1 == |
| 215 | |||
| 216 | **Updated Field Names**: | ||
| |
6.4 | 217 | |
| |
6.1 | 218 | * `SafetyCategory` → `RiskTier` (consistency with risk tier system A/B/C) |
| 219 | * `SafetyClass` removed from Scenario (redundant with claim-level RiskTier) | ||
| 220 | |||
| 221 | **Added Fields to All Version Entities**: | ||
| |
6.4 | 222 | |
| |
6.1 | 223 | * `PublicationMode` - Track Mode 1/2/3 status |
| 224 | * `ReviewStatus` - Track workflow state | ||
| 225 | * `NodeOrigin` - Federation provenance | ||
| 226 | * `CreatedBy` - FK to User/TechnicalUser (clarified) | ||
| 227 | |||
| 228 | **New Entity**: | ||
| |
6.4 | 229 | |
| |
6.1 | 230 | * `TECHNICAL_USER` - Separate system processes from human users |
| 231 | |||
| 232 | **Clarifications**: | ||
| |
6.4 | 233 | |
| |
6.1 | 234 | * `ScenarioVersionID` in Verdict (not just ScenarioID) - links to specific version |
| 235 | * `ContentHash` in Evidence - SHA256 for integrity checking | ||
| 236 | |||
| |
5.1 | 237 | ---- |
| 238 | |||
| 239 | == Data Model Behavior == | ||
| 240 | |||
| 241 | === Late-Arriving Evidence === | ||
| 242 | |||
| 243 | When new evidence versions appear: | ||
| |
6.4 | 244 | |
| |
5.1 | 245 | 1. Existing verdicts marked as **outdated** |
| 246 | 2. Scenario relevance must be re-evaluated | ||
| 247 | 3. Re-evaluation engine triggers verdict recomputation | ||
| 248 | 4. New verdict versions created | ||
| 249 | 5. Users notified of updates | ||
| 250 | |||
| 251 | === Scenario Evolution === | ||
| 252 | |||
| 253 | When a scenario's assumptions or definitions change: | ||
| |
6.4 | 254 | |
| |
6.1 | 255 | * Creates new scenario version (not in-place update) |
| |
5.1 | 256 | * All dependent verdicts must be recalculated |
| 257 | * Previous scenario versions remain accessible | ||
| |
6.1 | 258 | * Version lineage preserved |
| |
5.1 | 259 | |
| 260 | === Federated Nodes === | ||
| 261 | |||
| 262 | Each node may share partial data: | ||
| |
6.4 | 263 | |
| |
6.1 | 264 | * Claims and scenarios shared if relevant |
| 265 | * Evidence metadata shared, not always full files | ||
| 266 | * Version synchronization via NodeOrigin tracking | ||
| 267 | * Branching allowed for divergent interpretations | ||
| |
5.1 | 268 | |
| |
6.1 | 269 | ---- |
| |
5.1 | 270 | |
| |
6.1 | 271 | == Visual Diagrams == |
| |
5.1 | 272 | |
| |
6.1 | 273 | The following diagrams provide visual representations of the data model structure and relationships. |
| |
5.1 | 274 | |
| |
6.1 | 275 | === Core Data Model ERD === |
| |
5.1 | 276 | |
| |
6.15 | 277 | {{include reference="Archive.FactHarbor V0\.9\.23 Lost Data.Specification.Diagrams.Core Data Model ERD.WebHome"/}} |
| |
5.1 | 278 | |
| |
6.1 | 279 | === User Roles Structure === |
| |
5.1 | 280 | |
| |
6.1 | 281 | {{include reference="Test.FactHarborV09.Specification.Diagrams.User Roles ERD.WebHome"/}} |
| |
5.1 | 282 | |
| |
6.1 | 283 | === Content Workflow === |
| |
5.1 | 284 | |
| |
6.1 | 285 | {{include reference="Test.FactHarborV09.Specification.Diagrams.Content Workflow ERD.WebHome"/}} |
| |
5.1 | 286 | |
| 287 | ---- | ||
| 288 | |||
| |
6.1 | 289 | == Related Pages == |
| |
5.1 | 290 | |
| |
6.14 | 291 | * [[Federation & Decentralization>>Archive.FactHarbor V0\.9\.18 copy.Specification.Federation & Decentralization.WebHome]] |
| |
6.12 | 292 | * [[AKEL (AI Knowledge Extraction Layer)>>Archive.FactHarbor V0\.9\.18 copy.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] |
| |
6.13 | 293 | * [[Architecture>>Archive.FactHarbor V0\.9\.18 copy.Specification.Architecture.WebHome]] |