Skip to Content
Last modified by Robert Schaub on 2025/12/24 20:35
From version 8.1
edited by Robert Schaub
on 2025/11/27 12:55
Change comment: There is no comment for this version
To version 1.1
edited by Robert Schaub
on 2025/11/27 12:02
Change comment: There is no comment for this version

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -FactHarbor.Playground.WebHome
1 +FactHarbor.Specification.Data Model.WebHome
Content
... ... @@ -1,7 +5,3 @@
1 -(((
2 -
3 -)))
4 -
5 5  = 5. Data Model =
6 6  
7 7  The FactHarbor data model centers on four fully versioned, immutable entities:
... ... @@ -45,10 +45,10 @@
45 45   (scenarios live at the *claim* level, not per individual phrasing).
46 46  * Verdicts and Scenario–Evidence links are always attached to **versions**:
47 47  * {{code}}SCENARIO_VERSION{{/code}} +
48 -{{code}}EVIDENCE_VERSION{{/code}} →
49 -{{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
44 + {{code}}EVIDENCE_VERSION{{/code}} →
45 + {{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
50 50  * {{code}}SCENARIO_VERSION{{/code}} →
51 -{{code}}VERDICT_VERSION{{/code}}
47 + {{code}}VERDICT_VERSION{{/code}}
52 52  
53 53  This ensures that when a Scenario or Evidence changes, old verdicts and links
54 54  remain intact as historical records and can be revisited.
... ... @@ -61,9 +61,128 @@
61 61  The convention is that fields ending in {{code}}Id{{/code}} are primary keys,
62 62  and fields with {{code}}...IdFk{{/code}} are foreign keys.
63 63  
64 -{{comment}} Core Data Model ERD (Mermaid, from /Specification/Diagrams/Data Model) {{/comment}}
65 -{{include document="FactHarbor.Playground.Core Data Model ERD Page (from Specification chat).WebHome" reference="FactHarbor.Playground.data.Core Data Model ERD Page (from Specification chat).WebHome"/}}
60 +{{mermaid}}
61 +erDiagram
66 66  
63 + CLAIM_CLUSTER {
64 + string claimClusterId
65 + string theme
66 + string embeddingVectorRef
67 + string language
68 + datetime createdAt
69 + }
70 +
71 + CLAIM {
72 + string claimId
73 + string claimClusterIdFk
74 + string status
75 + datetime createdAt
76 + }
77 +
78 + CLAIM_VERSION {
79 + string claimVersionId
80 + string claimIdFk
81 + string text
82 + string language
83 + string claimType
84 + string domain
85 + string authorType
86 + datetime createdAt
87 + }
88 +
89 + SCENARIO {
90 + string scenarioId
91 + string claimIdFk
92 + string key
93 + string title
94 + boolean isDeprecated
95 + }
96 +
97 + SCENARIO_VERSION {
98 + string scenarioVersionId
99 + string scenarioIdFk
100 + string versionTag
101 + string definitionsJson
102 + string assumptionsJson
103 + string boundariesJson
104 + string notes
105 + datetime createdAt
106 + }
107 +
108 + EVIDENCE {
109 + string evidenceId
110 + string canonicalSourceId
111 + string mainUrl
112 + string evidenceType
113 + string language
114 + }
115 +
116 + EVIDENCE_VERSION {
117 + string evidenceVersionId
118 + string evidenceIdFk
119 + string snapshotLocation
120 + string extractionSummary
121 + string reliabilityModel
122 + datetime collectedAt
123 + datetime createdAt
124 + }
125 +
126 + SCENARIO_EVIDENCE_LINK {
127 + string scenarioEvidenceLinkId
128 + string scenarioIdFk
129 + string evidenceIdFk
130 + }
131 +
132 + SCENARIO_EVIDENCE_LINK_VERSION {
133 + string scenarioEvidenceLinkVersionId
134 + string scenarioEvidenceLinkIdFk
135 + string scenarioVersionIdFk
136 + string evidenceVersionIdFk
137 + float relevance
138 + string direction %% SUPPORTS / CONTRADICTS / MIXED / CONTEXT
139 + string rationale
140 + datetime createdAt
141 + }
142 +
143 + VERDICT {
144 + string verdictId
145 + string scenarioIdFk
146 + string verdictType %% e.g. likelihood, classification
147 + }
148 +
149 + VERDICT_VERSION {
150 + string verdictVersionId
151 + string verdictIdFk
152 + string scenarioVersionIdFk
153 + float probability
154 + float confidence
155 + string reasoningSummary
156 + string uncertaintyFactorsJson
157 + datetime createdAt
158 + }
159 +
160 + %% Relationships
161 +
162 + CLAIM_CLUSTER ||--o{ CLAIM : contains
163 + CLAIM ||--o{ CLAIM_VERSION : has_versions
164 + CLAIM ||--o{ SCENARIO : has_scenarios
165 + SCENARIO ||--o{ SCENARIO_VERSION : has_versions
166 +
167 + EVIDENCE ||--o{ EVIDENCE_VERSION : has_versions
168 +
169 + SCENARIO ||--o{ SCENARIO_EVIDENCE_LINK : may_link
170 + EVIDENCE ||--o{ SCENARIO_EVIDENCE_LINK : may_link
171 +
172 + SCENARIO_EVIDENCE_LINK ||--o{ SCENARIO_EVIDENCE_LINK_VERSION : has_versions
173 +
174 + SCENARIO_VERSION ||--o{ SCENARIO_EVIDENCE_LINK_VERSION : uses_evidence
175 + EVIDENCE_VERSION ||--o{ SCENARIO_EVIDENCE_LINK_VERSION : is_used_in
176 +
177 + SCENARIO ||--o{ VERDICT : has_verdicts
178 + VERDICT ||--o{ VERDICT_VERSION : has_versions
179 + SCENARIO_VERSION ||--o{ VERDICT_VERSION : assessed_in
180 +{{/mermaid}}
181 +
67 67  **Important points:**
68 68  
69 69  * Scenarios and Evidence are **linked via their versions**
... ... @@ -76,7 +76,7 @@
76 76  
77 77  ----
78 78  
79 -= 5.3 Data Use & Review ERD =
194 += 5.3 Data Use & Review ERD (expanded, versioned) =
80 80  
81 81  The **Data Use** model captures who does what with which versioned data:
82 82  
... ... @@ -84,10 +84,87 @@
84 84  * Roles and role assignments
85 85  * Review actions on versioned entities
86 86  
87 -{{comment}} Data Use ERD (Mermaid, from /Specification/Diagrams/Data Use ERD) {{/comment}}
88 -{{include document="FactHarbor.Playground.Data Use ERD Page (from Specification chat).WebHome" reference="FactHarbor.Playground.data.Data Use ERD Page (from Specification chat).WebHome"/}}
202 +{{mermaid}}
203 +erDiagram
89 89  
205 + USER {
206 + string userId
207 + string displayName
208 + string email
209 + string userType %% "human" or "technical"
210 + datetime createdAt
211 + }
90 90  
213 + TECHNICAL_USER {
214 + string technicalUserId
215 + string userIdFk
216 + string description
217 + string systemIdentifier
218 + }
219 +
220 + ROLE {
221 + string roleId
222 + string code %% e.g. READER, CONTRIBUTOR, REVIEWER, TRUSTED_CONTRIBUTOR, MODERATOR, SYSTEM_ADMIN, FEDERATION_OPERATOR, FEDERATION_ADMIN
223 + string description
224 + boolean isFederationRole
225 + }
226 +
227 + USER_ROLE_MEMBERSHIP {
228 + string membershipId
229 + string userIdFk
230 + string roleIdFk
231 + datetime grantedAt
232 + string grantedByUserIdFk
233 + }
234 +
235 + REVIEW_ACTION {
236 + string reviewActionId
237 + string subjectType %% e.g. CLAIM_VERSION, SCENARIO_VERSION...
238 + string subjectVersionId
239 + string actionType %% APPROVE, REJECT, FLAG, COMMENT, REQUEST_CHANGES...
240 + string outcome %% ACCEPTED, REJECTED, ESCALATED...
241 + string comment
242 + string createdByUserIdFk
243 + datetime createdAt
244 + }
245 +
246 + %% Versioned data entities (references from the core model)
247 +
248 + CLAIM_VERSION {
249 + string claimVersionId
250 + }
251 +
252 + SCENARIO_VERSION {
253 + string scenarioVersionId
254 + }
255 +
256 + EVIDENCE_VERSION {
257 + string evidenceVersionId
258 + }
259 +
260 + SCENARIO_EVIDENCE_LINK_VERSION {
261 + string scenarioEvidenceLinkVersionId
262 + }
263 +
264 + VERDICT_VERSION {
265 + string verdictVersionId
266 + }
267 +
268 + %% Relationships
269 +
270 + USER ||--o{ TECHNICAL_USER : may_be
271 + USER ||--o{ USER_ROLE_MEMBERSHIP : has_role
272 + ROLE ||--o{ USER_ROLE_MEMBERSHIP : assigned_to
273 +
274 + USER ||--o{ REVIEW_ACTION : performs
275 +
276 + CLAIM_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
277 + SCENARIO_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
278 + EVIDENCE_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
279 + SCENARIO_EVIDENCE_LINK_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
280 + VERDICT_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
281 +{{/mermaid}}
282 +
91 91  Notes:
92 92  
93 93  * Most roles (READER, CONTRIBUTOR, TRUSTED_CONTRIBUTOR, REVIEWER, MODERATOR,
... ... @@ -117,7 +117,7 @@
117 117  * It may inherit some links from earlier scenarios, or start empty depending
118 118   on the change classification (cosmetic vs. conceptual).
119 119  * All verdicts for that scenario are recalculated and stored as new
120 -{{code}}VERDICT_VERSION{{/code}} entries.
312 + {{code}}VERDICT_VERSION{{/code}} entries.
121 121  
122 122  * REVIEW_ACTIONs are always attached to the **exact version** that was seen by
123 123   the reviewer. This preserves a faithful audit trail if data later changes.
... ... @@ -154,173 +154,3 @@
154 154  Federation-specific entities (such as {{code}}FEDERATION_NODE{{/code}},
155 155  replication logs, and trust rules) are described in the Federation &
156 156  Decentralization chapter and build on top of the core data model defined here.
157 -
158 -----
159 -
160 -== 1. Overall analysis & review of the data model ==
161 -
162 -=== 1.1 Strengths of the current design ===
163 -
164 -* (((
165 -**Identity vs. version pattern**
166 -Using base entities plus version entities (CLAIM + CLAIM_VERSION, SCENARIO + SCENARIO_VERSION, etc.) is exactly how modern knowledge systems handle:
167 -
168 -* auditability
169 -* time evolution
170 -* re-evaluation triggers
171 -* federation and partial replication
172 -)))
173 -* (((
174 -**Scenario-centric reasoning**
175 -Separating //Claim// (what people argue about) from //Scenario// (interpretive frame) is very aligned with “truth landscape” style systems:
176 -
177 -* Scenarios explain //why people disagree//.
178 -* Verdicts are tied to specific scenario versions → avoids mixing incompatible assumptions.
179 -)))
180 -* **Evidence and verdicts as first-class entities**
181 -Evidence is explicit, linked to scenarios, and verdicts are per scenario. This matches good practice from fact-checking, scientific assessment panels, and trust graphs.
182 -* (((
183 -**Cluster level (CLAIM_CLUSTER)**
184 -Grouping related claims avoids duplication and lets you:
185 -
186 -* reuse scenarios across paraphrases
187 -* share embeddings / semantic search
188 -* keep the system scalable as the corpus grows.
189 -)))
190 -* (((
191 -**Explicit review layer (REVIEW_ACTION, roles, etc.)**
192 -Separating “data” from “who reviewed what” keeps the model clean, and is exactly what you want for:
193 -
194 -* governance
195 -* permissions
196 -* audit trails
197 -* future trust scoring per user / role.
198 -)))
199 -
200 -----
201 -
202 -=== 1.2 Design decisions I’m locking in (based on our discussions) ===
203 -
204 -To make the model consistent and “state-of-the-art”, I will assume the following as //current intended design//:
205 -
206 -1. (((
207 -**Claims vs Scenarios**
208 -
209 -* CLAIM is the stable identity for “what people argue about”.
210 -* CLAIM_VERSION are individual phrasings / formulations / metadata.
211 -* (((
212 -SCENARIO belongs to a **CLAIM**, not to a specific CLAIM_VERSION.
213 -Rationale:
214 -
215 -* Many different phrasings share the //same// scenario.
216 -* You avoid duplicating scenarios per wording.
217 -)))
218 -* SCENARIO_VERSION holds detailed definitions, assumptions, boundaries, etc.
219 -)))
220 -1. (((
221 -**Version-specific reasoning**
222 -
223 -* **Verdicts** are always attached to SCENARIO_VERSION (not base SCENARIO).
224 -* **Evidence links** are between SCENARIO_VERSION and EVIDENCE_VERSION.
225 -→ This is what we agreed when we said //“SCENARIO_EVIDENCE_LINK should link the respective versions instead”//.
226 -)))
227 -1. (((
228 -**Clusters**
229 -
230 -* CLAIM_CLUSTER groups Claims (semantically close claims).
231 -* It is visible in **both diagrams** (Core Data Model and Data Use).
232 -)))
233 -1. (((
234 -**Review vs data**
235 -
236 -* (((
237 -All review happens **on versioned entities**:
238 -
239 -* CLAIM_VERSION
240 -* SCENARIO_VERSION
241 -* EVIDENCE_VERSION
242 -* SCENARIO_EVIDENCE_LINK_VERSION
243 -* VERDICT_VERSION
244 -)))
245 -* REVIEW_ACTION is the generic log of //who// did //what// on //which version//.
246 -)))
247 -1. (((
248 -**Users & roles**
249 -
250 -* USER has an attribute (or a linked entity) that distinguishes **technical users** from normal accounts.
251 -* We //keep// TECHNICAL_USER as a specialisation of USER (strictly technical accounts).
252 -* All human & technical accounts can hold roles via USER_ROLE_MEMBERSHIP.
253 -* (((
254 -Roles include:
255 -
256 -* READER
257 -* CONTRIBUTOR
258 -* TRUSTED_CONTRIBUTOR
259 -* REVIEWER
260 -* MODERATOR
261 -* SYSTEM_ADMIN / MAINTAINER
262 -* FEDERATION_OPERATOR
263 -* FEDERATION_ADMIN
264 -(all present in the Data Use ERD, but as rows of ROLE rather than separate entities).
265 -)))
266 -)))
267 -
268 -----
269 -
270 -=== 1.3 Gaps / potential problems ===
271 -
272 -These are the main issues & missing areas I see:
273 -
274 -1. (((
275 -**Versioning text in chapter 5 is currently too thin (‘…’ placeholders)**
276 -
277 -* (((
278 -The spec does not yet //verbally// spell out:
279 -
280 -* the identity vs version pattern, systematically
281 -* how re-evaluation triggers are derived from version changes
282 -* how this aligns with federation (which versions are replicated where).
283 -)))
284 -)))
285 -1. (((
286 -**No explicit “provenance granularity” in the model**
287 -
288 -* (((
289 -EVIDENCE is a single entity. For more advanced use cases, you may later want:
290 -
291 -* EVIDENCE_SOURCE (the whole article/report/video)
292 -* EVIDENCE_FRAGMENT (specific paragraph/clip with its own reliability, quote, etc.)
293 -)))
294 -* For now, I’ll keep EVIDENCE/EVIDENCE_VERSION as is, but I’ll mention this as a possible extension.
295 -)))
296 -1. (((
297 -**Review target polymorphism**
298 -
299 -* (((
300 -REVIEW_ACTION can apply to multiple entity types. In the diagram this shows as multiple relationships:
301 -
302 -* CLAIM_VERSION → REVIEW_ACTION
303 -* SCENARIO_VERSION → REVIEW_ACTION
304 -* etc.
305 -)))
306 -* A more “pure” relational modeling would use a generic “subjectType + subjectId” or an intermediate “REVIEW_TARGET” table.
307 -* For readability, I’ll keep the simpler multi-edge representation and mention the polymorphism in text.
308 -)))
309 -1. (((
310 -**Federation details missing from core ERD**
311 -
312 -* There is no explicit FEDERATION_NODE / REPLICATION_LOG in the Data Model chapter.
313 -* This is ok for “core logical data model”, but I’ll add a short note that federation metadata is handled in the Federation chapter and via additional entities.
314 -)))
315 -1. (((
316 -**Automation / AKEL artifacts left implicit**
317 -
318 -* (((
319 -The Data Model chapter currently doesn’t describe:
320 -
321 -* AKEL task queues
322 -* extraction runs
323 -* model versions
324 -)))
325 -* That’s fine for now; I’ll just clarify that those belong to a “Processing / AKEL” submodel, not the core logical data model.
326 -)))