Skip to Content
Last modified by Robert Schaub on 2025/12/24 20:35
From version 1.1
edited by Robert Schaub
on 2025/11/27 12:02
Change comment: There is no comment for this version
To version 9.2
edited by Robert Schaub
on 2025/12/11 11:05
Change comment: Updated parent field.

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -FactHarbor.Specification.Data Model.WebHome
1 +xwiki:FactHarbor.Playground.FactHarbor.Playground.WebHome
Content
... ... @@ -1,3 +1,7 @@
1 +(((
2 +
3 +)))
4 +
1 1  = 5. Data Model =
2 2  
3 3  The FactHarbor data model centers on four fully versioned, immutable entities:
... ... @@ -41,10 +41,10 @@
41 41   (scenarios live at the *claim* level, not per individual phrasing).
42 42  * Verdicts and Scenario–Evidence links are always attached to **versions**:
43 43  * {{code}}SCENARIO_VERSION{{/code}} +
44 - {{code}}EVIDENCE_VERSION{{/code}} →
45 - {{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
48 +{{code}}EVIDENCE_VERSION{{/code}} →
49 +{{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
46 46  * {{code}}SCENARIO_VERSION{{/code}} →
47 - {{code}}VERDICT_VERSION{{/code}}
51 +{{code}}VERDICT_VERSION{{/code}}
48 48  
49 49  This ensures that when a Scenario or Evidence changes, old verdicts and links
50 50  remain intact as historical records and can be revisited.
... ... @@ -57,128 +57,111 @@
57 57  The convention is that fields ending in {{code}}Id{{/code}} are primary keys,
58 58  and fields with {{code}}...IdFk{{/code}} are foreign keys.
59 59  
64 +{{comment}} Core Data Model ERD (Mermaid, from /Specification/Diagrams/Data Model) {{/comment}}
65 +{{include document="FactHarbor.Playground.Core Data Model ERD Page (from Specification chat).WebHome" reference="FactHarbor.Playground.data.Core Data Model ERD Page (from Specification chat).WebHome"/}}
66 +
67 += Core Data Model ERD (Versioned) =
68 +
69 +This diagram shows the full core data model with all versioned entities.
70 +
60 60  {{mermaid}}
61 61  erDiagram
62 -
63 63   CLAIM_CLUSTER {
64 - string claimClusterId
65 - string theme
66 - string embeddingVectorRef
67 - string language
68 - datetime createdAt
74 + string ClusterID PK
75 + string EmbeddingVectorRef
76 + string Theme
69 69   }
70 70  
71 71   CLAIM {
72 - string claimId
73 - string claimClusterIdFk
74 - string status
75 - datetime createdAt
80 + string ClaimID PK
81 + string ClusterID FK
82 + string Status
83 + datetime CreatedAt
76 76   }
77 77  
78 78   CLAIM_VERSION {
79 - string claimVersionId
80 - string claimIdFk
81 - string text
82 - string language
83 - string claimType
84 - string domain
85 - string authorType
86 - datetime createdAt
87 + string ClaimVersionID PK
88 + string ClaimID FK
89 + string Text
90 + string ClaimType
91 + string Domain
92 + datetime CreatedAt
87 87   }
88 88  
89 89   SCENARIO {
90 - string scenarioId
91 - string claimIdFk
92 - string key
93 - string title
94 - boolean isDeprecated
96 + string ScenarioID PK
97 + string ClaimID FK
98 + string Name
99 + datetime CreatedAt
95 95   }
96 96  
97 97   SCENARIO_VERSION {
98 - string scenarioVersionId
99 - string scenarioIdFk
100 - string versionTag
101 - string definitionsJson
102 - string assumptionsJson
103 - string boundariesJson
104 - string notes
105 - datetime createdAt
103 + string ScenarioVersionID PK
104 + string ScenarioID FK
105 + string Definitions
106 + string Assumptions
107 + string Boundaries
108 + datetime CreatedAt
106 106   }
107 107  
108 108   EVIDENCE {
109 - string evidenceId
110 - string canonicalSourceId
111 - string mainUrl
112 - string evidenceType
113 - string language
112 + string EvidenceID PK
113 + string SourceType
114 + string URL
115 + float ReliabilityScore
114 114   }
115 115  
116 116   EVIDENCE_VERSION {
117 - string evidenceVersionId
118 - string evidenceIdFk
119 - string snapshotLocation
120 - string extractionSummary
121 - string reliabilityModel
122 - datetime collectedAt
123 - datetime createdAt
119 + string EvidenceVersionID PK
120 + string EvidenceID FK
121 + string Summary
122 + float ReliabilityScore
123 + datetime CreatedAt
124 124   }
125 125  
126 126   SCENARIO_EVIDENCE_LINK {
127 - string scenarioEvidenceLinkId
128 - string scenarioIdFk
129 - string evidenceIdFk
127 + string LinkID PK
128 + string ScenarioVersionID FK
129 + string EvidenceVersionID FK
130 + float Relevance
131 + string Direction
130 130   }
131 131  
132 - SCENARIO_EVIDENCE_LINK_VERSION {
133 - string scenarioEvidenceLinkVersionId
134 - string scenarioEvidenceLinkIdFk
135 - string scenarioVersionIdFk
136 - string evidenceVersionIdFk
137 - float relevance
138 - string direction %% SUPPORTS / CONTRADICTS / MIXED / CONTEXT
139 - string rationale
140 - datetime createdAt
141 - }
142 -
143 143   VERDICT {
144 - string verdictId
145 - string scenarioIdFk
146 - string verdictType %% e.g. likelihood, classification
135 + string VerdictID PK
136 + string ScenarioID FK
147 147   }
148 148  
149 149   VERDICT_VERSION {
150 - string verdictVersionId
151 - string verdictIdFk
152 - string scenarioVersionIdFk
153 - float probability
154 - float confidence
155 - string reasoningSummary
156 - string uncertaintyFactorsJson
157 - datetime createdAt
140 + string VerdictVersionID PK
141 + string VerdictID FK
142 + float Verdict
143 + float Confidence
144 + string Reasoning
145 + datetime CreatedAt
158 158   }
159 159  
160 - %% Relationships
161 -
162 162   CLAIM_CLUSTER ||--o{ CLAIM : contains
163 - CLAIM ||--o{ CLAIM_VERSION : has_versions
164 - CLAIM ||--o{ SCENARIO : has_scenarios
165 - SCENARIO ||--o{ SCENARIO_VERSION : has_versions
149 + CLAIM ||--o{ CLAIM_VERSION : versions
166 166  
167 - EVIDENCE ||--o{ EVIDENCE_VERSION : has_versions
151 + CLAIM ||--o{ SCENARIO : has
152 + SCENARIO ||--o{ SCENARIO_VERSION : versions
168 168  
169 - SCENARIO ||--o{ SCENARIO_EVIDENCE_LINK : may_link
170 - EVIDENCE ||--o{ SCENARIO_EVIDENCE_LINK : may_link
154 + EVIDENCE ||--o{ EVIDENCE_VERSION : versions
171 171  
172 - SCENARIO_EVIDENCE_LINK ||--o{ SCENARIO_EVIDENCE_LINK_VERSION : has_versions
156 + SCENARIO_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : links
157 + EVIDENCE_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : linked
173 173  
174 - SCENARIO_VERSION ||--o{ SCENARIO_EVIDENCE_LINK_VERSION : uses_evidence
175 - EVIDENCE_VERSION ||--o{ SCENARIO_EVIDENCE_LINK_VERSION : is_used_in
176 -
177 - SCENARIO ||--o{ VERDICT : has_verdicts
178 - VERDICT ||--o{ VERDICT_VERSION : has_versions
179 - SCENARIO_VERSION ||--o{ VERDICT_VERSION : assessed_in
159 + SCENARIO ||--o{ VERDICT : assessed
160 + VERDICT ||--o{ VERDICT_VERSION : versions
180 180  {{/mermaid}}
181 181  
163 +{{info}}
164 +All key entities are explicitly versioned here (…VERSION tables).
165 +This reflects the versioning requirements in the textual Data Model chapter.
166 +{{/info}}
167 +
168 +
182 182  **Important points:**
183 183  
184 184  * Scenarios and Evidence are **linked via their versions**
... ... @@ -191,7 +191,7 @@
191 191  
192 192  ----
193 193  
194 -= 5.3 Data Use & Review ERD (expanded, versioned) =
181 += 5.3 Data Use & Review ERD =
195 195  
196 196  The **Data Use** model captures who does what with which versioned data:
197 197  
... ... @@ -199,87 +199,165 @@
199 199  * Roles and role assignments
200 200  * Review actions on versioned entities
201 201  
189 +{{comment}} Data Use ERD (Mermaid, from /Specification/Diagrams/Data Use ERD) {{/comment}}
190 +{{include document="FactHarbor.Playground.Data Use ERD Page (from Specification chat).WebHome" reference="FactHarbor.Playground.data.Data Use ERD Page (from Specification chat).WebHome"/}}
191 +
192 += Data Use ERD (Roles, Review & Versioned Entities) =
193 +
194 +This diagram shows how users, roles, and review actions relate to the
195 +versioned core entities.
196 +
202 202  {{mermaid}}
203 203  erDiagram
199 + %% Core clusters shown for context
200 + CLAIM_CLUSTER {
201 + string ClusterID PK
202 + string EmbeddingVectorRef
203 + string Theme
204 + }
204 204  
205 - USER {
206 - string userId
207 - string displayName
208 - string email
209 - string userType %% "human" or "technical"
210 - datetime createdAt
206 + CLAIM {
207 + string ClaimID PK
208 + string ClusterID FK
209 + string Status
210 + datetime CreatedAt
211 211   }
212 212  
213 - TECHNICAL_USER {
214 - string technicalUserId
215 - string userIdFk
216 - string description
217 - string systemIdentifier
213 + CLAIM_VERSION {
214 + string ClaimVersionID PK
215 + string ClaimID FK
216 + string Text
217 + string ClaimType
218 + string Domain
219 + datetime CreatedAt
218 218   }
219 219  
220 - ROLE {
221 - string roleId
222 - string code %% e.g. READER, CONTRIBUTOR, REVIEWER, TRUSTED_CONTRIBUTOR, MODERATOR, SYSTEM_ADMIN, FEDERATION_OPERATOR, FEDERATION_ADMIN
223 - string description
224 - boolean isFederationRole
222 + SCENARIO {
223 + string ScenarioID PK
224 + string ClaimID FK
225 + string Name
226 + datetime CreatedAt
225 225   }
226 226  
227 - USER_ROLE_MEMBERSHIP {
228 - string membershipId
229 - string userIdFk
230 - string roleIdFk
231 - datetime grantedAt
232 - string grantedByUserIdFk
229 + SCENARIO_VERSION {
230 + string ScenarioVersionID PK
231 + string ScenarioID FK
232 + string Definitions
233 + string Assumptions
234 + string Boundaries
235 + datetime CreatedAt
233 233   }
234 234  
235 - REVIEW_ACTION {
236 - string reviewActionId
237 - string subjectType %% e.g. CLAIM_VERSION, SCENARIO_VERSION...
238 - string subjectVersionId
239 - string actionType %% APPROVE, REJECT, FLAG, COMMENT, REQUEST_CHANGES...
240 - string outcome %% ACCEPTED, REJECTED, ESCALATED...
241 - string comment
242 - string createdByUserIdFk
243 - datetime createdAt
238 + EVIDENCE {
239 + string EvidenceID PK
240 + string SourceType
241 + string URL
242 + float ReliabilityScore
244 244   }
245 245  
246 - %% Versioned data entities (references from the core model)
245 + EVIDENCE_VERSION {
246 + string EvidenceVersionID PK
247 + string EvidenceID FK
248 + string Summary
249 + float ReliabilityScore
250 + datetime CreatedAt
251 + }
247 247  
248 - CLAIM_VERSION {
249 - string claimVersionId
253 + VERDICT {
254 + string VerdictID PK
255 + string ScenarioID FK
250 250   }
251 251  
252 - SCENARIO_VERSION {
253 - string scenarioVersionId
258 + VERDICT_VERSION {
259 + string VerdictVersionID PK
260 + string VerdictID FK
261 + float Verdict
262 + float Confidence
263 + string Reasoning
264 + datetime CreatedAt
254 254   }
255 255  
256 - EVIDENCE_VERSION {
257 - string evidenceVersionId
267 + %% Users and roles
268 + USER {
269 + string UserID PK
270 + string Handle
271 + string Email
258 258   }
259 259  
260 - SCENARIO_EVIDENCE_LINK_VERSION {
261 - string scenarioEvidenceLinkVersionId
274 + TECHNICAL_USER {
275 + string UserID PK
276 + string SystemName
262 262   }
263 263  
264 - VERDICT_VERSION {
265 - string verdictVersionId
279 + CONTRIBUTING_USER {
280 + string UserID PK
281 + string DisplayName
266 266   }
267 267  
268 - %% Relationships
284 + TRUSTED_CONTRIBUTOR {
285 + string UserID PK
286 + string TrustLevel
287 + }
269 269  
270 - USER ||--o{ TECHNICAL_USER : may_be
271 - USER ||--o{ USER_ROLE_MEMBERSHIP : has_role
272 - ROLE ||--o{ USER_ROLE_MEMBERSHIP : assigned_to
289 + REVIEWER {
290 + string UserID PK
291 + string Domain
292 + }
273 273  
294 + EXPERT {
295 + string UserID PK
296 + string ExpertiseArea
297 + }
298 +
299 + FEDERATION_NODE {
300 + string NodeID PK
301 + string Region
302 + }
303 +
304 + FEDERATION_ADMIN {
305 + string UserID PK
306 + string Permissions
307 + }
308 +
309 + REVIEW_ACTION {
310 + string ReviewActionID PK
311 + string UserID FK
312 + string TargetEntityType
313 + string TargetEntityVersionID
314 + string ActionType
315 + string Comment
316 + datetime Timestamp
317 + }
318 +
319 + %% Inheritance / specialization (modelled as relationships)
320 + USER ||--o{ TECHNICAL_USER : "is a"
321 + USER ||--o{ CONTRIBUTING_USER : "is a"
322 +
323 + CONTRIBUTING_USER ||--o{ TRUSTED_CONTRIBUTOR : "subset"
324 + CONTRIBUTING_USER ||--o{ REVIEWER : "subset"
325 + CONTRIBUTING_USER ||--o{ EXPERT : "subset"
326 +
327 + TECHNICAL_USER ||--o{ FEDERATION_NODE : "operates"
328 + TECHNICAL_USER ||--o{ FEDERATION_ADMIN : "administers"
329 +
330 + %% Review actions on versioned entities
274 274   USER ||--o{ REVIEW_ACTION : performs
275 275  
276 - CLAIM_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
277 - SCENARIO_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
278 - EVIDENCE_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
279 - SCENARIO_EVIDENCE_LINK_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
280 - VERDICT_VERSION ||--o{ REVIEW_ACTION : is_reviewed_in
333 + REVIEW_ACTION }o--|| CLAIM_VERSION : reviews
334 + REVIEW_ACTION }o--|| SCENARIO_VERSION : reviews
335 + REVIEW_ACTION }o--|| EVIDENCE_VERSION : reviews
336 + REVIEW_ACTION }o--|| VERDICT_VERSION : reviews
281 281  {{/mermaid}}
282 282  
339 +{{info}}
340 +This diagram focuses on *who* uses and reviews *which* versioned entities.
341 +USER is the base type; TECHNICAL_USER and CONTRIBUTING_USER are specializations.
342 +Other roles (REVIEWER, EXPERT, TRUSTED_CONTRIBUTOR, FEDERATION_ADMIN, FEDERATION_NODE)
343 +are modelled as specializations or technical subtypes.
344 +{{/info}}
345 +
346 +
347 +
283 283  Notes:
284 284  
285 285  * Most roles (READER, CONTRIBUTOR, TRUSTED_CONTRIBUTOR, REVIEWER, MODERATOR,
... ... @@ -309,7 +309,7 @@
309 309  * It may inherit some links from earlier scenarios, or start empty depending
310 310   on the change classification (cosmetic vs. conceptual).
311 311  * All verdicts for that scenario are recalculated and stored as new
312 - {{code}}VERDICT_VERSION{{/code}} entries.
377 +{{code}}VERDICT_VERSION{{/code}} entries.
313 313  
314 314  * REVIEW_ACTIONs are always attached to the **exact version** that was seen by
315 315   the reviewer. This preserves a faithful audit trail if data later changes.
... ... @@ -346,3 +346,192 @@
346 346  Federation-specific entities (such as {{code}}FEDERATION_NODE{{/code}},
347 347  replication logs, and trust rules) are described in the Federation &
348 348  Decentralization chapter and build on top of the core data model defined here.
414 +
415 +
416 +
417 +
418 +USER
419 +├── TECHNICAL_USER
420 +│  ├── FEDERATION_ADMIN
421 +│  └── AKEL_AGENT (optional future)
422 +
423 +READER
424 +└── CONTRIBUTING_USER
425 + ├── TRUSTED_CONTRIBUTOR
426 + ├── REVIEWER
427 + ├── EXPERT
428 + ├── MODERATOR
429 +
430 +
431 +ADMIN
432 +
433 +FEDERATION_ADMIN (administrative, but human)
434 +
435 +
436 +== 1. Overall analysis & review of the data model ==
437 +
438 +=== 1.1 Strengths of the current design ===
439 +
440 +* (((
441 +**Identity vs. version pattern**
442 +Using base entities plus version entities (CLAIM + CLAIM_VERSION, SCENARIO + SCENARIO_VERSION, etc.) is exactly how modern knowledge systems handle:
443 +
444 +* auditability
445 +* time evolution
446 +* re-evaluation triggers
447 +* federation and partial replication
448 +)))
449 +* (((
450 +**Scenario-centric reasoning**
451 +Separating //Claim// (what people argue about) from //Scenario// (interpretive frame) is very aligned with “truth landscape” style systems:
452 +
453 +* Scenarios explain //why people disagree//.
454 +* Verdicts are tied to specific scenario versions → avoids mixing incompatible assumptions.
455 +)))
456 +* **Evidence and verdicts as first-class entities**
457 +Evidence is explicit, linked to scenarios, and verdicts are per scenario. This matches good practice from fact-checking, scientific assessment panels, and trust graphs.
458 +* (((
459 +**Cluster level (CLAIM_CLUSTER)**
460 +Grouping related claims avoids duplication and lets you:
461 +
462 +* reuse scenarios across paraphrases
463 +* share embeddings / semantic search
464 +* keep the system scalable as the corpus grows.
465 +)))
466 +* (((
467 +**Explicit review layer (REVIEW_ACTION, roles, etc.)**
468 +Separating “data” from “who reviewed what” keeps the model clean, and is exactly what you want for:
469 +
470 +* governance
471 +* permissions
472 +* audit trails
473 +* future trust scoring per user / role.
474 +)))
475 +
476 +----
477 +
478 +=== 1.2 Design decisions I’m locking in (based on our discussions) ===
479 +
480 +To make the model consistent and “state-of-the-art”, I will assume the following as //current intended design//:
481 +
482 +1. (((
483 +**Claims vs Scenarios**
484 +
485 +* CLAIM is the stable identity for “what people argue about”.
486 +* CLAIM_VERSION are individual phrasings / formulations / metadata.
487 +* (((
488 +SCENARIO belongs to a **CLAIM**, not to a specific CLAIM_VERSION.
489 +Rationale:
490 +
491 +* Many different phrasings share the //same// scenario.
492 +* You avoid duplicating scenarios per wording.
493 +)))
494 +* SCENARIO_VERSION holds detailed definitions, assumptions, boundaries, etc.
495 +)))
496 +1. (((
497 +**Version-specific reasoning**
498 +
499 +* **Verdicts** are always attached to SCENARIO_VERSION (not base SCENARIO).
500 +* **Evidence links** are between SCENARIO_VERSION and EVIDENCE_VERSION.
501 +→ This is what we agreed when we said //“SCENARIO_EVIDENCE_LINK should link the respective versions instead”//.
502 +)))
503 +1. (((
504 +**Clusters**
505 +
506 +* CLAIM_CLUSTER groups Claims (semantically close claims).
507 +* It is visible in **both diagrams** (Core Data Model and Data Use).
508 +)))
509 +1. (((
510 +**Review vs data**
511 +
512 +* (((
513 +All review happens **on versioned entities**:
514 +
515 +* CLAIM_VERSION
516 +* SCENARIO_VERSION
517 +* EVIDENCE_VERSION
518 +* SCENARIO_EVIDENCE_LINK_VERSION
519 +* VERDICT_VERSION
520 +)))
521 +* REVIEW_ACTION is the generic log of //who// did //what// on //which version//.
522 +)))
523 +1. (((
524 +**Users & roles**
525 +
526 +* USER has an attribute (or a linked entity) that distinguishes **technical users** from normal accounts.
527 +* We //keep// TECHNICAL_USER as a specialisation of USER (strictly technical accounts).
528 +* All human & technical accounts can hold roles via USER_ROLE_MEMBERSHIP.
529 +* (((
530 +Roles include:
531 +
532 +* READER
533 +* CONTRIBUTOR
534 +* TRUSTED_CONTRIBUTOR
535 +* REVIEWER
536 +* MODERATOR
537 +* SYSTEM_ADMIN / MAINTAINER
538 +* FEDERATION_OPERATOR
539 +* FEDERATION_ADMIN
540 +(all present in the Data Use ERD, but as rows of ROLE rather than separate entities).
541 +)))
542 +)))
543 +
544 +----
545 +
546 +=== 1.3 Gaps / potential problems ===
547 +
548 +These are the main issues & missing areas I see:
549 +
550 +1. (((
551 +**Versioning text in chapter 5 is currently too thin (‘…’ placeholders)**
552 +
553 +* (((
554 +The spec does not yet //verbally// spell out:
555 +
556 +* the identity vs version pattern, systematically
557 +* how re-evaluation triggers are derived from version changes
558 +* how this aligns with federation (which versions are replicated where).
559 +)))
560 +)))
561 +1. (((
562 +**No explicit “provenance granularity” in the model**
563 +
564 +* (((
565 +EVIDENCE is a single entity. For more advanced use cases, you may later want:
566 +
567 +* EVIDENCE_SOURCE (the whole article/report/video)
568 +* EVIDENCE_FRAGMENT (specific paragraph/clip with its own reliability, quote, etc.)
569 +)))
570 +* For now, I’ll keep EVIDENCE/EVIDENCE_VERSION as is, but I’ll mention this as a possible extension.
571 +)))
572 +1. (((
573 +**Review target polymorphism**
574 +
575 +* (((
576 +REVIEW_ACTION can apply to multiple entity types. In the diagram this shows as multiple relationships:
577 +
578 +* CLAIM_VERSION → REVIEW_ACTION
579 +* SCENARIO_VERSION → REVIEW_ACTION
580 +* etc.
581 +)))
582 +* A more “pure” relational modeling would use a generic “subjectType + subjectId” or an intermediate “REVIEW_TARGET” table.
583 +* For readability, I’ll keep the simpler multi-edge representation and mention the polymorphism in text.
584 +)))
585 +1. (((
586 +**Federation details missing from core ERD**
587 +
588 +* There is no explicit FEDERATION_NODE / REPLICATION_LOG in the Data Model chapter.
589 +* This is ok for “core logical data model”, but I’ll add a short note that federation metadata is handled in the Federation chapter and via additional entities.
590 +)))
591 +1. (((
592 +**Automation / AKEL artifacts left implicit**
593 +
594 +* (((
595 +The Data Model chapter currently doesn’t describe:
596 +
597 +* AKEL task queues
598 +* extraction runs
599 +* model versions
600 +)))
601 +* That’s fine for now; I’ll just clarify that those belong to a “Processing / AKEL” submodel, not the core logical data model.
602 +)))