Skip to Content
Last modified by Robert Schaub on 2025/12/24 20:35
From version 5.1
edited by Robert Schaub
on 2025/11/27 12:28
Change comment: There is no comment for this version
To version 2.2
edited by Robert Schaub
on 2025/11/27 12:08
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -1,171 +169,3 @@
1 -== 1. Overall analysis & review of the data model ==
2 -
3 -=== 1.1 Strengths of the current design ===
4 -
5 -* (((
6 -**Identity vs. version pattern**
7 -Using base entities plus version entities (CLAIM + CLAIM_VERSION, SCENARIO + SCENARIO_VERSION, etc.) is exactly how modern knowledge systems handle:
8 -
9 -* auditability
10 -* time evolution
11 -* re-evaluation triggers
12 -* federation and partial replication
13 -)))
14 -* (((
15 -**Scenario-centric reasoning**
16 -Separating //Claim// (what people argue about) from //Scenario// (interpretive frame) is very aligned with “truth landscape” style systems:
17 -
18 -* Scenarios explain //why people disagree//.
19 -* Verdicts are tied to specific scenario versions → avoids mixing incompatible assumptions.
20 -)))
21 -* **Evidence and verdicts as first-class entities**
22 -Evidence is explicit, linked to scenarios, and verdicts are per scenario. This matches good practice from fact-checking, scientific assessment panels, and trust graphs.
23 -* (((
24 -**Cluster level (CLAIM_CLUSTER)**
25 -Grouping related claims avoids duplication and lets you:
26 -
27 -* reuse scenarios across paraphrases
28 -* share embeddings / semantic search
29 -* keep the system scalable as the corpus grows.
30 -)))
31 -* (((
32 -**Explicit review layer (REVIEW_ACTION, roles, etc.)**
33 -Separating “data” from “who reviewed what” keeps the model clean, and is exactly what you want for:
34 -
35 -* governance
36 -* permissions
37 -* audit trails
38 -* future trust scoring per user / role.
39 -)))
40 -
41 -----
42 -
43 -=== 1.2 Design decisions I’m locking in (based on our discussions) ===
44 -
45 -To make the model consistent and “state-of-the-art”, I will assume the following as //current intended design//:
46 -
47 -1. (((
48 -**Claims vs Scenarios**
49 -
50 -* CLAIM is the stable identity for “what people argue about”.
51 -* CLAIM_VERSION are individual phrasings / formulations / metadata.
52 -* (((
53 -SCENARIO belongs to a **CLAIM**, not to a specific CLAIM_VERSION.
54 -Rationale:
55 -
56 -* Many different phrasings share the //same// scenario.
57 -* You avoid duplicating scenarios per wording.
58 -)))
59 -* SCENARIO_VERSION holds detailed definitions, assumptions, boundaries, etc.
60 -)))
61 -1. (((
62 -**Version-specific reasoning**
63 -
64 -* **Verdicts** are always attached to SCENARIO_VERSION (not base SCENARIO).
65 -* **Evidence links** are between SCENARIO_VERSION and EVIDENCE_VERSION.
66 -→ This is what we agreed when we said //“SCENARIO_EVIDENCE_LINK should link the respective versions instead”//.
67 -)))
68 -1. (((
69 -**Clusters**
70 -
71 -* CLAIM_CLUSTER groups Claims (semantically close claims).
72 -* It is visible in **both diagrams** (Core Data Model and Data Use).
73 -)))
74 -1. (((
75 -**Review vs data**
76 -
77 -* (((
78 -All review happens **on versioned entities**:
79 -
80 -* CLAIM_VERSION
81 -* SCENARIO_VERSION
82 -* EVIDENCE_VERSION
83 -* SCENARIO_EVIDENCE_LINK_VERSION
84 -* VERDICT_VERSION
85 -)))
86 -* REVIEW_ACTION is the generic log of //who// did //what// on //which version//.
87 -)))
88 -1. (((
89 -**Users & roles**
90 -
91 -* USER has an attribute (or a linked entity) that distinguishes **technical users** from normal accounts.
92 -* We //keep// TECHNICAL_USER as a specialisation of USER (strictly technical accounts).
93 -* All human & technical accounts can hold roles via USER_ROLE_MEMBERSHIP.
94 -* (((
95 -Roles include:
96 -
97 -* READER
98 -* CONTRIBUTOR
99 -* TRUSTED_CONTRIBUTOR
100 -* REVIEWER
101 -* MODERATOR
102 -* SYSTEM_ADMIN / MAINTAINER
103 -* FEDERATION_OPERATOR
104 -* FEDERATION_ADMIN
105 -(all present in the Data Use ERD, but as rows of ROLE rather than separate entities).
106 -)))
107 -)))
108 -
109 -----
110 -
111 -=== 1.3 Gaps / potential problems ===
112 -
113 -These are the main issues & missing areas I see:
114 -
115 -1. (((
116 -**Versioning text in chapter 5 is currently too thin (‘…’ placeholders)**
117 -
118 -* (((
119 -The spec does not yet //verbally// spell out:
120 -
121 -* the identity vs version pattern, systematically
122 -* how re-evaluation triggers are derived from version changes
123 -* how this aligns with federation (which versions are replicated where).
124 -)))
125 -)))
126 -1. (((
127 -**No explicit “provenance granularity” in the model**
128 -
129 -* (((
130 -EVIDENCE is a single entity. For more advanced use cases, you may later want:
131 -
132 -* EVIDENCE_SOURCE (the whole article/report/video)
133 -* EVIDENCE_FRAGMENT (specific paragraph/clip with its own reliability, quote, etc.)
134 -)))
135 -* For now, I’ll keep EVIDENCE/EVIDENCE_VERSION as is, but I’ll mention this as a possible extension.
136 -)))
137 -1. (((
138 -**Review target polymorphism**
139 -
140 -* (((
141 -REVIEW_ACTION can apply to multiple entity types. In the diagram this shows as multiple relationships:
142 -
143 -* CLAIM_VERSION → REVIEW_ACTION
144 -* SCENARIO_VERSION → REVIEW_ACTION
145 -* etc.
146 -)))
147 -* A more “pure” relational modeling would use a generic “subjectType + subjectId” or an intermediate “REVIEW_TARGET” table.
148 -* For readability, I’ll keep the simpler multi-edge representation and mention the polymorphism in text.
149 -)))
150 -1. (((
151 -**Federation details missing from core ERD**
152 -
153 -* There is no explicit FEDERATION_NODE / REPLICATION_LOG in the Data Model chapter.
154 -* This is ok for “core logical data model”, but I’ll add a short note that federation metadata is handled in the Federation chapter and via additional entities.
155 -)))
156 -1. (((
157 -**Automation / AKEL artifacts left implicit**
158 -
159 -* (((
160 -The Data Model chapter currently doesn’t describe:
161 -
162 -* AKEL task queues
163 -* extraction runs
164 -* model versions
165 -)))
166 -* That’s fine for now; I’ll just clarify that those belong to a “Processing / AKEL” submodel, not the core logical data model.
167 -)))
168 -
169 169  = 5. Data Model =
170 170  
171 171  The FactHarbor data model centers on four fully versioned, immutable entities:
... ... @@ -209,10 +209,10 @@
209 209   (scenarios live at the *claim* level, not per individual phrasing).
210 210  * Verdicts and Scenario–Evidence links are always attached to **versions**:
211 211  * {{code}}SCENARIO_VERSION{{/code}} +
212 -{{code}}EVIDENCE_VERSION{{/code}} →
213 -{{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
44 + {{code}}EVIDENCE_VERSION{{/code}} →
45 + {{code}}SCENARIO_EVIDENCE_LINK_VERSION{{/code}}
214 214  * {{code}}SCENARIO_VERSION{{/code}} →
215 -{{code}}VERDICT_VERSION{{/code}}
47 + {{code}}VERDICT_VERSION{{/code}}
216 216  
217 217  This ensures that when a Scenario or Evidence changes, old verdicts and links
218 218  remain intact as historical records and can be revisited.
... ... @@ -225,9 +225,99 @@
225 225  The convention is that fields ending in {{code}}Id{{/code}} are primary keys,
226 226  and fields with {{code}}...IdFk{{/code}} are foreign keys.
227 227  
228 -{{comment}} Core Data Model ERD (Mermaid, from /Specification/Diagrams/Data Model) {{/comment}}
229 -{{include document="FactHarbor.Playground.Core Data Model ERD Page (from Specification chat).WebHome"/}}
60 +{{mermaid}}
61 +erDiagram
62 + CLAIM_CLUSTER {
63 + string ClusterID PK
64 + string EmbeddingVectorRef
65 + string Theme
66 + }
230 230  
68 + CLAIM {
69 + string ClaimID PK
70 + string ClusterID FK
71 + string Status
72 + datetime CreatedAt
73 + }
74 +
75 + CLAIM_VERSION {
76 + string ClaimVersionID PK
77 + string ClaimID FK
78 + string Text
79 + string ClaimType
80 + string Domain
81 + datetime CreatedAt
82 + }
83 +
84 + SCENARIO {
85 + string ScenarioID PK
86 + string ClaimID FK
87 + string Name
88 + datetime CreatedAt
89 + }
90 +
91 + SCENARIO_VERSION {
92 + string ScenarioVersionID PK
93 + string ScenarioID FK
94 + string Definitions
95 + string Assumptions
96 + string Boundaries
97 + datetime CreatedAt
98 + }
99 +
100 + EVIDENCE {
101 + string EvidenceID PK
102 + string SourceType
103 + string URL
104 + float ReliabilityScore
105 + }
106 +
107 + EVIDENCE_VERSION {
108 + string EvidenceVersionID PK
109 + string EvidenceID FK
110 + string Summary
111 + float ReliabilityScore
112 + datetime CreatedAt
113 + }
114 +
115 + SCENARIO_EVIDENCE_LINK {
116 + string LinkID PK
117 + string ScenarioVersionID FK
118 + string EvidenceVersionID FK
119 + float Relevance
120 + string Direction
121 + }
122 +
123 + VERDICT {
124 + string VerdictID PK
125 + string ScenarioID FK
126 + }
127 +
128 + VERDICT_VERSION {
129 + string VerdictVersionID PK
130 + string VerdictID FK
131 + float Verdict
132 + float Confidence
133 + string Reasoning
134 + datetime CreatedAt
135 + }
136 +
137 + CLAIM_CLUSTER ||--o{ CLAIM : contains
138 + CLAIM ||--o{ CLAIM_VERSION : versions
139 +
140 + CLAIM ||--o{ SCENARIO : has
141 + SCENARIO ||--o{ SCENARIO_VERSION : versions
142 +
143 + EVIDENCE ||--o{ EVIDENCE_VERSION : versions
144 +
145 + SCENARIO_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : links
146 + EVIDENCE_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : linked
147 +
148 + SCENARIO ||--o{ VERDICT : assessed
149 + VERDICT ||--o{ VERDICT_VERSION : versions
150 +
151 +{{/mermaid}}
152 +
231 231  **Important points:**
232 232  
233 233  * Scenarios and Evidence are **linked via their versions**
... ... @@ -248,9 +248,149 @@
248 248  * Roles and role assignments
249 249  * Review actions on versioned entities
250 250  
251 -{{comment}} Data Use ERD (Mermaid, from /Specification/Diagrams/Data Use ERD) {{/comment}}
252 -{{include document="FactHarbor.Playground.Data Use ERD Page (from Specification chat).WebHome"/}}
173 +{{mermaid}}
174 +erDiagram
175 + %% Core clusters shown for context
176 + CLAIM_CLUSTER {
177 + string ClusterID PK
178 + string EmbeddingVectorRef
179 + string Theme
180 + }
253 253  
182 + CLAIM {
183 + string ClaimID PK
184 + string ClusterID FK
185 + string Status
186 + datetime CreatedAt
187 + }
188 +
189 + CLAIM_VERSION {
190 + string ClaimVersionID PK
191 + string ClaimID FK
192 + string Text
193 + string ClaimType
194 + string Domain
195 + datetime CreatedAt
196 + }
197 +
198 + SCENARIO {
199 + string ScenarioID PK
200 + string ClaimID FK
201 + string Name
202 + datetime CreatedAt
203 + }
204 +
205 + SCENARIO_VERSION {
206 + string ScenarioVersionID PK
207 + string ScenarioID FK
208 + string Definitions
209 + string Assumptions
210 + string Boundaries
211 + datetime CreatedAt
212 + }
213 +
214 + EVIDENCE {
215 + string EvidenceID PK
216 + string SourceType
217 + string URL
218 + float ReliabilityScore
219 + }
220 +
221 + EVIDENCE_VERSION {
222 + string EvidenceVersionID PK
223 + string EvidenceID FK
224 + string Summary
225 + float ReliabilityScore
226 + datetime CreatedAt
227 + }
228 +
229 + VERDICT {
230 + string VerdictID PK
231 + string ScenarioID FK
232 + }
233 +
234 + VERDICT_VERSION {
235 + string VerdictVersionID PK
236 + string VerdictID FK
237 + float Verdict
238 + float Confidence
239 + string Reasoning
240 + datetime CreatedAt
241 + }
242 +
243 + %% Users and roles
244 + USER {
245 + string UserID PK
246 + string Handle
247 + string Email
248 + }
249 +
250 + TECHNICAL_USER {
251 + string UserID PK
252 + string SystemName
253 + }
254 +
255 + CONTRIBUTING_USER {
256 + string UserID PK
257 + string DisplayName
258 + }
259 +
260 + TRUSTED_CONTRIBUTOR {
261 + string UserID PK
262 + string TrustLevel
263 + }
264 +
265 + REVIEWER {
266 + string UserID PK
267 + string Domain
268 + }
269 +
270 + EXPERT {
271 + string UserID PK
272 + string ExpertiseArea
273 + }
274 +
275 + FEDERATION_NODE {
276 + string NodeID PK
277 + string Region
278 + }
279 +
280 + FEDERATION_ADMIN {
281 + string UserID PK
282 + string Permissions
283 + }
284 +
285 + REVIEW_ACTION {
286 + string ReviewActionID PK
287 + string UserID FK
288 + string TargetEntityType
289 + string TargetEntityVersionID
290 + string ActionType
291 + string Comment
292 + datetime Timestamp
293 + }
294 +
295 + %% Inheritance / specialization (modelled as relationships)
296 + USER ||--o{ TECHNICAL_USER : "is a"
297 + USER ||--o{ CONTRIBUTING_USER : "is a"
298 +
299 + CONTRIBUTING_USER ||--o{ TRUSTED_CONTRIBUTOR : "subset"
300 + CONTRIBUTING_USER ||--o{ REVIEWER : "subset"
301 + CONTRIBUTING_USER ||--o{ EXPERT : "subset"
302 +
303 + TECHNICAL_USER ||--o{ FEDERATION_NODE : "operates"
304 + TECHNICAL_USER ||--o{ FEDERATION_ADMIN : "administers"
305 +
306 + %% Review actions on versioned entities
307 + USER ||--o{ REVIEW_ACTION : performs
308 +
309 + REVIEW_ACTION }o--|| CLAIM_VERSION : reviews
310 + REVIEW_ACTION }o--|| SCENARIO_VERSION : reviews
311 + REVIEW_ACTION }o--|| EVIDENCE_VERSION : reviews
312 + REVIEW_ACTION }o--|| VERDICT_VERSION : reviews
313 +
314 +{{/mermaid}}
315 +
254 254  Notes:
255 255  
256 256  * Most roles (READER, CONTRIBUTOR, TRUSTED_CONTRIBUTOR, REVIEWER, MODERATOR,
... ... @@ -280,7 +280,7 @@
280 280  * It may inherit some links from earlier scenarios, or start empty depending
281 281   on the change classification (cosmetic vs. conceptual).
282 282  * All verdicts for that scenario are recalculated and stored as new
283 -{{code}}VERDICT_VERSION{{/code}} entries.
345 + {{code}}VERDICT_VERSION{{/code}} entries.
284 284  
285 285  * REVIEW_ACTIONs are always attached to the **exact version** that was seen by
286 286   the reviewer. This preserves a faithful audit trail if data later changes.