Skip to Content
Last modified by Robert Schaub on 2025/12/24 20:35
From version 3.1
edited by Robert Schaub
on 2025/11/27 12:08
Change comment: There is no comment for this version
To version 5.1
edited by Robert Schaub
on 2025/11/27 12:28
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -1,3 +1,171 @@
1 +== 1. Overall analysis & review of the data model ==
2 +
3 +=== 1.1 Strengths of the current design ===
4 +
5 +* (((
6 +**Identity vs. version pattern**
7 +Using base entities plus version entities (CLAIM + CLAIM_VERSION, SCENARIO + SCENARIO_VERSION, etc.) is exactly how modern knowledge systems handle:
8 +
9 +* auditability
10 +* time evolution
11 +* re-evaluation triggers
12 +* federation and partial replication
13 +)))
14 +* (((
15 +**Scenario-centric reasoning**
16 +Separating //Claim// (what people argue about) from //Scenario// (interpretive frame) is very aligned with “truth landscape” style systems:
17 +
18 +* Scenarios explain //why people disagree//.
19 +* Verdicts are tied to specific scenario versions → avoids mixing incompatible assumptions.
20 +)))
21 +* **Evidence and verdicts as first-class entities**
22 +Evidence is explicit, linked to scenarios, and verdicts are per scenario. This matches good practice from fact-checking, scientific assessment panels, and trust graphs.
23 +* (((
24 +**Cluster level (CLAIM_CLUSTER)**
25 +Grouping related claims avoids duplication and lets you:
26 +
27 +* reuse scenarios across paraphrases
28 +* share embeddings / semantic search
29 +* keep the system scalable as the corpus grows.
30 +)))
31 +* (((
32 +**Explicit review layer (REVIEW_ACTION, roles, etc.)**
33 +Separating “data” from “who reviewed what” keeps the model clean, and is exactly what you want for:
34 +
35 +* governance
36 +* permissions
37 +* audit trails
38 +* future trust scoring per user / role.
39 +)))
40 +
41 +----
42 +
43 +=== 1.2 Design decisions I’m locking in (based on our discussions) ===
44 +
45 +To make the model consistent and “state-of-the-art”, I will assume the following as //current intended design//:
46 +
47 +1. (((
48 +**Claims vs Scenarios**
49 +
50 +* CLAIM is the stable identity for “what people argue about”.
51 +* CLAIM_VERSION are individual phrasings / formulations / metadata.
52 +* (((
53 +SCENARIO belongs to a **CLAIM**, not to a specific CLAIM_VERSION.
54 +Rationale:
55 +
56 +* Many different phrasings share the //same// scenario.
57 +* You avoid duplicating scenarios per wording.
58 +)))
59 +* SCENARIO_VERSION holds detailed definitions, assumptions, boundaries, etc.
60 +)))
61 +1. (((
62 +**Version-specific reasoning**
63 +
64 +* **Verdicts** are always attached to SCENARIO_VERSION (not base SCENARIO).
65 +* **Evidence links** are between SCENARIO_VERSION and EVIDENCE_VERSION.
66 +→ This is what we agreed when we said //“SCENARIO_EVIDENCE_LINK should link the respective versions instead”//.
67 +)))
68 +1. (((
69 +**Clusters**
70 +
71 +* CLAIM_CLUSTER groups Claims (semantically close claims).
72 +* It is visible in **both diagrams** (Core Data Model and Data Use).
73 +)))
74 +1. (((
75 +**Review vs data**
76 +
77 +* (((
78 +All review happens **on versioned entities**:
79 +
80 +* CLAIM_VERSION
81 +* SCENARIO_VERSION
82 +* EVIDENCE_VERSION
83 +* SCENARIO_EVIDENCE_LINK_VERSION
84 +* VERDICT_VERSION
85 +)))
86 +* REVIEW_ACTION is the generic log of //who// did //what// on //which version//.
87 +)))
88 +1. (((
89 +**Users & roles**
90 +
91 +* USER has an attribute (or a linked entity) that distinguishes **technical users** from normal accounts.
92 +* We //keep// TECHNICAL_USER as a specialisation of USER (strictly technical accounts).
93 +* All human & technical accounts can hold roles via USER_ROLE_MEMBERSHIP.
94 +* (((
95 +Roles include:
96 +
97 +* READER
98 +* CONTRIBUTOR
99 +* TRUSTED_CONTRIBUTOR
100 +* REVIEWER
101 +* MODERATOR
102 +* SYSTEM_ADMIN / MAINTAINER
103 +* FEDERATION_OPERATOR
104 +* FEDERATION_ADMIN
105 +(all present in the Data Use ERD, but as rows of ROLE rather than separate entities).
106 +)))
107 +)))
108 +
109 +----
110 +
111 +=== 1.3 Gaps / potential problems ===
112 +
113 +These are the main issues & missing areas I see:
114 +
115 +1. (((
116 +**Versioning text in chapter 5 is currently too thin (‘…’ placeholders)**
117 +
118 +* (((
119 +The spec does not yet //verbally// spell out:
120 +
121 +* the identity vs version pattern, systematically
122 +* how re-evaluation triggers are derived from version changes
123 +* how this aligns with federation (which versions are replicated where).
124 +)))
125 +)))
126 +1. (((
127 +**No explicit “provenance granularity” in the model**
128 +
129 +* (((
130 +EVIDENCE is a single entity. For more advanced use cases, you may later want:
131 +
132 +* EVIDENCE_SOURCE (the whole article/report/video)
133 +* EVIDENCE_FRAGMENT (specific paragraph/clip with its own reliability, quote, etc.)
134 +)))
135 +* For now, I’ll keep EVIDENCE/EVIDENCE_VERSION as is, but I’ll mention this as a possible extension.
136 +)))
137 +1. (((
138 +**Review target polymorphism**
139 +
140 +* (((
141 +REVIEW_ACTION can apply to multiple entity types. In the diagram this shows as multiple relationships:
142 +
143 +* CLAIM_VERSION → REVIEW_ACTION
144 +* SCENARIO_VERSION → REVIEW_ACTION
145 +* etc.
146 +)))
147 +* A more “pure” relational modeling would use a generic “subjectType + subjectId” or an intermediate “REVIEW_TARGET” table.
148 +* For readability, I’ll keep the simpler multi-edge representation and mention the polymorphism in text.
149 +)))
150 +1. (((
151 +**Federation details missing from core ERD**
152 +
153 +* There is no explicit FEDERATION_NODE / REPLICATION_LOG in the Data Model chapter.
154 +* This is ok for “core logical data model”, but I’ll add a short note that federation metadata is handled in the Federation chapter and via additional entities.
155 +)))
156 +1. (((
157 +**Automation / AKEL artifacts left implicit**
158 +
159 +* (((
160 +The Data Model chapter currently doesn’t describe:
161 +
162 +* AKEL task queues
163 +* extraction runs
164 +* model versions
165 +)))
166 +* That’s fine for now; I’ll just clarify that those belong to a “Processing / AKEL” submodel, not the core logical data model.
167 +)))
168 +
1 1  = 5. Data Model =
2 2  
3 3  The FactHarbor data model centers on four fully versioned, immutable entities:
... ... @@ -57,99 +57,9 @@
57 57  The convention is that fields ending in {{code}}Id{{/code}} are primary keys,
58 58  and fields with {{code}}...IdFk{{/code}} are foreign keys.
59 59  
60 -{{mermaid}}
61 -erDiagram
62 - CLAIM_CLUSTER {
63 - string ClusterID PK
64 - string EmbeddingVectorRef
65 - string Theme
66 - }
228 +{{comment}} Core Data Model ERD (Mermaid, from /Specification/Diagrams/Data Model) {{/comment}}
229 +{{include document="FactHarbor.Playground.Core Data Model ERD Page (from Specification chat).WebHome"/}}
67 67  
68 - CLAIM {
69 - string ClaimID PK
70 - string ClusterID FK
71 - string Status
72 - datetime CreatedAt
73 - }
74 -
75 - CLAIM_VERSION {
76 - string ClaimVersionID PK
77 - string ClaimID FK
78 - string Text
79 - string ClaimType
80 - string Domain
81 - datetime CreatedAt
82 - }
83 -
84 - SCENARIO {
85 - string ScenarioID PK
86 - string ClaimID FK
87 - string Name
88 - datetime CreatedAt
89 - }
90 -
91 - SCENARIO_VERSION {
92 - string ScenarioVersionID PK
93 - string ScenarioID FK
94 - string Definitions
95 - string Assumptions
96 - string Boundaries
97 - datetime CreatedAt
98 - }
99 -
100 - EVIDENCE {
101 - string EvidenceID PK
102 - string SourceType
103 - string URL
104 - float ReliabilityScore
105 - }
106 -
107 - EVIDENCE_VERSION {
108 - string EvidenceVersionID PK
109 - string EvidenceID FK
110 - string Summary
111 - float ReliabilityScore
112 - datetime CreatedAt
113 - }
114 -
115 - SCENARIO_EVIDENCE_LINK {
116 - string LinkID PK
117 - string ScenarioVersionID FK
118 - string EvidenceVersionID FK
119 - float Relevance
120 - string Direction
121 - }
122 -
123 - VERDICT {
124 - string VerdictID PK
125 - string ScenarioID FK
126 - }
127 -
128 - VERDICT_VERSION {
129 - string VerdictVersionID PK
130 - string VerdictID FK
131 - float Verdict
132 - float Confidence
133 - string Reasoning
134 - datetime CreatedAt
135 - }
136 -
137 - CLAIM_CLUSTER ||--o{ CLAIM : contains
138 - CLAIM ||--o{ CLAIM_VERSION : versions
139 -
140 - CLAIM ||--o{ SCENARIO : has
141 - SCENARIO ||--o{ SCENARIO_VERSION : versions
142 -
143 - EVIDENCE ||--o{ EVIDENCE_VERSION : versions
144 -
145 - SCENARIO_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : links
146 - EVIDENCE_VERSION ||--o{ SCENARIO_EVIDENCE_LINK : linked
147 -
148 - SCENARIO ||--o{ VERDICT : assessed
149 - VERDICT ||--o{ VERDICT_VERSION : versions
150 -
151 -{{/mermaid}}
152 -
153 153  **Important points:**
154 154  
155 155  * Scenarios and Evidence are **linked via their versions**
... ... @@ -170,149 +170,9 @@
170 170  * Roles and role assignments
171 171  * Review actions on versioned entities
172 172  
173 -{{mermaid}}
174 -erDiagram
175 - %% Core clusters shown for context
176 - CLAIM_CLUSTER {
177 - string ClusterID PK
178 - string EmbeddingVectorRef
179 - string Theme
180 - }
251 +{{comment}} Data Use ERD (Mermaid, from /Specification/Diagrams/Data Use ERD) {{/comment}}
252 +{{include document="FactHarbor.Playground.Data Use ERD Page (from Specification chat).WebHome"/}}
181 181  
182 - CLAIM {
183 - string ClaimID PK
184 - string ClusterID FK
185 - string Status
186 - datetime CreatedAt
187 - }
188 -
189 - CLAIM_VERSION {
190 - string ClaimVersionID PK
191 - string ClaimID FK
192 - string Text
193 - string ClaimType
194 - string Domain
195 - datetime CreatedAt
196 - }
197 -
198 - SCENARIO {
199 - string ScenarioID PK
200 - string ClaimID FK
201 - string Name
202 - datetime CreatedAt
203 - }
204 -
205 - SCENARIO_VERSION {
206 - string ScenarioVersionID PK
207 - string ScenarioID FK
208 - string Definitions
209 - string Assumptions
210 - string Boundaries
211 - datetime CreatedAt
212 - }
213 -
214 - EVIDENCE {
215 - string EvidenceID PK
216 - string SourceType
217 - string URL
218 - float ReliabilityScore
219 - }
220 -
221 - EVIDENCE_VERSION {
222 - string EvidenceVersionID PK
223 - string EvidenceID FK
224 - string Summary
225 - float ReliabilityScore
226 - datetime CreatedAt
227 - }
228 -
229 - VERDICT {
230 - string VerdictID PK
231 - string ScenarioID FK
232 - }
233 -
234 - VERDICT_VERSION {
235 - string VerdictVersionID PK
236 - string VerdictID FK
237 - float Verdict
238 - float Confidence
239 - string Reasoning
240 - datetime CreatedAt
241 - }
242 -
243 - %% Users and roles
244 - USER {
245 - string UserID PK
246 - string Handle
247 - string Email
248 - }
249 -
250 - TECHNICAL_USER {
251 - string UserID PK
252 - string SystemName
253 - }
254 -
255 - CONTRIBUTING_USER {
256 - string UserID PK
257 - string DisplayName
258 - }
259 -
260 - TRUSTED_CONTRIBUTOR {
261 - string UserID PK
262 - string TrustLevel
263 - }
264 -
265 - REVIEWER {
266 - string UserID PK
267 - string Domain
268 - }
269 -
270 - EXPERT {
271 - string UserID PK
272 - string ExpertiseArea
273 - }
274 -
275 - FEDERATION_NODE {
276 - string NodeID PK
277 - string Region
278 - }
279 -
280 - FEDERATION_ADMIN {
281 - string UserID PK
282 - string Permissions
283 - }
284 -
285 - REVIEW_ACTION {
286 - string ReviewActionID PK
287 - string UserID FK
288 - string TargetEntityType
289 - string TargetEntityVersionID
290 - string ActionType
291 - string Comment
292 - datetime Timestamp
293 - }
294 -
295 - %% Inheritance / specialization (modelled as relationships)
296 - USER ||--o{ TECHNICAL_USER : "is a"
297 - USER ||--o{ CONTRIBUTING_USER : "is a"
298 -
299 - CONTRIBUTING_USER ||--o{ TRUSTED_CONTRIBUTOR : "subset"
300 - CONTRIBUTING_USER ||--o{ REVIEWER : "subset"
301 - CONTRIBUTING_USER ||--o{ EXPERT : "subset"
302 -
303 - TECHNICAL_USER ||--o{ FEDERATION_NODE : "operates"
304 - TECHNICAL_USER ||--o{ FEDERATION_ADMIN : "administers"
305 -
306 - %% Review actions on versioned entities
307 - USER ||--o{ REVIEW_ACTION : performs
308 -
309 - REVIEW_ACTION }o--|| CLAIM_VERSION : reviews
310 - REVIEW_ACTION }o--|| SCENARIO_VERSION : reviews
311 - REVIEW_ACTION }o--|| EVIDENCE_VERSION : reviews
312 - REVIEW_ACTION }o--|| VERDICT_VERSION : reviews
313 -
314 -{{/mermaid}}
315 -
316 316  Notes:
317 317  
318 318  * Most roles (READER, CONTRIBUTOR, TRUSTED_CONTRIBUTOR, REVIEWER, MODERATOR,