Skip to Content

Wiki source code of Requirements

Version 2.1 by Robert Schaub on 2025/12/19 09:13
Show last authors
1 = Requirements =
2
3 **This page defines Roles, Content States, Rules, and System Requirements for FactHarbor.**
4
5 **Core Philosophy:** Invest in system improvement, not manual data correction. When AI makes errors, improve the algorithm and re-process automatically.
6
7 == Navigation ==
8
9 * **[[User Needs>>FactHarbor.Specification.Requirements.User-Needs]]** - What users need from FactHarbor (drives these requirements)
10 * **This page** - How we fulfill those needs through system design
11
12 (% class="box infomessage" %)
13 (((
14 **How to read this page:**
15
16 1. **User Needs drive Requirements**: See [[User Needs>>FactHarbor.Specification.Requirements.User-Needs]] for what users need
17 2. **Requirements define implementation**: This page shows how we fulfill those needs
18 3. **Functional Requirements (FR)**: Specific features and capabilities
19 4. **Non-Functional Requirements (NFR)**: Quality attributes (performance, security, etc.)
20
21 Each requirement references which User Needs it fulfills.
22 )))
23
24 == 1. Roles ==
25
26 **Fulfills**: UN-12 (Submit claims), UN-13 (Cite verdicts), UN-14 (API access)
27
28 FactHarbor uses three simple roles plus a reputation system.
29
30 === 1.1 Reader ===
31
32 **Who**: Anyone (no login required)
33
34 **Can**:
35 * Browse and search claims
36 * View scenarios, evidence, verdicts, and confidence scores
37 * Flag issues or errors
38 * Use filters, search, and visualization tools
39 * Submit claims automatically (new claims added if not duplicates)
40
41 **Cannot**:
42 * Modify content
43 * Access edit history details
44
45 **User Needs served**: UN-1 (Trust assessment), UN-2 (Claim verification), UN-3 (Summary + verdict), UN-4 (Social media fact-checking), UN-5 (Source tracing), UN-7 (Evidence transparency), UN-8 (Understanding disagreement), UN-12 (Submit claims)
46
47 === 1.2 Contributor ===
48
49 **Who**: Registered users (earns reputation through contributions)
50
51 **Can**:
52 * Everything a Reader can do
53 * Edit claims, evidence, and scenarios
54 * Add sources and citations
55 * Suggest improvements to AI-generated content
56 * Participate in discussions
57 * Earn reputation points for quality contributions
58
59 **Reputation System**:
60 * New contributors: Limited edit privileges
61 * Established contributors (established reputation): Full edit access
62 * Trusted contributors (substantial reputation): Can approve certain changes
63 * Reputation earned through: Accepted edits, helpful flags, quality contributions
64 * Reputation lost through: Reverted edits, invalid flags, abuse
65
66 **Cannot**:
67 * Delete or hide content (only moderators)
68 * Override moderation decisions
69
70 **User Needs served**: UN-13 (Cite and contribute)
71
72 === 1.3 Moderator ===
73
74 **Who**: Trusted community members with proven track record, appointed by governance board
75
76 **Can**:
77 * Review flagged content
78 * Hide harmful or abusive content
79 * Resolve disputes between contributors
80 * Issue warnings or temporary bans
81 * Make final decisions on content disputes
82 * Access full audit logs
83
84 **Cannot**:
85 * Change governance rules
86 * Permanently ban users without board approval
87 * Override technical quality gates
88
89 **Note**: Small team (3-5 initially), supported by automated moderation tools.
90
91 === 1.4 Domain Trusted Contributors (Optional, Task-Specific) ===
92
93 **Who**: Subject matter specialists invited for specific high-stakes disputes
94
95 **Not a permanent role**: Contacted externally when needed for contested claims in their domain
96
97 **When used**:
98 * Medical claims with life/safety implications
99 * Legal interpretations with significant impact
100 * Scientific claims with high controversy
101 * Technical claims requiring specialized knowledge
102
103 **Process**:
104 * Moderator identifies need for expert input
105 * Contact expert externally (don't require them to be users)
106 * Trusted Contributor provides written opinion with sources
107 * Opinion added to claim record
108 * Trusted Contributor acknowledged in claim
109
110 **User Needs served**: UN-16 (Expert validation status)
111
112 == 2. Content States ==
113
114 **Fulfills**: UN-1 (Trust indicators), UN-16 (Review status transparency)
115
116 FactHarbor uses two content states. Focus is on transparency and confidence scoring, not gatekeeping.
117
118 === 2.1 Published ===
119
120 **Status**: Visible to all users
121
122 **Includes**:
123 * AI-generated analyses (default state)
124 * User-contributed content
125 * Edited/improved content
126
127 **Quality Indicators** (displayed with content):
128 * **Confidence Score**: 0-100% (AI's confidence in analysis)
129 * **Source Quality Score**: 0-100% (based on source track record)
130 * **Controversy Flag**: If high dispute/edit activity
131 * **Completeness Score**: % of expected fields filled
132 * **Last Updated**: Date of most recent change
133 * **Edit Count**: Number of revisions
134 * **Review Status**: AI-generated / Human-reviewed / Expert-validated
135
136 **Automatic Warnings**:
137 * Confidence < 60%: "Low confidence - use caution"
138 * Source quality < 40%: "Sources may be unreliable"
139 * High controversy: "Disputed - multiple interpretations exist"
140 * Medical/Legal/Safety domain: "Seek professional advice"
141
142 **User Needs served**: UN-1 (Trust score), UN-9 (Methodology transparency), UN-15 (Evolution timeline), UN-16 (Review status)
143
144 === 2.2 Hidden ===
145
146 **Status**: Not visible to regular users (only to moderators)
147
148 **Reasons**:
149 * Spam or advertising
150 * Personal attacks or harassment
151 * Illegal content
152 * Privacy violations
153 * Deliberate misinformation (verified)
154 * Abuse or harmful content
155
156 **Process**:
157 * Automated detection flags for moderator review
158 * Moderator confirms and hides
159 * Original author notified with reason
160 * Can appeal to board if disputes moderator decision
161
162 **Note**: Content is hidden, not deleted (for audit trail)
163
164 == 3. Contribution Rules ==
165
166 === 3.1 All Contributors Must ===
167
168 * Provide sources for factual claims
169 * Use clear, neutral language in FactHarbor's own summaries
170 * Respect others and maintain civil discourse
171 * Accept community feedback constructively
172 * Focus on improving quality, not protecting ego
173
174 === 3.2 AKEL (AI System) ===
175
176 **AKEL is the primary system**. Human contributions supplement and train AKEL.
177
178 **AKEL Must**:
179 * Mark all outputs as AI-generated
180 * Display confidence scores prominently
181 * Provide source citations
182 * Flag uncertainty clearly
183 * Identify contradictions in evidence
184 * Learn from human corrections
185
186 **When AKEL Makes Errors**:
187 1. Capture the error pattern (what, why, how common)
188 2. Improve the system (better prompt, model, validation)
189 3. Re-process affected claims automatically
190 4. Measure improvement (did quality increase?)
191
192 **Human Role**: Train AKEL through corrections, not replace AKEL
193
194 === 3.3 Contributors Should ===
195
196 * Improve clarity and structure
197 * Add missing sources
198 * Flag errors for system improvement
199 * Suggest better ways to present information
200 * Participate in quality discussions
201
202 === 3.4 Moderators Must ===
203
204 * Be impartial
205 * Document moderation decisions
206 * Respond to appeals promptly
207 * Use automated tools to scale efforts
208 * Focus on abuse/harm, not routine quality control
209
210 == 4. Quality Standards ==
211
212 **Fulfills**: UN-5 (Source reliability), UN-6 (Publisher track records), UN-7 (Evidence transparency), UN-9 (Methodology transparency)
213
214 === 4.1 Source Requirements ===
215
216 **Track Record Over Credentials**:
217 * Sources evaluated by historical accuracy
218 * Correction policy matters
219 * Independence from conflicts of interest
220 * Methodology transparency
221
222 **Source Quality Database**:
223 * Automated tracking of source accuracy
224 * Correction frequency
225 * Reliability score (updated continuously)
226 * Users can see source track record
227
228 **No automatic trust** for government, academia, or media - all evaluated by track record.
229
230 **User Needs served**: UN-5 (Source provenance), UN-6 (Publisher reliability)
231
232 === 4.2 Claim Requirements ===
233
234 * Clear subject and assertion
235 * Verifiable with available information
236 * Sourced (or explicitly marked as needing sources)
237 * Neutral language in FactHarbor summaries
238 * Appropriate context provided
239
240 **User Needs served**: UN-2 (Claim extraction and verification)
241
242 === 4.3 Evidence Requirements ===
243
244 * Publicly accessible (or explain why not)
245 * Properly cited with attribution
246 * Relevant to claim being evaluated
247 * Original source preferred over secondary
248
249 **User Needs served**: UN-7 (Evidence transparency)
250
251 === 4.4 Confidence Scoring ===
252
253 **Automated confidence calculation based on**:
254 * Source quality scores
255 * Evidence consistency
256 * Contradiction detection
257 * Completeness of analysis
258 * Historical accuracy of similar claims
259
260 **Thresholds**:
261 * < 40%: Too low to publish (needs improvement)
262 * 40-60%: Published with "Low confidence" warning
263 * 60-80%: Published as standard
264 * 80-100%: Published as "High confidence"
265
266 **User Needs served**: UN-1 (Trust assessment), UN-9 (Methodology transparency)
267
268 == 5. Automated Risk Scoring ==
269
270 **Fulfills**: UN-10 (Manipulation detection), UN-16 (Appropriate review level)
271
272 **Replace manual risk tiers with continuous automated scoring**.
273
274 === 5.1 Risk Score Calculation ===
275
276 **Factors** (weighted algorithm):
277 * **Domain sensitivity**: Medical, legal, safety auto-flagged higher
278 * **Potential impact**: Views, citations, spread
279 * **Controversy level**: Flags, disputes, edit wars
280 * **Uncertainty**: Low confidence, contradictory evidence
281 * **Source reliability**: Track record of sources used
282
283 **Score**: 0-100 (higher = more risk)
284
285 === 5.2 Automated Actions ===
286
287 * **Score > 80**: Flag for moderator review before publication
288 * **Score 60-80**: Publish with prominent warnings
289 * **Score 40-60**: Publish with standard warnings
290 * **Score < 40**: Publish normally
291
292 **Continuous monitoring**: Risk score recalculated as new information emerges
293
294 **User Needs served**: UN-10 (Detect manipulation tactics), UN-16 (Review status)
295
296 == 6. System Improvement Process ==
297
298 **Core principle**: Fix the system, not just the data.
299
300 === 6.1 Error Capture ===
301
302 **When users flag errors or make corrections**:
303 1. What was wrong? (categorize)
304 2. What should it have been?
305 3. Why did the system fail? (root cause)
306 4. How common is this pattern?
307 5. Store in ErrorPattern table (improvement queue)
308
309 === 6.2 Weekly Improvement Cycle ===
310
311 1. **Review**: Analyze top error patterns
312 2. **Develop**: Create fix (prompt, model, validation)
313 3. **Test**: Validate fix on sample claims
314 4. **Deploy**: Roll out if quality improves
315 5. **Re-process**: Automatically update affected claims
316 6. **Monitor**: Track quality metrics
317
318 === 6.3 Quality Metrics Dashboard ===
319
320 **Track continuously**:
321 * Error rate by category
322 * Source quality distribution
323 * Confidence score trends
324 * User flag rate (issues found)
325 * Correction acceptance rate
326 * Re-work rate
327 * Claims processed per hour
328
329 **Goal**: 10% monthly improvement in error rate
330
331 == 7. Automated Quality Monitoring ==
332
333 **Replace manual audit sampling with automated monitoring**.
334
335 === 7.1 Continuous Metrics ===
336
337 * **Source quality**: Track record database
338 * **Consistency**: Contradiction detection
339 * **Clarity**: Readability scores
340 * **Completeness**: Field validation
341 * **Accuracy**: User corrections tracked
342
343 === 7.2 Anomaly Detection ===
344
345 **Automated alerts for**:
346 * Sudden quality drops
347 * Unusual patterns
348 * Contradiction clusters
349 * Source reliability changes
350 * User behavior anomalies
351
352 === 7.3 Targeted Review ===
353
354 * Review only flagged items
355 * Random sampling for calibration (not quotas)
356 * Learn from corrections to improve automation
357
358 == 8. Functional Requirements ==
359
360 This section defines specific features that fulfill user needs.
361
362 === 8.1 Claim Intake & Normalization ===
363
364 ==== FR1 — Claim Intake ====
365
366 **Fulfills**: UN-2 (Claim extraction), UN-4 (Quick fact-checking), UN-12 (Submit claims)
367
368 * Users submit claims via simple form or API
369 * Claims can be text, URL, or image
370 * Duplicate detection (semantic similarity)
371 * Auto-categorization by domain
372
373 ==== FR2 — Claim Normalization ====
374
375 **Fulfills**: UN-2 (Claim verification)
376
377 * Standardize to clear assertion format
378 * Extract key entities (who, what, when, where)
379 * Identify claim type (factual, predictive, evaluative)
380 * Link to existing similar claims
381
382 ==== FR3 — Claim Classification ====
383
384 **Fulfills**: UN-11 (Filtered research)
385
386 * Domain: Politics, Science, Health, etc.
387 * Type: Historical fact, current stat, prediction, etc.
388 * Risk score: Automated calculation
389 * Complexity: Simple, moderate, complex
390
391 === 8.2 Scenario System ===
392
393 ==== FR4 — Scenario Generation ====
394
395 **Fulfills**: UN-2 (Context-dependent verification), UN-3 (Summary with scenarios), UN-8 (Understanding disagreement)
396
397 **Automated scenario creation**:
398 * AKEL analyzes claim and generates likely scenarios (use-cases and contexts)
399 * Each scenario includes: assumptions, definitions, boundaries, evidence context
400 * Users can flag incorrect scenarios
401 * System learns from corrections
402
403 **Key Concept**: Scenarios represent different interpretations or contexts (e.g., "Clinical trials with healthy adults" vs. "Real-world data with diverse populations")
404
405 ==== FR5 — Evidence Linking ====
406
407 **Fulfills**: UN-5 (Source tracing), UN-7 (Evidence transparency)
408
409 * Automated evidence discovery from sources
410 * Relevance scoring
411 * Contradiction detection
412 * Source quality assessment
413
414 ==== FR6 — Scenario Comparison ====
415
416 **Fulfills**: UN-3 (Summary with verdicts), UN-8 (Understanding disagreement)
417
418 * Side-by-side comparison interface
419 * Highlight key differences between scenarios
420 * Show evidence supporting each scenario
421 * Display confidence scores per scenario
422
423 === 8.3 Verdicts & Analysis ===
424
425 ==== FR7 — Automated Verdicts ====
426
427 **Fulfills**: UN-1 (Trust score), UN-2 (Verification verdicts), UN-3 (Verdict summaries), UN-13 (Cite verdicts)
428
429 * AKEL generates verdict based on evidence within each scenario
430 * **Likelihood range** displayed (e.g., "0.70-0.85 (likely true)") - NOT binary true/false
431 * **Uncertainty factors** explicitly listed (e.g., "Small sample sizes", "Long-term effects unknown")
432 * Confidence score displayed prominently
433 * Source quality indicators shown
434 * Contradictions noted
435 * Uncertainty acknowledged
436
437 **Key Innovation**: Detailed probabilistic verdicts with explicit uncertainty, not binary judgments
438
439 ==== FR8 — Time Evolution ====
440
441 **Fulfills**: UN-15 (Verdict evolution timeline)
442
443 * Claims and verdicts update as new evidence emerges
444 * Version history maintained for all verdicts
445 * Changes highlighted
446 * Confidence score trends visible
447 * Users can see "as of date X, what did we know?"
448
449 === 8.4 Workflow & Moderation ===
450
451 ==== FR9 — Publication Workflow ====
452
453 **Fulfills**: UN-1 (Fast access to verified content), UN-16 (Clear review status)
454
455 **Simple flow**:
456 1. Claim submitted
457 2. AKEL processes (automated)
458 3. If confidence > threshold: Publish (labeled as AI-generated)
459 4. If confidence < threshold: Flag for improvement
460 5. If risk score > threshold: Flag for moderator
461
462 **No multi-stage approval process**
463
464 ==== FR10 — Moderation ====
465
466 **Focus on abuse, not routine quality**:
467 * Automated abuse detection
468 * Moderators handle flags
469 * Quick response to harmful content
470 * Minimal involvement in routine content
471
472 ==== FR11 — Audit Trail ====
473
474 **Fulfills**: UN-14 (API access to histories), UN-15 (Evolution tracking)
475
476 * All edits logged
477 * Version history public
478 * Moderation decisions documented
479 * System improvements tracked
480
481 == 9. Non-Functional Requirements ==
482
483 === 9.1 NFR1 — Performance ===
484
485 **Fulfills**: UN-4 (Fast fact-checking), UN-11 (Responsive filtering)
486
487 * Claim processing: < 30 seconds
488 * Search response: < 2 seconds
489 * Page load: < 3 seconds
490 * 99% uptime
491
492 === 9.2 NFR2 — Scalability ===
493
494 **Fulfills**: UN-14 (API access at scale)
495
496 * Handle 10,000 claims initially
497 * Scale to 1M+ claims
498 * Support 100K+ concurrent users
499 * Automated processing scales linearly
500
501 === 9.3 NFR3 — Transparency ===
502
503 **Fulfills**: UN-7 (Evidence transparency), UN-9 (Methodology transparency), UN-13 (Citable verdicts), UN-15 (Evolution visibility)
504
505 * All algorithms open source
506 * All data exportable
507 * All decisions documented
508 * Quality metrics public
509
510 === 9.4 NFR4 — Security & Privacy ===
511
512 * Follow [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]
513 * Secure authentication
514 * Data encryption
515 * Regular security audits
516
517 === 9.5 NFR5 — Maintainability ===
518
519 * Modular architecture
520 * Automated testing
521 * Continuous integration
522 * Comprehensive documentation
523
524 == 10. MVP Scope ==
525
526 **Phase 1 (Months 1-3): Read-Only MVP**
527
528 Build:
529 * Automated claim analysis
530 * Confidence scoring
531 * Source evaluation
532 * Browse/search interface
533 * User flagging system
534
535 **Goal**: Prove AI quality before adding user editing
536
537 **User Needs fulfilled in Phase 1**: UN-1, UN-2, UN-3, UN-4, UN-5, UN-6, UN-7, UN-8, UN-9, UN-12
538
539 **Phase 2 (Months 4-6): User Contributions**
540
541 Add only if needed:
542 * Simple editing (Wikipedia-style)
543 * Reputation system
544 * Basic moderation
545
546 **Additional User Needs fulfilled**: UN-13
547
548 **Phase 3 (Months 7-12): Refinement**
549
550 * Continuous quality improvement
551 * Feature additions based on real usage
552 * Scale infrastructure
553
554 **Additional User Needs fulfilled**: UN-14 (API access), UN-15 (Full evolution tracking)
555
556 **Deferred**:
557 * Federation (until multiple successful instances exist)
558 * Complex contribution workflows (focus on automation)
559 * Extensive role hierarchy (keep simple)
560
561 == 11. Success Metrics ==
562
563 **System Quality** (track weekly):
564 * Error rate by category (target: -10%/month)
565 * Average confidence score (target: increase)
566 * Source quality distribution (target: more high-quality)
567 * Contradiction detection rate (target: increase)
568
569 **Efficiency** (track monthly):
570 * Claims processed per hour (target: increase)
571 * Human hours per claim (target: decrease)
572 * Automation coverage (target: >90%)
573 * Re-work rate (target: <5%)
574
575 **User Satisfaction** (track quarterly):
576 * User flag rate (issues found)
577 * Correction acceptance rate (flags valid)
578 * Return user rate
579 * Trust indicators (surveys)
580
581 **User Needs Metrics** (track quarterly):
582 * UN-1: % users who understand trust scores
583 * UN-4: Time to verify social media claim (target: <30s)
584 * UN-7: % users who access evidence details
585 * UN-8: % users who view multiple scenarios
586 * UN-15: % users who check evolution timeline
587
588 == 12. Requirements Traceability ==
589
590 For full traceability matrix showing which requirements fulfill which user needs, see:
591
592 * [[User Needs>>FactHarbor.Specification.Requirements.User-Needs]] - Section 8 includes comprehensive mapping tables
593
594 == 13. Related Pages ==
595
596 * **[[User Needs>>FactHarbor.Specification.Requirements.User-Needs]]** - What users need (drives these requirements)
597 * [[Architecture>>FactHarbor.Specification.Architecture.WebHome]] - How requirements are implemented
598 * [[Data Model>>FactHarbor.Specification.Data Model.WebHome]] - Data structures supporting requirements
599 * [[Workflows>>FactHarbor.Specification.Workflows.WebHome]] - User interaction workflows
600 * [[AKEL>>FactHarbor.Specification.AI Knowledge Extraction Layer (AKEL).WebHome]] - AI system fulfilling automation requirements
601 * [[Global Rules>>FactHarbor.Organisation.How-We-Work-Together.GlobalRules.WebHome]]
602 * [[Privacy Policy>>FactHarbor.Organisation.How-We-Work-Together.Privacy-Policy]]