Wiki source code of Claim Analysis Workflow (v2.6.33)
Last modified by Robert Schaub on 2026/02/08 08:31
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | {{info}} | ||
| 2 | **Current Implementation (v2.6.33)** - Scenarios have been replaced by **KeyFactors** - optional decomposition questions discovered during the understanding phase. | ||
| 3 | {{/info}} | ||
| 4 | |||
| 5 | = Claim Analysis Workflow = | ||
| 6 | |||
| 7 | {{mermaid}} | ||
| 8 | |||
| 9 | graph TB | ||
| 10 | Start[User Submission] | ||
| 11 | |||
| 12 | subgraph Step1[Step 1 Understand] | ||
| 13 | Extract{understandClaim LLM Analysis} | ||
| 14 | Gate1{Gate 1 Claim Validation} | ||
| 15 | DetectType[Detect Input Type] | ||
| 16 | DetectScopes[Detect Scopes] | ||
| 17 | KeyFactors[Discover KeyFactors] | ||
| 18 | end | ||
| 19 | |||
| 20 | subgraph Step2[Step 2 Research] | ||
| 21 | Decide[decideNextResearch] | ||
| 22 | Search[Web Search] | ||
| 23 | Fetch[Fetch Sources] | ||
| 24 | Facts[extractFacts] | ||
| 25 | end | ||
| 26 | |||
| 27 | subgraph Step3[Step 3 Verdict] | ||
| 28 | Verdict[generateVerdicts] | ||
| 29 | Gate4{Gate 4 Confidence Check} | ||
| 30 | end | ||
| 31 | |||
| 32 | subgraph Output[Output] | ||
| 33 | Publish[Publish Result] | ||
| 34 | LowConf[Low Confidence Flag] | ||
| 35 | end | ||
| 36 | |||
| 37 | Start --> Extract | ||
| 38 | Extract --> Gate1 | ||
| 39 | Gate1 -->|Pass Factual| DetectType | ||
| 40 | Gate1 -->|Fail Opinion| Exclude[Exclude from analysis] | ||
| 41 | DetectType --> DetectScopes | ||
| 42 | DetectScopes --> KeyFactors | ||
| 43 | KeyFactors --> Decide | ||
| 44 | Decide --> Search | ||
| 45 | Search --> Fetch | ||
| 46 | Fetch --> Facts | ||
| 47 | Facts -->|More research needed| Decide | ||
| 48 | Facts -->|Sufficient evidence| Verdict | ||
| 49 | Verdict --> Gate4 | ||
| 50 | Gate4 -->|High or Medium confidence| Publish | ||
| 51 | Gate4 -->|Low or Insufficient| LowConf | ||
| 52 | |||
| 53 | {{/mermaid}} | ||
| 54 | |||
| 55 | == Quality Gates (Implemented) == | ||
| 56 | |||
| 57 | |= Gate |= Name |= Purpose |= Pass Criteria | ||
| 58 | | **Gate 1** | Claim Validation | Filter non-factual claims | Factual, opinion score 0.3 or less, specificity 0.3 or more | ||
| 59 | | **Gate 4** | Verdict Confidence | Ensure sufficient evidence | 2 or more sources, avg quality 0.6 or more, agreement 60% or more | ||
| 60 | |||
| 61 | //Gates 2 (Contradiction Search) and 3 (Uncertainty Quantification) are not yet implemented.// | ||
| 62 | |||
| 63 | == KeyFactors (Replaces Scenarios) == | ||
| 64 | |||
| 65 | **KeyFactors** are optional decomposition questions discovered during the understanding phase: | ||
| 66 | * Not stored as separate entities | ||
| 67 | * Help break down complex claims into checkable sub-questions | ||
| 68 | * See ##Docs/ARCHITECTURE/KeyFactors_Design.md## for design rationale | ||
| 69 | |||
| 70 | == 7-Point Verdict Scale == | ||
| 71 | |||
| 72 | * **TRUE** (86-100%) - Claim is well-supported by evidence | ||
| 73 | * **MOSTLY-TRUE** (72-85%) - Largely accurate with minor caveats | ||
| 74 | * **LEANING-TRUE** (58-71%) - More evidence supports than contradicts | ||
| 75 | * **MIXED** (43-57%, high confidence) - Roughly equal evidence both ways | ||
| 76 | * **UNVERIFIED** (43-57%, low confidence) - Insufficient evidence to determine | ||
| 77 | * **LEANING-FALSE** (29-42%) - More evidence contradicts than supports | ||
| 78 | * **MOSTLY-FALSE** (15-28%) - Largely inaccurate | ||
| 79 | * **FALSE** (0-14%) - Claim is refuted by evidence |