POC1 API & Schemas Specification

1

= POC1 API & Schemas Specification =

2

3

----

4

5

== Version History ==

6

7

|=Version|=Date|=Changes

8

|0.4.1|2025-12-24|Applied 9 critical fixes: file format notice, verdict taxonomy, canonicalization algorithm, Stage 1 cost policy, BullMQ fix, language in cache key, historical claims TTL, idempotency, copyright policy

9

|0.4|2025-12-24|**BREAKING:** 3-stage pipeline with claim-level caching, user tier system, cache-only mode for free users, Redis cache architecture

10

|0.3.1|2025-12-24|Fixed single-prompt strategy, SSE clarification, schema canonicalization, cost constraints

11

|0.3|2025-12-24|Added complete API endpoints, LLM config, risk tiers, scraping details

----

== 1. Core Objective (POC1) ==

16

17

The primary technical goal of POC1 is to validate **Approach 1 (Single-Pass Holistic Analysis)** while implementing **claim-level caching** to achieve cost sustainability.

18

19

The system must prove that AI can identify an article's **Main Thesis** and determine if supporting claims logically support that thesis without committing fallacies.

20

21

=== Success Criteria: ===

22

23

* Test with 30 diverse articles

24

* Target: ≥70% accuracy detecting misleading articles

25

* Cost: <$0.25 per NEW analysis (uncached)

26

* Cost: $0.00 for cached claim reuse

27

* Cache hit rate: ≥50% after 1,000 articles

28

* Processing time: <2 minutes (standard depth)

29

30

=== Economic Model: ===

31

32

* **Free tier:** $10 credit per month (~~40-140 articles depending on cache hits)

33

* **After limit:** Cache-only mode (instant, free access to cached claims)

34

* **Paid tier:** Unlimited new analyses

----

== 2. Architecture Overview ==

39

40

=== 2.1 3-Stage Pipeline with Caching ===

41

42

FactHarbor POC1 uses a **3-stage architecture** designed for claim-level caching and cost efficiency:

graph TD

A[Article Input] --> B[Stage 1: Extract Claims]

47

B --> C{For Each Claim}

48

C --> D[Check Cache]

49

D -->|Cache HIT| E[Return Cached Verdict]

50

D -->|Cache MISS| F[Stage 2: Analyze Claim]

51

F --> G[Store in Cache]

52

G --> E

53

E --> H[Stage 3: Holistic Assessment]

54

H --> I[Final Report]

55

56

57

==== Stage 1: Claim Extraction (Haiku, no cache) ====

58

59

* **Input:** Article text

60

* **Output:** 5 canonical claims (normalized, deduplicated)

61

* **Model:** Claude Haiku 4.5.5 (default, configurable via LLM abstraction layer)

62

* **Cost:** $0.003 per article

63

* **Cache strategy:** No caching (article-specific)

64

65

==== Stage 2: Claim Analysis (Sonnet, CACHED) ====

66

67

* **Input:** Single canonical claim

68

* **Output:** Scenarios + Evidence + Verdicts

69

* **Model:** Claude Sonnet 4.5 (default, configurable via LLM abstraction layer)

70

* **Cost:** $0.081 per NEW claim

71

* **Cache strategy:** Redis, 90-day TTL

72

* **Cache key:** claim:v1norm1:{language}:{sha256(canonical_claim)}

73

74

==== Stage 3: Holistic Assessment (Sonnet, no cache) ====

75

76

* **Input:** Article + Claim verdicts (from cache or Stage 2)

77

* **Output:** Article verdict + Fallacies + Logic quality

78

* **Model:** Claude Sonnet 4.5 (default, configurable via LLM abstraction layer)

79

* **Cost:** $0.030 per article

80

* **Cache strategy:** No caching (article-specific)

**Note:** Stage 3 implements **Approach 1 (Single-Pass Holistic Analysis)** from the [[Article Verdict Problem>>Test.FactHarbor.Specification.POC.Article-Verdict-Problem]]. While claim analysis (Stage 2) is cached for efficiency, the holistic assessment maintains the integrated evaluation philosophy of Approach 1.

85

86

=== Total Cost Formula: ===

87

88

{{{Cost = $0.003 (extraction) + (N_new_claims × $0.081) + $0.030 (holistic)

89

90

Examples:

91

- 0 new claims (100% cache hit): $0.033

92

- 1 new claim (80% cache hit): $0.114

93

- 3 new claims (40% cache hit): $0.276

94

- 5 new claims (0% cache hit): $0.438

}}}

----

=== 2.2 User Tier System ===

**Free Tier Economics:**

107

108

* $10 credit = 40-140 articles analyzed (depending on cache hit rate)

109

* Average 70 articles/month at 70% cache hit rate

110

* After limit: Cache-only mode

----

=== 2.3 Cache-Only Mode (Free Tier Feature) ===

115

116

When free users reach their $10 monthly limit, they enter **Cache-Only Mode**:

==== Stage 3: Holistic Assessment - Complete Specification ====

121

122

===== 3.3.1 Overview =====

123

124

**Purpose:** Synthesize individual claim analyses into an overall article assessment, identifying logical fallacies, reasoning quality, and publication readiness.

125

126

**Approach:** **Single-Pass Holistic Analysis** (Approach 1 from Comparison Matrix)

127

128

**Why This Approach for POC1:**

129

* ✅ **1 API call** (vs 2 for Two-Pass or Judge)

130

* ✅ **Low cost** ($0.030 per article)

131

* ✅ **Fast** (4-6 seconds)

132

* ✅ **Low complexity** (simple implementation)

133

* ⚠️ **Medium reliability** (acceptable for POC1, will improve in POC2/Production)

134

135

**Alternative Approaches Considered:**

136

137

138

| **1. Single-Pass** ⭐ | 1 | 💰 Low | ⚡ Fast | 🟢 Low | ⚠️ Medium | **POC1**

139

| 2. Two-Pass | 2 | 💰💰 Med | 🐢 Slow | 🟡 Med | ✅ High | POC2/Prod

140

| 3. Structured | 1 | 💰 Low | ⚡ Fast | 🟡 Med | ✅ High | POC1 (alternative)

141

| 4. Weighted | 1 | 💰 Low | ⚡ Fast | 🟢 Low | ⚠️ Medium | POC1 (alternative)

142

| 5. Heuristics | 1 | 💰 Lowest | ⚡⚡ Fastest | 🟡 Med | ⚠️ Medium | Any

143

| 6. Hybrid | 1 | 💰 Low | ⚡ Fast | 🔴 Med-High | ✅ High | POC2

144

| 7. Judge | 2 | 💰💰 Med | 🐢 Slow | 🟡 Med | ✅ High | Production

145

146

**POC1 Choice:** Approach 1 (Single-Pass) for speed and simplicity. Will upgrade to Approach 2 (Two-Pass) or 6 (Hybrid) in POC2 for higher reliability.

147

148

===== 3.3.2 What Stage 3 Evaluates =====

149

150

Stage 3 performs **integrated holistic analysis** considering:

151

152

**1. Claim-Level Aggregation:**

153

* Verdict distribution (how many TRUE vs FALSE vs DISPUTED)

154

* Average confidence across all claims

155

* Claim interdependencies (do claims support/contradict each other?)

156

* Critical claim identification (which claims are most important?)

157

158

**2. Contextual Factors:**

159

* **Source credibility**: Is the article from a reputable publisher?

160

* **Author expertise**: Does the author have relevant credentials?

161

* **Publication date**: Is information current or outdated?

162

* **Claim coherence**: Do claims form a logical narrative?

163

* **Missing context**: Are important caveats or qualifications missing?

164

165

**3. Logical Fallacies:**

166

* **Cherry-picking**: Selective evidence presentation

167

* **False equivalence**: Treating unequal things as equal

168

* **Straw man**: Misrepresenting opposing arguments

169

* **Ad hominem**: Attacking person instead of argument

170

* **Slippery slope**: Assuming extreme consequences without justification

171

* **Circular reasoning**: Conclusion assumes premise

172

* **False dichotomy**: Presenting only two options when more exist

173

174

**4. Reasoning Quality:**

175

* **Evidence strength**: Quality and quantity of supporting evidence

176

* **Logical coherence**: Arguments follow logically

177

* **Transparency**: Assumptions and limitations acknowledged

178

* **Nuance**: Complexity and uncertainty appropriately addressed

179

180

**5. Publication Readiness:**

181

* **Risk tier assignment**: A (high risk), B (medium), or C (low risk)

182

* **Publication mode**: DRAFT_ONLY, AI_GENERATED, or HUMAN_REVIEWED

183

* **Required disclaimers**: What warnings should accompany this content?

184

185

===== 3.3.3 Implementation: Single-Pass Approach =====

186

187

**Input:**

188

* Original article text (full content)

189

* Stage 2 claim analyses (array of ClaimAnalysis objects)

190

* Article metadata (URL, title, author, date, source)

**Processing:**

# Pseudo-code for Stage 3 (Single-Pass)

196

197

def stage3_holistic_assessment(article, claim_analyses, metadata):

198

"""

199

Single-pass holistic assessment using Claude Sonnet 4.5.

200

201

Approach 1: One comprehensive prompt that asks the LLM to:

202

1. Review all claim verdicts

203

2. Identify patterns and dependencies

204

3. Detect logical fallacies

205

4. Assess reasoning quality

206

5. Determine credibility score and risk tier

207

6. Generate publication recommendations

208

"""

209

210

# Construct comprehensive prompt

211

prompt = f"""

212

You are analyzing an article for factual accuracy and logical reasoning.

213

214

ARTICLE METADATA:

215

- Title: {metadata['title']}

216

- Source: {metadata['source']}

217

- Date: {metadata['date']}

218

- Author: {metadata['author']}

ARTICLE TEXT:

{article}

INDIVIDUAL CLAIM ANALYSES:

224

{format_claim_analyses(claim_analyses)}

225

226

YOUR TASK:

227

Perform a holistic assessment considering:

228

229

1. CLAIM AGGREGATION:

230

- Review the verdict for each claim

231

- Identify any interdependencies between claims

232

- Determine which claims are most critical to the article's thesis

233

234

2. CONTEXTUAL EVALUATION:

235

- Assess source credibility

236

- Evaluate author expertise

237

- Consider publication timeliness

238

- Identify missing context or important caveats

239

240

3. LOGICAL FALLACIES:

241

- Identify any logical fallacies present

242

- For each fallacy, provide:

243

* Type of fallacy

244

* Where it occurs in the article

245

* Why it's problematic

246

* Severity (minor/moderate/severe)

247

248

4. REASONING QUALITY:

249

- Evaluate evidence strength

250

- Assess logical coherence

251

- Check for transparency in assumptions

252

- Evaluate handling of nuance and uncertainty

253

254

5. CREDIBILITY SCORING:

255

- Calculate overall credibility score (0.0-1.0)

256

- Assign risk tier:

257

* A (high risk): ≤0.5 credibility OR severe fallacies

258

* B (medium risk): 0.5-0.8 credibility OR moderate issues

259

* C (low risk): >0.8 credibility AND no significant issues

260

261

6. PUBLICATION RECOMMENDATIONS:

262

- Determine publication mode:

263

* DRAFT_ONLY: Tier A, multiple severe issues

264

* AI_GENERATED: Tier B/C, acceptable quality with disclaimers

265

* HUMAN_REVIEWED: Complex or borderline cases

266

- List required disclaimers

267

- Explain decision rationale

268

269

OUTPUT FORMAT:

270

Return a JSON object matching the ArticleAssessment schema.

"""

# Call LLM

response = llm_client.complete(

275

model="claude-sonnet-4-5-20250929",

276

prompt=prompt,

277

max_tokens=4000,

278

response_format="json"

279

)

280

281

# Parse and validate response

282

assessment = parse_json(response.content)

283

validate_article_assessment_schema(assessment)

return assessment

**Prompt Engineering Notes:**

289

290

1. **Structured Instructions**: Break down task into 6 clear sections

291

2. **Context-Rich**: Provide article + all claim analyses + metadata

292

3. **Explicit Criteria**: Define credibility scoring and risk tiers precisely

293

4. **JSON Schema**: Request structured output matching ArticleAssessment schema

294

5. **Examples** (in production): Include 2-3 example assessments for consistency

295

296

===== 3.3.4 Credibility Scoring Algorithm =====

297

298

**Base Score Calculation:**

299

300

301

def calculate_credibility_score(claim_analyses, fallacies, contextual_factors):

302

"""

303

Calculate overall credibility score (0.0-1.0).

304

305

This is a GUIDELINE for the LLM, not strict code.

306

The LLM has flexibility to adjust based on context.

307

"""

308

309

# 1. Claim Verdict Score (60% weight)

310

verdict_weights = {

311

"TRUE": 1.0,

312

"PARTIALLY_TRUE": 0.7,

"DISPUTED": 0.5,

"UNSUPPORTED": 0.3,

"FALSE": 0.0,

"UNVERIFIABLE": 0.4

}

claim_scores = [

verdict_weights[c.verdict.label] * c.verdict.confidence

321

for c in claim_analyses

322

]

323

avg_claim_score = sum(claim_scores) / len(claim_scores)

324

claim_component = avg_claim_score * 0.6

325

326

# 2. Fallacy Penalty (20% weight)

327

fallacy_penalties = {

"minor": -0.05,

"moderate": -0.15,

"severe": -0.30

}

fallacy_score = 1.0

for fallacy in fallacies:

335

fallacy_score += fallacy_penalties[fallacy.severity]

336

337

fallacy_score = max(0.0, min(1.0, fallacy_score))

338

fallacy_component = fallacy_score * 0.2

339

340

# 3. Contextual Factors (20% weight)

341

context_adjustments = {

342

"source_credibility": {"positive": +0.1, "neutral": 0, "negative": -0.1},

343

"author_expertise": {"positive": +0.1, "neutral": 0, "negative": -0.1},

344

"timeliness": {"positive": +0.05, "neutral": 0, "negative": -0.05},

345

"transparency": {"positive": +0.05, "neutral": 0, "negative": -0.05}

}

context_score = 1.0

for factor in contextual_factors:

350

adjustment = context_adjustments.get(factor.factor, {}).get(factor.impact, 0)

351

context_score += adjustment

352

353

context_score = max(0.0, min(1.0, context_score))

354

context_component = context_score * 0.2

355

356

# 4. Combine components

357

final_score = claim_component + fallacy_component + context_component

358

359

# 5. Apply confidence modifier

360

avg_confidence = sum(c.verdict.confidence for c in claim_analyses) / len(claim_analyses)

361

final_score = final_score * (0.8 + 0.2 * avg_confidence)

362

363

return max(0.0, min(1.0, final_score))

364

365

366

**Note:** This algorithm is a **guideline** provided to the LLM in the system prompt. The LLM has flexibility to adjust based on specific article context, but should generally follow this structure for consistency.

367

368

===== 3.3.5 Risk Tier Assignment =====

369

370

**Automatic Risk Tier Rules:**

371

372

373

Risk Tier A (High Risk - Requires Review):

374

- Credibility score ≤ 0.5, OR

375

- Any severe fallacies detected, OR

376

- Multiple (3+) moderate fallacies, OR

377

- 50%+ of claims are FALSE or UNSUPPORTED

378

379

Risk Tier B (Medium Risk - May Publish with Disclaimers):

380

- Credibility score 0.5-0.8, OR

381

- 1-2 moderate fallacies, OR

382

- 20-49% of claims are DISPUTED or PARTIALLY_TRUE

383

384

Risk Tier C (Low Risk - Safe to Publish):

385

- Credibility score > 0.8, AND

386

- No severe or moderate fallacies, AND

387

- <20% disputed/problematic claims, AND

388

- No critical missing context

389

390

391

===== 3.3.6 Output: ArticleAssessment Schema =====

392

393

(See Stage 3 Output Schema section above for complete JSON schema)

394

395

===== 3.3.7 Performance Metrics =====

396

397

**POC1 Targets:**

398

* **Processing time**: 4-6 seconds per article

399

* **Cost**: $0.030 per article (Sonnet 4.5 tokens)

400

* **Quality**: 70-80% agreement with human reviewers (acceptable for POC)

401

* **API calls**: 1 per article

402

403

**Future Improvements (POC2/Production):**

404

* Upgrade to Two-Pass (Approach 2): +15% accuracy, +$0.020 cost

405

* Add human review sampling: 10% of Tier B articles

406

* Implement Judge approach (Approach 7) for Tier A: Highest quality

407

408

===== 3.3.8 Example Stage 3 Execution =====

409

410

**Input:**

411

* Article: "Biden won the 2020 election"

412

* Claim analyses: [{claim: "Biden won", verdict: "TRUE", confidence: 0.95}]

413

414

**Stage 3 Processing:**

415

1. Analyzes single claim with high confidence

416

2. Checks for contextual factors (source credibility)

417

3. Searches for logical fallacies (none found)

418

4. Calculates credibility: 0.6 * 0.95 + 0.2 * 1.0 + 0.2 * 1.0 = 0.97

419

5. Assigns risk tier: C (low risk)

420

6. Recommends: AI_GENERATED publication mode

**Output:**

```json

{

"article_id": "a1",

"overall_assessment": {

427

"credibility_score": 0.97,

428

"risk_tier": "C",

429

"summary": "Article makes single verifiable claim with strong evidence support",

430

"confidence": 0.95

431

},

432

"claim_aggregation": {

433

"total_claims": 1,

434

"verdict_distribution": {"TRUE": 1},

435

"avg_confidence": 0.95

436

},

437

"contextual_factors": [

438

{"factor": "source_credibility", "impact": "positive", "description": "Reputable news source"}

439

],

440

"recommendations": {

441

"publication_mode": "AI_GENERATED",

442

"requires_review": false,

443

"suggested_disclaimers": []

}

}

```

==== What Cache-Only Mode Provides: ====

449

450

✅ **Claim Extraction (Platform-Funded):**

451

452

* Stage 1 extraction runs at $0.003 per article

453

* **Cost: Absorbed by platform** (not charged to user credit)

454

* Rationale: Extraction is necessary to check cache, and cost is negligible

455

* Rate limit: Max 50 extractions/day in cache-only mode (prevents abuse)

456

457

✅ **Instant Access to Cached Claims:**

458

459

* Any claim that exists in cache → Full verdict returned

460

* Cost: $0 (no LLM calls)

461

* Response time: <100ms

462

463

✅ **Partial Article Analysis:**

464

465

* Check each claim against cache

466

* Return verdicts for ALL cached claims

467

* For uncached claims: Return "status": "cache_miss"

468

469

✅ **Cache Coverage Report:**

470

471

* "3 of 5 claims available in cache (60% coverage)"

472

* Links to cached analyses

473

* Estimated cost to complete: $0.162 (2 new claims)

474

475

❌ **Not Available in Cache-Only Mode:**

476

477

* New claim analysis (Stage 2 LLM calls blocked)

478

* Full holistic assessment (Stage 3 blocked if any claims missing)

479

480

==== User Experience Example: ====

481

482

{{{{

483

"status": "cache_only_mode",

484

"message": "Monthly credit limit reached. Showing cached results only.",

"cache_coverage": {

"claims_total": 5,

"claims_cached": 3,

"claims_missing": 2,

"coverage_percent": 60

490

},

491

"cached_claims": [

492

{"claim_id": "C1", "verdict": "Likely", "confidence": 0.82},

493

{"claim_id": "C2", "verdict": "Highly Likely", "confidence": 0.91},

494

{"claim_id": "C4", "verdict": "Unclear", "confidence": 0.55}

495

],

496

"missing_claims": [

497

{"claim_id": "C3", "claim_text": "...", "estimated_cost": "$0.081"},

498

{"claim_id": "C5", "claim_text": "...", "estimated_cost": "$0.081"}

499

],

500

"upgrade_options": {

501

"top_up": "$5 for 20-70 more articles",

502

"pro_tier": "$50/month unlimited"

}

}

}}}

**Design Rationale:**

508

509

* Free users still get value (cached claims often answer their question)

510

* Demonstrates FactHarbor's value (partial results encourage upgrade)

511

* Sustainable for platform (no additional cost)

512

* Fair to all users (everyone contributes to cache)

----

== 6. LLM Abstraction Layer ==

519

520

=== 6.1 Design Principle ===

521

522

**FactHarbor uses provider-agnostic LLM abstraction** to avoid vendor lock-in and enable:

523

524

* **Provider switching:** Change LLM providers without code changes

525

* **Cost optimization:** Use different providers for different stages

526

* **Resilience:** Automatic fallback if primary provider fails

527

* **Cross-checking:** Compare outputs from multiple providers

528

* **A/B testing:** Test new models without deployment changes

529

530

**Implementation:** All LLM calls go through an abstraction layer that routes to configured providers.

----

=== 6.2 LLM Provider Interface ===

535

536

**Abstract Interface:**

537

538

{{{

539

interface LLMProvider {

540

// Core methods

541

complete(prompt: string, options: CompletionOptions): Promise<CompletionResponse>

542

stream(prompt: string, options: CompletionOptions): AsyncIterator<StreamChunk>

// Provider metadata

getName(): string

getMaxTokens(): number

547

getCostPer1kTokens(): { input: number, output: number }

548

549

// Health check

550

isAvailable(): Promise<boolean>

551

}

552

553

interface CompletionOptions {

model?: string

maxTokens?: number

temperature?: number

stopSequences?: string[]

558

systemPrompt?: string

}

}}}

----

=== 6.3 Supported Providers (POC1) ===

565

566

**Primary Provider (Default):**

567

568

* **Anthropic Claude API**

569

* Models: Claude Haiku 4.5, Claude Sonnet 4.5, Claude Opus 4

570

* Used by default in POC1

571

* Best quality for holistic analysis

572

573

**Secondary Providers (Future):**

574

575

* **OpenAI API**

576

* Models: GPT-4o, GPT-4o-mini

577

* For cost comparison

578

579

* **Google Vertex AI**

580

* Models: Gemini 1.5 Pro, Gemini 1.5 Flash

581

* For diversity in evidence gathering

582

583

* **Local Models** (Post-POC)

584

* Models: Llama 3.1, Mistral

585

* For privacy-sensitive deployments

----

=== 6.4 Provider Configuration ===

590

591

**Environment Variables:**

{{{

# Primary provider

LLM_PRIMARY_PROVIDER=anthropic

596

ANTHROPIC_API_KEY=sk-ant-...

597

598

# Fallback provider

599

LLM_FALLBACK_PROVIDER=openai

600

OPENAI_API_KEY=sk-...

601

602

# Provider selection per stage

603

LLM_STAGE1_PROVIDER=anthropic

604

LLM_STAGE1_MODEL=claude-haiku-4

605

LLM_STAGE2_PROVIDER=anthropic

606

LLM_STAGE2_MODEL=claude-sonnet-4-5-20250929

607

LLM_STAGE3_PROVIDER=anthropic

608

LLM_STAGE3_MODEL=claude-sonnet-4-5-20250929

609

610

# Cost limits

611

LLM_MAX_COST_PER_REQUEST=1.00

612

}}}

613

614

**Database Configuration (Alternative):**

{{{{

{

"providers": [

{

"name": "anthropic",

"api_key_ref": "vault://anthropic-api-key",

"enabled": true,

"priority": 1

},

{

"name": "openai",

"api_key_ref": "vault://openai-api-key",

"enabled": true,

"priority": 2

}

],

"stage_config": {

"stage1": {

"provider": "anthropic",

635

"model": "claude-haiku-4-5-20251001",

"max_tokens": 4096,

"temperature": 0.0

},

"stage2": {

"provider": "anthropic",

641

"model": "claude-sonnet-4-5-20250929",

"max_tokens": 16384,

"temperature": 0.3

},

"stage3": {

"provider": "anthropic",

647

"model": "claude-sonnet-4-5-20250929",

"max_tokens": 8192,

"temperature": 0.2

}

}

}

}}}

----

=== 6.5 Stage-Specific Models (POC1 Defaults) ===

658

659

**Stage 1: Claim Extraction**

660

661

* **Default:** Anthropic Claude Haiku 4.5

662

* **Alternative:** OpenAI GPT-4o-mini, Google Gemini 1.5 Flash

663

* **Rationale:** Fast, cheap, simple task

664

* **Cost:** ~$0.003 per article

665

666

**Stage 2: Claim Analysis** (CACHEABLE)

667

668

* **Default:** Anthropic Claude Sonnet 4.5

669

* **Alternative:** OpenAI GPT-4o, Google Gemini 1.5 Pro

670

* **Rationale:** High-quality analysis, cached 90 days

671

* **Cost:** ~$0.081 per NEW claim

672

673

**Stage 3: Holistic Assessment**

674

675

* **Default:** Anthropic Claude Sonnet 4.5

676

* **Alternative:** OpenAI GPT-4o, Claude Opus 4 (for high-stakes)

677

* **Rationale:** Complex reasoning, logical fallacy detection

678

* **Cost:** ~$0.030 per article

679

680

**Cost Comparison (Example):**

681

682

|=Stage|=Anthropic (Default)|=OpenAI Alternative|=Google Alternative

683

|Stage 1|Claude Haiku 4.5.5 ($0.003)|GPT-4o-mini ($0.002)|Gemini Flash ($0.002)

684

|Stage 2|Claude Sonnet 4.5 ($0.081)|GPT-4o ($0.045)|Gemini Pro ($0.050)

685

|Stage 3|Claude Sonnet 4.5 ($0.030)|GPT-4o ($0.018)|Gemini Pro ($0.020)

686

|**Total (0% cache)**|**$0.114**|**$0.065**|**$0.072**

687

688

**Note:** POC1 uses Anthropic exclusively for consistency. Multi-provider support planned for POC2.

----

=== 6.6 Failover Strategy ===

693

694

**Automatic Failover:**

695

696

{{{

697

async function completeLLM(stage: string, prompt: string): Promise<string> {

698

const primaryProvider = getProviderForStage(stage)

699

const fallbackProvider = getFallbackProvider()

700

701

try {

702

return await primaryProvider.complete(prompt)

703

} catch (error) {

704

if (error.type === 'rate_limit' || error.type === 'service_unavailable') {

705

logger.warn(`Primary provider failed, using fallback`)

706

return await fallbackProvider.complete(prompt)

}

throw error

}

}

}}}

**Fallback Priority:**

714

715

1. **Primary:** Configured provider for stage

716

2. **Secondary:** Fallback provider (if configured)

717

3. **Cache:** Return cached result (if available for Stage 2)

718

4. **Error:** Return 503 Service Unavailable

----

=== 6.7 Provider Selection API ===

723

724

**Admin Endpoint:** POST /admin/v1/llm/configure

725

726

**Update provider for specific stage:**

{{{{

{

"stage": "stage2",

"provider": "openai",

"model": "gpt-4o",

"max_tokens": 16384,

"temperature": 0.3

}

}}}

**Response:** 200 OK

{{{{

{

"message": "LLM configuration updated",

743

"stage": "stage2",

744

"previous": {

745

"provider": "anthropic",

746

"model": "claude-sonnet-4-5-20250929"

747

},

748

"current": {

749

"provider": "openai",

"model": "gpt-4o"

},

"cost_impact": {

"previous_cost_per_claim": 0.081,

754

"new_cost_per_claim": 0.045,

755

"savings_percent": 44

}

}

}}}

**Get current configuration:**

761

762

GET /admin/v1/llm/config

{{{{

{

"providers": ["anthropic", "openai"],

767

"primary": "anthropic",

768

"fallback": "openai",

769

"stages": {

770

"stage1": {

771

"provider": "anthropic",

772

"model": "claude-haiku-4-5-20251001",

773

"cost_per_request": 0.003

774

},

775

"stage2": {

776

"provider": "anthropic",

777

"model": "claude-sonnet-4-5-20250929",

778

"cost_per_new_claim": 0.081

779

},

780

"stage3": {

781

"provider": "anthropic",

782

"model": "claude-sonnet-4-5-20250929",

783

"cost_per_request": 0.030

}

}

}

}}}

----

=== 6.8 Implementation Notes ===

792

793

**Provider Adapter Pattern:**

794

795

{{{

796

class AnthropicProvider implements LLMProvider {

797

async complete(prompt: string, options: CompletionOptions) {

798

const response = await anthropic.messages.create({

799

model: options.model || 'claude-sonnet-4-5-20250929',

800

max_tokens: options.maxTokens || 4096,

801

messages: [{ role: 'user', content: prompt }],

802

system: options.systemPrompt

803

})

804

return response.content[0].text

}

}

class OpenAIProvider implements LLMProvider {

809

async complete(prompt: string, options: CompletionOptions) {

810

const response = await openai.chat.completions.create({

811

model: options.model || 'gpt-4o',

812

max_tokens: options.maxTokens || 4096,

813

messages: [

814

{ role: 'system', content: options.systemPrompt },

815

{ role: 'user', content: prompt }

816

]

817

})

818

return response.choices[0].message.content

}

}

}}}

**Provider Registry:**

824

825

{{{

826

const providers = new Map<string, LLMProvider>()

827

providers.set('anthropic', new AnthropicProvider())

828

providers.set('openai', new OpenAIProvider())

829

providers.set('google', new GoogleProvider())

830

831

function getProvider(name: string): LLMProvider {

832

return providers.get(name) || providers.get(config.primaryProvider)

}

}}}

----

== 3. REST API Contract ==

839

840

=== 3.1 User Credit Tracking ===

841

842

**Endpoint:** GET /v1/user/credit

**Response:** 200 OK

{{{{

"user_id": "user_abc123",

848

"tier": "free",

849

"credit_limit": 10.00,

850

"credit_used": 7.42,

851

"credit_remaining": 2.58,

852

"reset_date": "2025-02-01T00:00:00Z",

853

"cache_only_mode": false,

854

"usage_stats": {

855

"articles_analyzed": 67,

856

"claims_from_cache": 189,

857

"claims_newly_analyzed": 113,

858

"cache_hit_rate": 0.626

}

}

}}}

----

==== Stage 2 Output Schema: ClaimAnalysis ====

868

869

**Complete schema for each claim's analysis result:**

{

"claim_id": "claim_abc123",

874

"claim_text": "Biden won the 2020 election",

875

"scenarios": [

876

{

877

"scenario_id": "scenario_1",

878

"description": "Interpreting 'won' as Electoral College victory",

"verdict": {

"label": "TRUE",

"confidence": 0.95,

"explanation": "Joe Biden won 306 electoral votes vs Trump's 232"

},

"evidence": {

"supporting": [

{

"text": "Biden certified with 306 electoral votes",

888

"source_url": "https://www.archives.gov/electoral-college/2020",

889

"source_title": "2020 Electoral College Results",

890

"credibility_score": 0.98

}

],

"opposing": []

}

}

],

"recommended_scenario": "scenario_1",

898

"metadata": {

899

"analysis_timestamp": "2024-12-24T18:00:00Z",

900

"model_used": "claude-sonnet-4-5-20250929",

901

"processing_time_seconds": 8.5

}

}

**Required Fields:**

* **claim_id**: Unique identifier matching Stage 1 output

908

* **claim_text**: The exact claim being analyzed

909

* **scenarios**: Array of interpretation scenarios (minimum 1)

910

* **scenario_id**: Unique ID for this scenario

911

* **description**: Clear interpretation of the claim

912

* **verdict**: Verdict object with label, confidence, explanation

913

* **evidence**: Supporting and opposing evidence arrays

914

* **recommended_scenario**: ID of the primary/recommended scenario

915

* **metadata**: Processing metadata (timestamp, model, timing)

916

917

**Optional Fields:**

918

* Additional context, warnings, or quality scores

919

920

**Minimum Viable Example:**

{

"claim_id": "c1",

"claim_text": "The sky is blue",

926

"scenarios": [{

927

"scenario_id": "s1",

928

"description": "Under clear daytime conditions",

929

"verdict": {"label": "TRUE", "confidence": 0.99, "explanation": "Rayleigh scattering"},

930

"evidence": {"supporting": [], "opposing": []}

931

}],

932

"recommended_scenario": "s1",

933

"metadata": {"analysis_timestamp": "2024-12-24T18:00:00Z"}

}

==== Stage 3 Output Schema: ArticleAssessment ====

940

941

**Complete schema for holistic article-level assessment:**

{

"article_id": "article_xyz789",

946

"overall_assessment": {

947

"credibility_score": 0.72,

948

"risk_tier": "B",

949

"summary": "Article contains mostly accurate claims with one disputed claim requiring expert review",

950

"confidence": 0.85

951

},

952

"claim_aggregation": {

953

"total_claims": 5,

954

"verdict_distribution": {

"TRUE": 3,

"PARTIALLY_TRUE": 1,

"DISPUTED": 1,

"FALSE": 0,

"UNSUPPORTED": 0,

"UNVERIFIABLE": 0

},

"avg_confidence": 0.82

963

},

964

"contextual_factors": [

965

{

966

"factor": "Source credibility",

967

"impact": "positive",

968

"description": "Published by reputable news organization"

969

},

970

{

971

"factor": "Claim interdependence",

972

"impact": "neutral",

973

"description": "Claims are independent; no logical chains"

}

],

"recommendations": {

"publication_mode": "AI_GENERATED",

978

"requires_review": false,

979

"review_reason": null,

980

"suggested_disclaimers": [

981

"One claim (Claim 4) has conflicting expert opinions"

]

},

"metadata": {

"holistic_timestamp": "2024-12-24T18:00:10Z",

986

"model_used": "claude-sonnet-4-5-20250929",

987

"processing_time_seconds": 4.2,

"cache_used": false

}

}

**Required Fields:**

* **article_id**: Unique identifier for this article

995

* **overall_assessment**: Top-level assessment

996

* **credibility_score**: 0.0-1.0 composite score

997

* **risk_tier**: A, B, or C (per AKEL quality gates)

998

* **summary**: Human-readable assessment

999

* **confidence**: How confident the holistic assessment is

1000

* **claim_aggregation**: Statistics across all claims

1001

* **total_claims**: Count of claims analyzed

1002

* **verdict_distribution**: Count per verdict label

1003

* **avg_confidence**: Average confidence across verdicts

1004

* **contextual_factors**: Array of contextual considerations

1005

* **recommendations**: Publication decision support

1006

* **publication_mode**: DRAFT_ONLY, AI_GENERATED, or HUMAN_REVIEWED

1007

* **requires_review**: Boolean flag

1008

* **suggested_disclaimers**: Array of disclaimer texts

1009

* **metadata**: Processing metadata

1010

1011

**Minimum Viable Example:**

{

"article_id": "a1",

"overall_assessment": {

1017

"credibility_score": 0.95,

1018

"risk_tier": "C",

1019

"summary": "All claims verified as true",

1020

"confidence": 0.98

1021

},

1022

"claim_aggregation": {

1023

"total_claims": 1,

1024

"verdict_distribution": {"TRUE": 1},

1025

"avg_confidence": 0.99

1026

},

1027

"contextual_factors": [],

1028

"recommendations": {

1029

"publication_mode": "AI_GENERATED",

1030

"requires_review": false,

1031

"suggested_disclaimers": []

1032

},

1033

"metadata": {"holistic_timestamp": "2024-12-24T18:00:00Z"}

}

=== 3.2 Create Analysis Job (3-Stage) ===

1038

1039

**Endpoint:** POST /v1/analyze

1040

1041

==== Idempotency Support: ====

1042

1043

To prevent duplicate job creation on network retries, clients SHOULD include:

1044

1045

{{{POST /v1/analyze

1046

Idempotency-Key: {client-generated-uuid}

1047

}}}

1048

1049

OR use the client.request_id field:

{{{{

"input_url": "...",

"client": {

"request_id": "client-uuid-12345",

1055

"source_label": "optional"

}

}

}}}

**Server Behavior:**

* If Idempotency-Key or request_id seen before (within 24 hours):

1063

** Return existing job (200 OK, not 202 Accepted)

1064

** Do NOT create duplicate job or charge twice

1065

* Idempotency keys expire after 24 hours (matches job retention)

1066

1067

**Example Response (Idempotent):**

1068

1069

{{{{

1070

"job_id": "01J...ULID",

1071

"status": "RUNNING",

1072

"idempotent": true,

1073

"original_request_at": "2025-12-24T10:31:00Z",

1074

"message": "Returning existing job (idempotency key matched)"

}

}}}

==== Request Body: ====

{{{{

"input_type": "url",

"input_url": "https://example.com/medical-report-01",

"input_text": null,

"options": {

"browsing": "on",

"depth": "standard",

"max_claims": 5,

* **cache_preference** (optional): Cache usage preference

1090

* **Type:** string

1091

* **Enum:** {{code}}["prefer_cache", "allow_partial", "skip_cache"]{{/code}}

1092

* **Default:** {{code}}"prefer_cache"{{/code}}

1093

* **Semantics:**

1094

* {{code}}"prefer_cache"{{/code}}: Use full cache if available, otherwise run all stages

1095

* {{code}}"allow_partial"{{/code}}: Use cached Stage 2 results if available, rerun only Stage 3

1096

* {{code}}"skip_cache"{{/code}}: Always rerun all stages (ignore cache)

1097

* **Behavior:** When set to {{code}}"allow_partial"{{/code}} and Stage 2 cached results exist:

1098

* Stage 1 & 2 are skipped

1099

* Stage 3 (holistic assessment) runs fresh with cached claim analyses

1100

* Response includes {{code}}"cache_used": true{{/code}} and {{code}}"stages_cached": ["stage1", "stage2"]{{/code}}

1101

1102

"scenarios_per_claim": 2,

1103

"max_evidence_per_scenario": 6,

1104

"context_aware_analysis": true

1105

},

1106

"client": {

1107

"request_id": "optional-client-tracking-id",

1108

"source_label": "optional"

}

}

}}}

**Options:**

* browsing: on | off (retrieve web sources or just output queries)

1116

* depth: standard | deep (evidence thoroughness)

1117

* max_claims: 1-10 (default: **5** for cost control)

1118

* scenarios_per_claim: 1-5 (default: **2** for cost control)

1119

* max_evidence_per_scenario: 3-10 (default: **6**)

1120

* context_aware_analysis: true | false (experimental)

1121

1122

**Response:** 202 Accepted

1123

1124

{{{{

1125

"job_id": "01J...ULID",

1126

"status": "QUEUED",

1127

"created_at": "2025-12-24T10:31:00Z",

1128

"estimated_cost": 0.114,

1129

"cost_breakdown": {

1130

"stage1_extraction": 0.003,

1131

"stage2_new_claims": 0.081,

1132

"stage2_cached_claims": 0.000,

1133

"stage3_holistic": 0.030

1134

},

1135

"cache_info": {

1136

"claims_to_extract": 5,

1137

"estimated_cache_hits": 4,

1138

"estimated_new_claims": 1

1139

},

1140

"links": {

1141

"self": "/v1/jobs/01J...ULID",

1142

"result": "/v1/jobs/01J...ULID/result",

1143

"report": "/v1/jobs/01J...ULID/report",

1144

"events": "/v1/jobs/01J...ULID/events"

}

}

}}}

**Error Responses:**

402 Payment Required - Free tier limit reached, cache-only mode

1152

1153

{{{{

1154

"error": "credit_limit_reached",

1155

"message": "Monthly credit limit reached. Entering cache-only mode.",

1156

"cache_only_mode": true,

1157

"credit_remaining": 0.00,

1158

"reset_date": "2025-02-01T00:00:00Z",

1159

"action": "Resubmit with cache_preference=allow_partial for cached results"

}

}}}

----

== 4. Data Schemas ==

1166

1167

=== 4.1 Stage 1 Output: ClaimExtraction ===

1168

1169

{{{{

1170

"job_id": "01J...ULID",

1171

"stage": "stage1_extraction",

1172

"article_metadata": {

1173

"title": "Article title",

1174

"source_url": "https://example.com/article",

1175

"extracted_text_length": 5234,

"language": "en"

},

"claims": [

{

"claim_id": "C1",

"claim_text": "Original claim text from article",

1182

"canonical_claim": "Normalized, deduplicated phrasing",

1183

"claim_hash": "sha256:abc123...",

1184

"is_central_to_thesis": true,

1185

"claim_type": "causal",

1186

"evaluability": "evaluable",

1187

"risk_tier": "B",

1188

"domain": "public_health"

1189

}

1190

],

1191

"article_thesis": "Main argument detected",

"cost": 0.003

}

}}}

----

=== 4.5 Verdict Label Taxonomy ===

1199

1200

FactHarbor uses **three distinct verdict taxonomies** depending on analysis level:

1201

1202

==== 4.5.1 Scenario Verdict Labels (Stage 2) ====

1203

1204

Used for individual scenario verdicts within a claim.

**Enum Values:**

* Highly Likely - Probability 0.85-1.0, high confidence

1209

* Likely - Probability 0.65-0.84, moderate-high confidence

1210

* Unclear - Probability 0.35-0.64, or low confidence

1211

* Unlikely - Probability 0.16-0.34, moderate-high confidence

1212

* Highly Unlikely - Probability 0.0-0.15, high confidence

1213

* Unsubstantiated - Insufficient evidence to determine probability

1214

1215

==== 4.5.2 Claim Verdict Labels (Rollup) ====

1216

1217

Used when summarizing a claim across all scenarios.

**Enum Values:**

* Supported - Majority of scenarios are Likely or Highly Likely

1222

* Refuted - Majority of scenarios are Unlikely or Highly Unlikely

1223

* Inconclusive - Mixed scenarios or majority Unclear/Unsubstantiated

**Mapping Logic:**

* If ≥60% scenarios are (Highly Likely | Likely) → Supported

1228

* If ≥60% scenarios are (Highly Unlikely | Unlikely) → Refuted

1229

* Otherwise → Inconclusive

1230

1231

==== 4.5.3 Article Verdict Labels (Stage 3) ====

1232

1233

Used for holistic article-level assessment.

**Enum Values:**

* WELL-SUPPORTED - Article thesis logically follows from supported claims

1238

* MISLEADING - Claims may be true but article commits logical fallacies

1239

* REFUTED - Central claims are refuted, invalidating thesis

1240

* UNCERTAIN - Insufficient evidence or highly mixed claim verdicts

1241

1242

**Note:** Article verdict considers **claim centrality** (central claims override supporting claims).

1243

1244

==== 4.5.4 API Field Mapping ====

1245

1246

|=Level|=API Field|=Enum Name

1247

|Scenario|scenarios[].verdict.label|scenario_verdict_label

1248

|Claim|claims[].rollup_verdict (optional)|claim_verdict_label

1249

|Article|article_holistic_assessment.overall_verdict|article_verdict_label

----

== 5. Cache Architecture ==

1254

1255

=== 5.1 Redis Cache Design ===

1256

1257

**Technology:** Redis 7.0+ (in-memory key-value store)

1258

1259

**Cache Key Schema:**

1260

1261

{{{claim:v1norm1:{language}:{sha256(canonical_claim)}

}}}

**Example:**

{{{Claim (English): "COVID vaccines are 95% effective"

1267

Canonical: "covid vaccines are 95 percent effective"

1268

Language: "en"

1269

SHA256: abc123...def456

1270

Key: claim:v1norm1:en:abc123...def456

1271

}}}

1272

1273

**Rationale:** Prevents cross-language collisions and enables per-language cache analytics.

**Data Structure:**

{{{SET claim:v1norm1:en:abc123...def456 '{...ClaimAnalysis JSON...}'

1278

EXPIRE claim:v1norm1:en:abc123...def456 7776000 # 90 days

}}}

----

=== 5.1.1 Canonical Claim Normalization (v1) ===

1284

1285

The cache key depends on deterministic claim normalization. All implementations MUST follow this algorithm exactly.

1286

1287

**Algorithm: Canonical Claim Normalization v1**

1288

1289

1290

**Normative Algorithm:**

1291

1292

1293

def normalize_claim(text: str) -> str:

1294

"""

1295

Canonical claim normalization for deduplication.

1296

MUST follow this algorithm exactly.

Version: v1norm1

"""

import re

import unicodedata

# 1. Unicode normalization (NFD)

1304

text = unicodedata.normalize('NFD', text)

# 2. Lowercase

text = text.lower()

# 3. Remove diacritics

1310

text = ''.join(c for c in text if unicodedata.category(c) != 'Mn')

1311

1312

# 4. Normalize whitespace

1313

text = re.sub(r'\s+', ' ', text)

1314

text = text.strip()

1315

1316

# 5. Remove punctuation except apostrophes in contractions

1317

text = re.sub(r"[^\w\s']", '', text)

1318

1319

# 6. Normalize common contractions

1320

contractions = {

1321

"don't": "do not",

1322

"doesn't": "does not",

"didn't": "did not",

"can't": "cannot",

"won't": "will not",

"shouldn't": "should not",

1327

"wouldn't": "would not",

"isn't": "is not",

"aren't": "are not",

"wasn't": "was not",

"weren't": "were not",

1332

"haven't": "have not",

"hasn't": "has not",

"hadn't": "had not"

}

for contraction, expansion in contractions.items():

1338

text = re.sub(r'\b' + contraction + r'\b', expansion, text)

1339

1340

# 7. Remove remaining apostrophes

1341

text = text.replace("'", "")

1342

1343

# 8. Final whitespace normalization

1344

text = re.sub(r'\s+', ' ', text)

text = text.strip()

return text

**Normalization Examples:**

1351

1352

|= Input |= Normalized Output

1353

| "Biden won the 2020 election" | {{code}}biden won the 2020 election{{/code}}

1354

| "Biden won the 2020 election!" | {{code}}biden won the 2020 election{{/code}}

1355

| "Biden won the 2020 election" | {{code}}biden won the 2020 election{{/code}}

1356

| "Biden didn't win the 2020 election" | {{code}}biden did not win the 2020 election{{/code}}

1357

| "BIDEN WON THE 2020 ELECTION" | {{code}}biden won the 2020 election{{/code}}

1358

1359

**Versioning:** Algorithm version is {{code}}v1norm1{{/code}}. Changes to the algorithm require a new version identifier.

1360

1361

=== 5.1.2 Copyright & Data Retention Policy ===

1362

1363

**Evidence Excerpt Storage:**

1364

1365

To comply with copyright law and fair use principles:

**What We Store:**

* **Metadata only:** Title, author, publisher, URL, publication date

1370

* **Short excerpts:** Max 25 words per quote, max 3 quotes per evidence item

1371

* **Summaries:** AI-generated bullet points (not verbatim text)

1372

* **No full articles:** Never store complete article text beyond job processing

1373

1374

**Total per Cached Claim:**

1375

1376

* Scenarios: 2 per claim

1377

* Evidence items: 6 per scenario (12 total)

1378

* Quotes: 3 per evidence × 25 words = 75 words per item

1379

* **Maximum stored verbatim text:** ~~900 words per claim (12 × 75)

**Retention:**

* Cache TTL: 90 days

* Job outputs: 24 hours (then archived or deleted)

1385

* No persistent full-text article storage

**Rationale:**

* Short excerpts for citation = fair use

1390

* Summaries are transformative (not copyrightable)

1391

* Limited retention (90 days max)

1392

* No commercial republication of excerpts

**DMCA Compliance:**

* Cache invalidation endpoint available for rights holders

1397

* Contact: dmca@factharbor.org

----

== Summary ==

This WYSIWYG preview shows the **structure and key sections** of the 1,515-line API specification.

1404

1405

**Full specification includes:**

1406

1407

* Complete API endpoints (7 total)

1408

* All data schemas (ClaimExtraction, ClaimAnalysis, HolisticAssessment, Complete)

1409

* Quality gates & validation rules

1410

* LLM configuration for all 3 stages

1411

* Implementation notes with code samples

1412

* Testing strategy

1413

* Cross-references to other pages

1414

1415

**The complete specification is available in:**

1416

1417

* FactHarbor_POC1_API_and_Schemas_Spec_v0_4_1_PATCHED.md (45 KB standalone)

1418

* Export files (TEST/PRODUCTION) for xWiki import

Wiki source code of POC1 API & Schemas Specification