POC1 API & Schemas Specification

Version: 0.4.1 (POC1)  
Scope: POC1 (Test) — URL/Text → Stage 1 Claim Extraction → Stage 2 Claim Analysis (cached) → Stage 3 Article Assessment → result.json + report.md

Version History

POC1 Codegen Contract (Canonical)

1) Goals

POC1 provides:

• A REST API to submit either a URL or raw text
• Asynchronous execution (job-based) with progress/status and optional SSE events
• Deterministic outputs:
  • ``result.json`` (machine-readable; schema-validated)
  • ``report.md`` (human-readable; rendered from JSON via template)
• Mandatory counter-evidence attempt per scenario (or explicit “not found despite search” uncertainty note)
• Claim-level caching to validate cost sustainability

2) Non-goals (POC1)

• Full editorial workflow / human review UI
• Full provenance ledger beyond minimal job metadata
• Complex billing/rate-limit systems (simple caps are fine)
• Multi-language i18n beyond best-effort language detection

3) Pipeline model (locked)

• Stage 1 — Claim Extraction
  • Input: URL-text or pasted text
  • Output: claim candidates including canonical claim + ``claim_hash``
• Stage 2 — Claim Analysis (cached by claim_hash)
  • Input: canonical claim(s)
  • Output: scenarios, evidence/counter-evidence, scenario verdicts, plus a claim-level verdict summary
• Stage 3 — Article Assessment
  • Input: article text + Stage 2 analyses (from cache and/or freshly computed)
  • Output: article-level assessment (main thesis, reasoning quality, key risks, context)

4) Locked verdict taxonomies (two enums + mapping)

Scenario verdict enum (``ScenarioVerdict.verdict_label``):

• ``Highly likely`` | ``Likely`` | ``Unclear`` | ``Unlikely`` | ``Highly unlikely`` | ``Unsubstantiated``

Claim verdict enum (``ClaimVerdict.verdict_label``):

• ``Supported`` | ``Refuted`` | ``Inconclusive``

Mapping rule (for summaries):

• If the claim’s primary-interpretation scenario verdict is:
  • ``Highly likely`` / ``Likely`` ⇒ ``Supported``
  • ``Highly unlikely`` / ``Unlikely`` ⇒ ``Refuted``
  • ``Unclear`` / ``Unsubstantiated`` ⇒ ``Inconclusive``
• If scenarios materially disagree (different assumptions lead to different outcomes) ⇒ ``Inconclusive``, and explain why.

5) Deterministic canonical claim normalization (locked)

Normalization version: ``v1norm1``  
Cache key namespace: ``claim:v1norm1:{language}:{sha256(canonical_claim_text)}``

Rules (MUST be implemented exactly; any change requires bumping normalization version):

1. Unicode normalize: NFD
   2. Lowercase
   3. Strip diacritics
   4. Normalize apostrophes: ``’`` and ``‘`` → ``'``
   5. Normalize percent: ``%`` → `` percent``
   6. Normalize whitespace (collapse)
   7. Remove punctuation except apostrophes
   8. Expand contractions (fixed list for v1norm1)
   9. Normalize whitespace again

Contractions list (v1norm1):

• don't→do not, doesn't→does not, didn't→did not, can't→cannot, won't→will not,
• shouldn't→should not, wouldn't→would not, isn't→is not, aren't→are not,
• wasn't→was not, weren't→were not

Reference implementation (normative):
import re
import unicodedata

# Canonical claim normalization for deduplication.
# Version: v1norm1
#
# IMPORTANT:
# - Any change to these rules REQUIRES a new normalization version.
# - Cache keys MUST include the normalization version to avoid collisions.

CONTRACTIONS_V1NORM1 = {
   "don't": "do not",
   "doesn't": "does not",
   "didn't": "did not",
   "can't": "cannot",
   "won't": "will not",
   "shouldn't": "should not",
   "wouldn't": "would not",
   "isn't": "is not",
   "aren't": "are not",
   "wasn't": "was not",
   "weren't": "were not",
}

def normalize_claim(text: str) -> str:
   """Canonical claim normalization (v1norm1)."""
   if text is None:
       return ""

   # 1) Unicode normalization
    text = unicodedata.normalize("NFD", text)

   # 2) Lowercase
    text = text.lower()

   # 3) Remove diacritics
    text = "".join(c for c in text if unicodedata.category(c) != "Mn")

   # 4) Normalize apostrophes (common unicode variants)
    text = text.replace("’", "'").replace("‘", "'")

   # 5) Normalize percent sign
    text = text.replace("%", " percent")

   # 6) Normalize whitespace
    text = re.sub(r"\s+", " ", text).strip()

   # 7) Remove punctuation except apostrophes
    text = re.sub(r"[^\w\s']", "", text)

   # 8) Expand contractions (fixed list for v1norm1)
   for k, v in CONTRACTIONS_V1NORM1.items():
        text = re.sub(rf"\b{re.escape(k)}\b", v, text)

   # 9) Final whitespace normalization
    text = re.sub(r"\s+", " ", text).strip()

   return text

6) Report generation (locked)

• ``report.md`` MUST be generated from ``result.json`` using a deterministic template (server-side).
• The LLM MUST NOT be the sole renderer of report markdown.

7) No chain-of-thought storage/exposure (locked)

• Store/expose only short, user-visible rationale bullets.
• Never store or expose internal reasoning traces.

REST API (POC1)

1) API Versioning

Base path: ``/v1``

2) Content types

• Requests: ``application/json``
• JSON responses: ``application/json``
• Report download: ``text/markdown; charset=utf-8``
• SSE events: ``text/event-stream``

3) Time & IDs

• All timestamps: ISO 8601 UTC (e.g., ``2025-12-24T19:31:30Z``)
• ``job_id``: ULID (string)
• ``claim_hash``: sha256 hex string (lowercase), computed over canonical claim + version + language as specified

4) Authentication

All ``/v1`` endpoints require:

• Header: ``Authorization: Bearer <API_KEY>``

5) Error Envelope (all non-2xx)

Endpoints

1) POST /v1/analyze

Creates an asynchronous job. Exactly one of ``input_url`` or ``input_text`` MUST be provided.

Request (AnalyzeRequest)
{
 "input_url": "https://example.org/article",
 "input_text": null,
 "options": {
   "max_claims": 5,
   "cache_preference": "prefer_cache",
   "browsing": "on",
   "output_report": true
  },
 "client": {
   "request_id": "optional-idempotency-key"
  }
}

Options

• ``max_claims``: integer, 1..50 (default 5)
• ``cache_preference``: ``prefer_cache`` | ``allow_partial`` | ``cache_only`` | ``skip_cache``
  • ``prefer_cache``: use cache when available; otherwise run full pipeline
  • ``allow_partial``: if Stage 2 cached analyses exist, server MAY skip Stage 1+2 and rerun only Stage 3 using cached analyses
  • ``cache_only``: do not run Stage 2 for uncached claims; fail with CACHE_MISS (402) if required claim analyses are missing
  • ``skip_cache``: ignore cache and recompute
• ``browsing``: ``on`` | ``off``
  • ``off``: do not retrieve evidence; mark evidence items as ``NEEDS_RETRIEVAL`` and output retrieval queries
• ``output_report``: boolean (default true)

Response: 202 Accepted (JobCreated)
{
 "job_id": "01J8Y9K6M2Q1J0JZ7E5P8H7Y9C",
 "status": "QUEUED",
 "created_at": "2025-12-24T19:30:00Z",
 "estimated_cost": {
   "credits": 420,
   "explain": "Estimate may change after Stage 1 (claim count)."
  },
 "links": {
   "self": "/v1/jobs/01J8Y9K6M2Q1J0JZ7E5P8H7Y9C",
   "events": "/v1/jobs/01J8Y9K6M2Q1J0JZ7E5P8H7Y9C/events",
   "result": "/v1/jobs/01J8Y9K6M2Q1J0JZ7E5P8H7Y9C/result",
   "report": "/v1/jobs/01J8Y9K6M2Q1J0JZ7E5P8H7Y9C/report"
  }
}

Cache-only failure: 402 (CACHE_MISS)
{
 "error": {
   "code": "CACHE_MISS",
   "message": "cache_only requested but a required claim analysis is not cached.",
   "details": {
     "missing_claim_hash": "9f2a...c01",
     "normalization_version": "v1norm1"
    }
  }
}

2) GET /v1/jobs/{job_id}

Returns job status and progress.

Response: 200 OK (Job)
{
 "job_id": "01J...",
 "status": "RUNNING",
 "created_at": "2025-12-24T19:30:00Z",
 "updated_at": "2025-12-24T19:31:12Z",
 "progress": {
   "stage": "STAGE2_CLAIM_ANALYSIS",
   "stage_progress": 0.4,
   "message": "Analyzing claim 3/8"
  },
 "links": {
   "events": "/v1/jobs/01J.../events",
   "result": "/v1/jobs/01J.../result",
   "report": "/v1/jobs/01J.../report"
  }
}

Statuses:

• ``QUEUED`` → ``RUNNING`` → ``SUCCEEDED`` | ``FAILED`` | ``CANCELED``

Stages:

• ``STAGE1_CLAIM_EXTRACT``
• ``STAGE2_CLAIM_ANALYSIS``
• ``STAGE3_ARTICLE_ASSESSMENT``

3) GET /v1/jobs/{job_id}/events

Server-Sent Events (SSE) for job progress reporting (no token streaming).

Event types (minimum):

• ``job.created``
• ``stage.started``
• ``stage.progress``
• ``stage.completed``
• ``job.succeeded``
• ``job.failed``

4) GET /v1/jobs/{job_id}/result

Returns JSON result.

• ``200`` if job ``SUCCEEDED``
• ``409`` if not finished

5) GET /v1/jobs/{job_id}/report

Returns ``text/markdown`` report.

• ``200`` if job ``SUCCEEDED``
• ``409`` if not finished

6) DELETE /v1/jobs/{job_id}

Cancels a job (best-effort) and deletes stored artifacts if supported.

• ``204`` success
• ``404`` if not found

7) GET /v1/health

Response: 200 OK
{
 "status": "ok",
 "service": "factharbor-poc1",
 "version": "v0.9.105",
 "time": "2025-12-24T19:31:30Z"
}

Idempotency (Required)

Clients SHOULD send:

• Header: ``Idempotency-Key: <string>`` (preferred)
  or
• Body: ``client.request_id``

Server behavior:

• Same key + same request body → return existing job (``200``) with:
  • ``idempotent=true``
  • ``original_request_at``
• Same key + different request body → ``409`` with ``VALIDATION_ERROR`` and mismatch details

Idempotency TTL: 24 hours (matches minimum job retention)

Schemas (result.json)

Top-level AnalysisResult

Mandatory counter-evidence rule (POC1)

For each scenario, attempt to include at least one evidence item with:

• ``stance`` ∈ {``undermines``, ``mixed``, ``context_dependent``}

If not found:

• include explicit “not found despite targeted search” note in ``uncertainty_factors``
• and/or include evidence items with ``retrieval_status=FAILED`` indicating the attempted search

URL Extraction Rules (POC1)

• Primary extraction: Jina Reader (if enabled)
• Fallback: Trafilatura (or equivalent)

SSRF protections (required):

• Block local IPs, metadata IPs, file:// URLs, and internal hostnames
• If blocked: return ``UPSTREAM_FETCH_ERROR`` with reason

Copyright/ToS safe storage policy (POC1):

• Store only metadata + short excerpts
• Excerpts: max 25 words per quote, and cap total quotes per source

Cost Control Knobs (POC1 defaults)

Defaults:

• ``max_claims = 5``
• ``scenarios_per_claim = 2..3`` (internal Stage 2 policy)
• Cap evidence items per scenario (recommended: 6 total; at least 1 counter)
• Keep rationales concise (bullets)

Minimal OpenAPI 3.1 (POC1)

End of page.