LLM Configuration Guide

Overview

FactHarbor supports multiple LLM (Large Language Model) providers and search providers. This guide explains how to configure, optimize, and switch between providers to balance cost, performance, and quality.

Table of Contents

• Supported Providers
• Provider Selection
• Environment Configuration
• Provider-Specific Optimization
• Search Provider Configuration
• Cost Optimization
• Troubleshooting

Supported Providers

LLM Providers

Search Providers

Provider Selection

Setting the LLM Provider

Current (UCM): LLM provider is configured via UCM pipeline config.

To change provider:

1. Navigate to Admin → Config → Pipeline
2. Update llmProvider field (values: anthropic, openai, google, mistral)
3. Save and activate

Deprecated: The LLM_PROVIDER environment variable is no longer used (removed 2026-02-02).
If present in your .env file, it will be ignored.

Migration: If you previously set LLM_PROVIDER=openai, update your pipeline config to
"llmProvider": "openai" via the admin UI.

Provider API Keys

Important: Provider selection is configured in UCM (Admin → Config → Pipeline), but API keys remain environment-based for security.

Anthropic Claude (Recommended):

• Get key: https://console.anthropic.com/settings/keys
• Legacy default model (tiering off): claude-sonnet-4-20250514
• Best for: Complex analysis, high-quality reasoning

OpenAI:

• Get key: https://platform.openai.com/api-keys
• Legacy default model (tiering off): gpt-4o
• Best for: General purpose, established ecosystem

Google Gemini:

• Get key: https://aistudio.google.com/app/apikey
• Legacy default model (tiering off): gemini-1.5-pro
• Best for: Long context, multimodal inputs

Mistral AI:

• Get key: https://console.mistral.ai/api-keys
• Legacy default model (tiering off): mistral-large-latest
• Best for: Cost-conscious deployments

Environment Configuration

Complete .env.local Example

Provider selection and analysis behavior are configured in UCM (Admin → Config). The environment only
stores API keys and infrastructure settings.

Configuration Validation

Test your configuration at: http://localhost:3000/admin/test-config

This admin interface will:

• Validate all API keys
• Test LLM provider connectivity
• Test search provider connectivity
• Show which services are active

Provider-Specific Optimization

 Comprehensive Guide: See Provider-Specific Formatting for detailed documentation on v2.8.0 prompt architecture.

Overview (v2.8.0+)

FactHarbor uses provider-specific prompt variants to optimize performance across different LLMs. Each provider has unique strengths and preferred prompt structures:

General Optimization Principles

1. Prompt Composition:

• • Base prompt (universal logic)
  • Provider variant (format optimization)
  • Config adaptation (tiering, knowledge mode)

1. Temperature Settings:

• • Use FH_DETERMINISTIC=true for reproducible results (temperature = 0.0)
  • Override per-provider if needed via PromptConfig

1. Token Limits:

• • Claude: 200k tokens
  • GPT-4: 128k tokens
  • Gemini: 2M tokens
  • Mistral: 128k tokens

Configuration

Provider-specific settings are configured via PromptConfig in the Unified Config Management system:

See: Unified Config Management User Guide for profile management

Search Provider Configuration

Google Custom Search (Free Tier)

1. Get API Key: https://developers.google.com/custom-search/v1/introduction
2. Create Custom Search Engine: https://cse.google.com/cse/
3. Configure in UCM (Admin → Config → Web Search):

• • enabled: true
  • provider: google-cse

Environment keys:

Limits: 100 queries/day (free tier)

SerpAPI (Pay-per-use)

1. Sign up: https://serpapi.com
2. Get API Key: https://serpapi.com/manage-api-key
3. Configure in UCM (Admin → Config → Web Search):

• • enabled: true
  • provider: serpapi

Environment key:

Pricing: $0.002 per search

Auto Mode (Recommended)

Let FactHarbor choose the best available provider:

Set the search provider to auto in UCM (Admin → Config → Web Search).

Behavior:

• Tries Google CSE first (if configured)
• Falls back to SerpAPI for remaining slots
• Uses Gemini Grounded Search when pipeline.llmProvider=google and search mode=grounded (experimental). If grounding metadata/citations are missing, FactHarbor should fall back to standard search rather than treating model synthesis as evidence.

Domain Whitelist

Restrict searches to trusted domains:

This improves:

• Source reliability
• Result relevance
• Analysis quality

Cost Optimization

Prompt Optimization Baseline (v2.8.0)

Automatic estimated 20-30% cost reduction across all providers and models:

As of February 2026 (version 2.8.0), all prompts have been optimized to reduce token usage by an estimated 20-30% while maintaining quality. This optimization:

• Applies automatically to all LLM providers (Anthropic, OpenAI, Google, Mistral)
• Works with both tiered and non-tiered model configurations
• Reduces costs by an estimated 20-30% per analysis compared to previous versions
• Maintains analytical quality and accuracy

No configuration required - all users benefit from these savings automatically.

See Prompt Architecture v2.8.0 for technical details.

Multi-Tier Model Strategy

Use cheaper models for simple tasks, premium models for complex reasoning:

Note: Prompt optimization savings (estimated 20-30%) apply to ALL models and providers automatically.

Tiered model routing is configured via the pipeline config in Unified Configuration Management (UCM). Adjust the tiering toggle and per-task model names in the pipeline config editor (Admin UI → Config → Pipeline).

When enabled, FactHarbor routes models per pipeline task:

• Understand: cheaper/faster model (provider default)
• Extract evidence: cheaper/faster model (provider default)
• Verdict: higher-quality model (provider default)

Default per-provider routing (unless overridden in pipeline config):

• Anthropic: understand/extract → claude-3-5-haiku-20241022, verdict → claude-sonnet-4-20250514
• OpenAI: understand/extract → gpt-4o-mini, verdict → gpt-4o
• Google: understand/extract → gemini-1.5-flash, verdict → gemini-1.5-pro
• Mistral: understand/extract → mistral-small-latest, verdict → mistral-large-latest

Per-task model overrides (optional)

Override the model names per task via the pipeline config (UCM), not environment variables. See Unified Config Management User Guide for editing pipeline profiles.

Tuning analysis depth (implemented)

FactHarbor's primary "cost vs quality" knob is still analysis mode; tiered model routing is optional and can further reduce cost by using cheaper models for extraction-style steps:

This affects limits like max iterations and max total sources (see apps/web/src/lib/analyzer/config.ts).

Cost Control Settings

Cost Monitoring

Track costs via the admin dashboard (when implemented):

• LLM token usage
• Search API calls
• Estimated monthly spend

Troubleshooting

LLM Provider Issues

"Invalid API key" error:

1. Verify key format:

• • Anthropic: starts with sk-ant-
  • OpenAI: starts with sk-
  • Google: starts with AIza

1. Test key at provider's console
2. Check for spaces/quotes in .env.local
3. Restart web server after changes

"Rate limit exceeded":

• Check your API plan limits
• Wait for rate limit reset
• Consider upgrading to paid tier

"Model not found":

• Verify model name matches provider's API
• Check if you have access to the model
• Some models require special access approval

Search Provider Issues

"No search results":

1. Verify search is enabled in UCM (Search config)
2. Check search provider API key is valid
3. Verify domain whitelist isn't too restrictive
4. Check search provider status page

"Quota exceeded" (Google CSE):

• Free tier: 100 queries/day
• Upgrade to paid tier or switch to SerpAPI
• Use provider=auto in UCM for automatic fallback

"Search provider timeout":

• Network connectivity issues
• Provider service disruption
• Try alternative provider

"No sources were fetched" (All pipelines including Dynamic):

All three pipelines (Orchestrated, Monolithic Canonical, Monolithic Dynamic) require search provider credentials to perform web searches. Without them, the pipeline will:

1. Generate search queries (correctly)
2. Attempt searches (loop runs but returns empty)
3. Continue without external sources (LLM uses only internal knowledge)

Verify search providers are configured:

Solution: Configure at least one search provider:

Verify configuration is working:
Look for successful search logs:

Configuration Validation

Test configuration without running analysis:

Visit: http://localhost:3000/admin/test-config

Manual API test:

Provider Comparison

Quality vs Cost Trade-offs

Recommended Provider by Use Case

• Production fact-checking: Anthropic Claude Sonnet
• Development/testing: Claude Haiku or Mistral
• Long articles (>10k words): Google Gemini
• Tight budget: Mistral Large
• Maximum accuracy: GPT-4 or Claude Sonnet

Advanced Configuration

Custom Model Selection

FactHarbor supports task-tiered model routing. Configure tiering and per-task model overrides in the pipeline config (UCM) rather than environment variables. See Unified Config Management User Guide for editing pipeline profiles.

Fallback Configuration

The text-analysis pipeline is LLM-only and does not support heuristic or provider fallbacks. LLM provider selection and routing are controlled via the pipeline config.

Knowledge Toggle

Control whether the LLM can use background knowledge:

 Known Issue: This toggle is not fully respected in all analysis phases. The Understanding step currently allows model knowledge regardless of this setting.

Testing Recommendations

When switching providers or configurations:

1. Run test analysis with known inputs

• • Compare output quality
  • Check for parsing errors
  • Verify consistency

1. Test edge cases

• • Long articles
  • Complex claims
  • Multi-language content

1. Monitor costs

• • Track token usage
  • Compare pricing across providers
  • Optimize based on actual usage

1. Measure performance

• • Response times
  • Success rates
  • Error frequency

Getting Help

Resources

• Anthropic Docs: https://docs.anthropic.com
• OpenAI Docs: https://platform.openai.com/docs
• Google AI Docs: https://ai.google.dev/docs
• Mistral Docs: https://docs.mistral.ai

Support

• Check apps/web/debug-analyzer.log for detailed logs
• Use admin test config to validate setup
• Review provider status pages for outages
• Search GitHub issues for similar problems

Last Updated: February 2, 2026