LLM Abstraction Architecture

graph LR
 subgraph AKEL["AKEL Pipeline"]
 S1[Stage 1
Extract Claims]
 S2[Stage 2
Analyze Claims]
 S3[Stage 3
Holistic Assessment]
 end
 
 subgraph LLM["LLM Abstraction Layer"]
 INT[Provider Interface]
 CFG[Configuration
Registry]
 FAIL[Failover
Handler]
 end
 
 subgraph Providers["LLM Providers"]
 ANT[Anthropic
Claude API
PRIMARY]
 OAI[OpenAI
GPT API
SECONDARY]
 GOO[Google
Gemini API
TERTIARY]
 LOC[Local Models
Llama/Mistral
FUTURE]
 end
 
 S1 --> INT
 S2 --> INT
 S3 --> INT
 
 INT --> CFG
 INT --> FAIL
 
 CFG --> ANT
 FAIL --> ANT
 FAIL --> OAI
 FAIL --> GOO
 
 ANT -.fallback.-> OAI
 OAI -.fallback.-> GOO
 
 style AKEL fill:#ffcccc
 style LLM fill:#ccffcc
 style Providers fill:#e1f5ff
 style ANT fill:#ff9999
 style OAI fill:#99ccff
 style GOO fill:#99ff99
 style LOC fill:#cccccc

LLM Abstraction Architecture - AKEL stages call through provider interface. Configuration registry selects provider per stage. Failover handler implements automatic fallback chain.

POC1 Implementation:

• PRIMARY: Provider A API (FAST model for Stage 1, REASONING model for Stages 2 & 3)
• Failover: Basic error handling with cache fallback

Future (POC2/Beta):

• SECONDARY: OpenAI GPT API (automatic failover)
• TERTIARY: Google Gemini API (tertiary fallback)
• FUTURE: Local models (Llama/Mistral for on-premises deployments)

Architecture Benefits:

• Prevents vendor lock-in
• Ensures resilience through automatic failover
• Enables cost optimization per stage
• Supports regulatory compliance (provider selection for data residency)

Description: Shows how AKEL stages interact with multiple LLM providers through an abstraction layer, enabling provider-agnostic AI calls with automatic failover.