Cordy

Changed - Provider Architecture Refactoring

🎯 Major Architecture Improvement: Unified LiteLLM Provider

Breaking Changes: None - API interface remains unchanged

Summary: Consolidated all LLM provider implementations into a single unified LiteLLM-based architecture, reducing codebase by 73% while enhancing functionality.

Removed

src/core/providers/openai.py (501 lines) - Redundant OpenAI provider
src/core/providers/anthropic.py (585 lines) - Redundant Anthropic provider
src/core/providers/openrouter.py (541 lines) - Redundant OpenRouter provider
Total removed: 1,627 lines of code (73% reduction)

Enhanced

LiteLLM Provider (src/core/providers/litellm_provider.py)
- Added OpenRouter-specific configuration parameters:
  - openrouter_site_url: Optional site URL for analytics
  - openrouter_app_name: Optional app name for analytics
- Automatic HTTP-Referer header injection for OpenRouter models
- Automatic X-Title header injection for OpenRouter models
- Enhanced error handling and timeout management
- Support for 100+ LLM providers out of the box

Modified

Gateway Service (src/client_api/services/gateway.py)
- Simplified _initialize_providers() to use only LiteLLM
- Reduced _select_provider() from 50+ lines to 10 lines
- Unified provider routing logic
- Enhanced OpenRouter support with automatic header management
Provider Exports (src/core/providers/__init__.py)
- Updated to export only LiteLLM provider
- Removed references to deprecated direct providers

Benefits

Code Quality:

73% less code to maintain (2,200 → 600 lines)
Single source of truth for all provider logic
Consistent error handling across all providers
Unified retry mechanism

Features:

✅ OpenRouter analytics headers automatically injected
✅ Support for 100+ providers via LiteLLM
✅ Automatic parameter normalization
✅ Built-in cost calculation
✅ Enhanced streaming support

Maintenance:

Single codebase to debug and test
Automatic benefit from LiteLLM updates
Reduced testing complexity
Easier to add new providers (just configure API keys)

Supported Providers

All providers now accessed through unified LiteLLM interface:

OpenAI (GPT-3.5, GPT-4, GPT-4o, o1-preview, o1-mini)
Anthropic (Claude 3 Opus, Sonnet, Haiku; Claude 2.x)
OpenRouter (100+ models with automatic failover)
Google (Gemini Pro, PaLM 2)
Azure OpenAI
AWS Bedrock (Claude, Llama 2, Mistral, Cohere)
Cohere, Replicate, Hugging Face
Together AI, DeepInfra, Groq
Google Vertex AI
Ollama (local models)

Migration Guide

For Users: No changes required - API interface remains identical

For Developers:

# Old configuration (no longer needed)
USE_DIRECT_OPENAI="false"
USE_DIRECT_ANTHROPIC="false"

# New configuration (automatic via LiteLLM)
OPENAI_API_KEY="sk-..."
ANTHROPIC_API_KEY="sk-ant-..."
OPENROUTER_API_KEY="sk-or-..."

# OpenRouter analytics (optional but recommended)
OPENROUTER_SITE_URL="https://your-app.com"
OPENROUTER_APP_NAME="Your App Name"

OpenRouter Usage:

{
  "model": "openrouter/google/gemini-2.0-flash-exp:free",
  "messages": [{"role": "user", "content": "Hello"}]
}

Headers automatically injected:

{
  "HTTP-Referer": "https://your-app.com",
  "X-Title": "Your App Name"
}

Technical Details

Files Changed:

Modified: src/core/providers/litellm_provider.py
Modified: src/client_api/services/gateway.py
Modified: src/core/providers/__init__.py
Deleted: src/core/providers/openai.py
Deleted: src/core/providers/anthropic.py
Deleted: src/core/providers/openrouter.py

Testing:

✅ Health checks passing
✅ 50 models available
✅ OpenRouter headers correctly injected
✅ Error handling verified (rate limits, timeouts)
✅ Parameter validation confirmed

Documentation Updates

Updated docs/PROVIDERS_GUIDE.md with unified architecture details
Added comprehensive ADR in docs/ARCHITECTURE-DECISION.md
Updated README_V3.md with new architecture diagram
Updated CHANGELOG.md (this file)

References

ADR: "决策：统一 LLM Provider 架构" in docs/ARCHITECTURE-DECISION.md
Provider Guide: docs/PROVIDERS_GUIDE.md
Architecture Diagram: README_V3.md

Cordy Gateway