Backend Architecture Documentation

NOTE: THIS IS CREATED BY LLM. USE THIS AS A REFERENCE, BUT DYOR

Overview

The backend is a FastAPI-based cryptocurrency price tracking service that provides real-time price data comparison across multiple exchanges. It follows a modular architecture with clear separation of concerns, utilizing the CCXT library for exchange integrations and Redis for caching (though caching is currently not actively implemented in the data flow).

Project Structure

backend/
├── src/
│   ├── config/
│   │   └── config.py              # Configuration settings and constants
│   ├── routes/
│   │   ├── main.py               # FastAPI app setup and middleware
│   │   ├── crypto_data.py        # Cryptocurrency data endpoints
│   │   ├── static_data.py        # Configuration data endpoints
│   │   └── models/
│   │       └── schemas.py        # Pydantic models and data validation
│   ├── services/
│   │   ├── api_call_manager.py   # Main orchestration service
│   │   ├── external_api_caller.py # CCXT wrapper for exchange APIs
│   │   └── caching.py           # Redis caching implementation (not used)
│   └── utils/
│       └── timeframes_equalizer.py # Data synchronization utilities
├── tests/
│   ├── __init__.py
│   └── test_api.py              # API endpoint tests
├── requirements.txt
└── Dockerfile

Core Components

1. Configuration Layer (config/config.py)

Purpose: Centralized configuration management and constants definition.

Key Components:

• Settings: Pydantic settings class for environment variables

  • REDIS_HOST: Redis server hostname (default: "redis")
  • REDIS_PORT: Redis server port (default: 6379)
  • REDIS_DB: Redis database number (default: 0)

• Exchange: Enum defining supported cryptocurrency exchanges

  class Exchange(str, Enum):
      BINANCE = "binance"
      OKX = "okx"
      BYBIT = "bybit"
      MEXC = "mexc"
      BINGX = "bingx"
      GATEIO = "gateio"
      KUCION = "kucoin"
  

• TickerType: Enum for different data types

  class TickerType(StrEnum):
      OHLC = auto()
      CHART_LINE = auto()
  

Constants:

• CACHE_TTL_CONFIG: TTL settings for different time intervals
• SUPPORTED_PAIRS: Available cryptocurrency pairs (18 major cryptocurrencies)
• TIME_RANGES: Available time intervals (5m, 30m, 1h, 4h, 1d, 1w, 1M)
• SUPPORTED_EXCHANGES: Exchange mapping dictionary

2. Data Models (routes/models/schemas.py)

Purpose: Define request/response schemas and data validation using Pydantic.

Key Models:

• CompareRequest: Parameters for comparing data between two exchanges

  class CompareRequest(BaseModel):
      exchange1: Exchange
      exchange2: Exchange
      crypto_id: str
      interval: str = "1h"
  

• PriceTicketRequest: Parameters for fetching data from a single exchange

  class PriceTicketRequest(BaseModel):
      crypto_id: str
      interval: str = "1h"
      api_provider: Exchange
  

• config_types: Static configuration data for frontend consumption

  config_types: dict[str, dict] = {
      "timeranges": TIME_RANGES,
      "pairs": SUPPORTED_PAIRS,
      "exchanges": SUPPORTED_EXCHANGES,
  }
  

3. API Routes Layer

Main Application (routes/main.py)

Responsibilities:

• FastAPI app initialization
• CORS middleware configuration (allows all origins)
• Router registration
• Global exception handling for ValueError
• Dependency injection for ApiCallManager

Crypto Data Routes (routes/crypto_data.py)

Endpoints:

• GET /crypto/ohlc: OHLC (candlestick) data comparison between two exchanges
• GET /crypto/line-compare: Line chart data comparison between two exchanges

Implementation:

• Uses FastAPI's Query() for automatic request parsing
• Delegates processing to ApiCallManager
• Returns JSON responses with exchange data keyed by exchange name

Static Data Routes (routes/static_data.py)

Endpoints:

• GET /static/config/{config_type}: Configuration data (exchanges, pairs, timeranges)

Supported config types:

• exchanges: List of supported exchanges
• pairs: Available cryptocurrency pairs
• timeranges: Available time intervals

4. Services Layer

API Call Manager (services/api_call_manager.py)

Purpose: Main orchestration service that coordinates data fetching and processing.

Key Methods:

• get_timeframe_aligned(): Fetches and synchronizes data from multiple exchanges
  • Creates PriceTicketRequest objects for each exchange
  • Uses asyncio.gather() for concurrent API calls
  • Processes data through Equalizer for timestamp alignment
  • Returns data formatted by exchange name

Dependencies:

• CryptoFetcher: External API calls
• Equalizer: Data synchronization
• Cacher: Redis caching (instantiated but not used)

Data Flow:

1. Creates requests for both exchanges
2. Fetches data concurrently from CCXT
3. Determines columns to drop based on chart type
4. Equalizes timeframes between exchanges
5. Returns formatted response

External API Caller (services/external_api_caller.py)

Purpose: CCXT wrapper for cryptocurrency exchange APIs.

Key Features:

• Exchange Management: Maintains internal dictionary of exchange instances
• Lazy Loading: Creates exchange instances on first use
• Async Operations: All API calls are asynchronous
• Connection Cleanup: Provides method to close all connections

Key Methods:

• get_ohlc(): Fetches OHLC data from specified exchange
• get_exchange(): Returns cached or creates new exchange instance
• get_ccxt_exchange(): Factory method for CCXT exchange instances
• close_all(): Cleanup method for all exchange connections

Caching Service (services/caching.py)

Purpose: Redis-based caching implementation.

⚠️ Current Status: The caching service is implemented but not integrated into the data flow. The ApiCallManager instantiates a Cacher but never calls its methods.

Key Methods:

• set(): Cache data with TTL

  def set(self, data: str, request: PriceTicketRequest, ttl: int) -> None
  

• get(): Retrieve cached data

  def get(self, request: PriceTicketRequest) -> str
  

• construct_key(): Generate cache keys in format {exchange}:{crypto_id}:{interval}

Cache Key Examples:

• binance:BTC-USDT:1h
• okx:ETH-USDT:4h

5. Utilities Layer

Timeframes Equalizer (utils/timeframes_equalizer.py)

Purpose: Synchronize data from different exchanges to common timeframes using pandas.

Key Features:

• DataFrame Operations: Converts raw CCXT data to pandas DataFrames
• Timestamp Alignment: Uses align() with inner join to match timestamps
• Column Filtering: Removes unnecessary columns based on chart type
• Data Transformation: Converts timestamps from milliseconds to seconds

Key Methods:

• equalize_timeframes(): Main method for data synchronization
  • Takes two raw data arrays from CCXT
  • Converts to DataFrames with timestamp index
  • Drops unnecessary columns (volume for line charts)
  • Aligns timestamps using inner join
  • Returns list of dictionaries for JSON serialization

Column Management:

self.cnames = ["time", "open", "high", "low", "close", "volume"]

• OHLC charts: Drops volume column
• Line charts: Drops high, low, close, volume columns

Data Flow Architecture

graph TD
    A[Frontend Request] --> B[FastAPI Routes]
    B --> C[API Call Manager]
    C --> D[Create PriceTicketRequests]
    D --> E[External API Caller]
    E --> F[CCXT Exchange APIs]
    F --> G[Raw OHLC Data Arrays]
    G --> H[Timeframes Equalizer]
    H --> I[Synchronized DataFrames]
    I --> J[JSON Response]
    J --> K[Return to Frontend]
    
    C -.->|Not Used| L[Redis Cache]
    
    style L fill:#ffcccc,stroke:#ff0000

Note: Redis caching is implemented but not integrated into the current data flow.

Class Communication Patterns

1. Request Processing Flow

1. FastAPI Routes receive HTTP requests and validate using Pydantic schemas
2. API Call Manager orchestrates the data fetching process
3. External API Caller manages CCXT exchange instances and fetches data
4. Timeframes Equalizer synchronizes data from multiple sources
5. Response returned as JSON to frontend

2. Dependency Injection Pattern

• ApiCallManager acts as the main coordinator
• All services are instantiated within ApiCallManager
• Configuration is imported statically where needed
• No circular dependencies between components

3. Async Operations

• All external API calls use asyncio.gather() for concurrency
• CCXT exchanges support async operations
• Connection cleanup handled properly

4. Error Handling

• Pydantic Validation: At request level with automatic error responses
• Global Exception Handler: Catches ValueError and returns 400 status
• CCXT Errors: Propagated up through the call stack

Key Design Patterns

1. Facade Pattern

• ApiCallManager acts as a facade, hiding complexity of multiple services
• Provides simple interface for route handlers

2. Repository Pattern

• CryptoFetcher abstracts CCXT operations
• Cacher abstracts Redis operations (though not used)

3. Factory Pattern

• Dynamic exchange instantiation in CryptoFetcher
• get_ccxt_exchange() creates exchange instances by name

4. Strategy Pattern

• Different data processing strategies based on TickerType
• Column filtering varies by chart type

Current Issues and Limitations

1. Unused Redis Caching ⚠️

Problem: The Redis caching system is fully implemented but never used in the actual data flow.

Impact:

• All requests hit external APIs directly
• No performance benefits from caching
• Increased API rate limiting risk
• Higher response times

Solution: Integrate caching into ApiCallManager.get_timeframe_aligned():

# Check cache first
cached_data = self.redis_cacher.get(request)
if cached_data:
    return json.loads(cached_data)

# Fetch from API and cache result
data = await self.fetcher.get_ohlc(request)
ttl = CACHE_TTL_CONFIG.get(request.interval, 3600)
self.redis_cacher.set(json.dumps(data), request, ttl)

2. Missing Error Handling

• No specific handling for CCXT API failures
• No retry mechanisms for failed requests
• Limited error context for debugging

3. No Rate Limiting

• Direct API calls without throttling
• Risk of hitting exchange rate limits

Configuration Management

Environment Variables

Managed through Pydantic Settings:

class Settings(BaseSettings):
    REDIS_HOST: str = "redis"      # Docker service name
    REDIS_PORT: int = 6379         # Standard Redis port
    REDIS_DB: int = 0              # Default Redis database

Static Configuration

All configuration is centralized in config.py:

• Exchanges: 7 supported exchanges via CCXT
• Pairs: 18 major cryptocurrency pairs
• Time Ranges: 7 intervals from 5m to 1M
• Cache TTL: Corresponding TTL for each interval

Testing Structure

API Tests (tests/test_api.py)

Test Coverage:

• Configuration endpoint validation
• Data endpoint functionality
• Error handling scenarios
• Response format validation
• Invalid parameter handling

Test Patterns:

• Uses FastAPI TestClient
• Tests both success and error paths
• Validates response schemas

Performance Considerations

Current Performance Characteristics

1. Concurrent API Calls: Uses asyncio.gather() for parallel requests
2. Connection Reuse: CCXT exchanges are cached and reused
3. DataFrame Operations: Pandas for efficient data processing
4. No Caching: All requests hit external APIs (performance issue)

Recommended Optimizations

1. Implement Redis Caching: Massive performance improvement
2. Connection Pooling: Better CCXT connection management
3. Request Batching: Group similar requests
4. Background Tasks: Pre-fetch popular data

Adding New Features

Adding a New Exchange

1. Add to Exchange enum in config.py
2. Update SUPPORTED_EXCHANGES dictionary
3. CCXT handles integration automatically (if supported)

Adding New Time Intervals

1. Add to TIME_RANGES in config.py
2. Add corresponding TTL to CACHE_TTL_CONFIG
3. Ensure interval format matches CCXT standards

Adding New Cryptocurrency Pairs

1. Add to SUPPORTED_PAIRS in config.py
2. Verify pair exists on target exchanges
3. Test with different exchanges for availability

Implementing Caching (Recommended)

1. Modify ApiCallManager.get_timeframe_aligned() to check cache first
2. Serialize/deserialize data properly (JSON)
3. Handle cache misses gracefully
4. Add cache invalidation strategies

Development Setup

Prerequisites

1. Python 3.11+: Backend runtime
2. Redis: Caching (Docker or local)
3. Dependencies: Install from requirements.txt

Key Dependencies

fastapi          # Web framework
uvicorn          # ASGI server
ccxt             # Cryptocurrency exchange library
pandas           # Data processing
redis            # Caching
pydantic         # Data validation
pydantic-settings # Configuration management
pytest           # Testing

Running the Backend

cd backend
pip install -r requirements.txt
uvicorn src.routes.main:app --reload --host 0.0.0.0 --port 8000

API Documentation

FastAPI automatically generates interactive documentation at:

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc

Security Considerations

Current Security

• CORS: Currently allows all origins (allow_origins=["*"])
• No Authentication: All endpoints are public
• No Rate Limiting: Vulnerable to abuse

Recommended Security Improvements

1. Restrict CORS: Limit to specific frontend domains
2. API Rate Limiting: Implement per-IP throttling
3. Input Validation: Enhanced Pydantic schemas
4. Monitoring: Add request logging and alerting

Monitoring and Observability

Current Logging

• Uses uvicorn.error logger
• Basic cache hit/miss logging in Cacher

Recommended Improvements

1. Structured Logging: JSON format with correlation IDs
2. Metrics: Track API response times, cache hit rates
3. Health Checks: Endpoint for system status
4. Error Tracking: Integration with error monitoring services

Deployment Considerations

Docker Configuration

• Multi-stage builds: Optimize image sizes
• Health checks: Container health monitoring
• Resource limits: Memory and CPU constraints
• Environment-specific configs: Development vs. production settings

Production Readiness Checklist

• Implement Redis caching
• Add proper error handling and retries
• Configure rate limiting
• Set up monitoring and alerting
• Implement proper logging
• Add health checks
• Configure CORS for production
• Set up database connection pooling
• Add request/response validation
• Implement circuit breakers for external APIs