Complete OpenAI Codex Setup Guide for Python Projects with Tests - PrototypeJam/lake_merritt GitHub Wiki

Complete Codex Setup Guide for Python Projects with Tests

1. Project Structure Requirements

Essential Files to Create Before Using Codex:

AGENTS.md (Project Root)

# AGENTS.md

## [Project Name]

### Environment Setup
This project requires Python 3.9+ and uses requirements.txt for dependency management.
We use `uv` for fast, reliable dependency installation.

### Testing Guidelines

**IMPORTANT**: Many tests require API keys that are not available in the CI environment. 
Always run tests with the marker filter to skip API-dependent tests:

```bash
# Run all tests EXCEPT those requiring API keys
pytest -v -m "not requires_api"

# Run only unit tests (recommended for CI)
pytest tests/unit -v -m "not requires_api"

# If you need to run a specific test file
pytest tests/unit/test_exact_match.py -v

Code Style

  • Use Black for formatting
  • Type hints are required for all new functions
  • Docstrings follow Google style

Common Tasks

  • Install dependencies: uv pip install -r requirements.txt
  • Run safe tests: pytest -v -m "not requires_api"
  • Run a specific test: pytest tests/unit/test_example.py -v
  • Check types: mypy core --ignore-missing-imports
  • Format code: black core tests

Important Notes

  • Do NOT commit API keys or .env files
  • Tests marked with requires_api will be skipped in CI environments
  • Focus test efforts on business logic that doesn't require external APIs

#### **tests/conftest.py**
```python
import pytest
import os

def pytest_configure(config):
    config.addinivalue_line(
        "markers", "requires_api: marks tests that need API keys"
    )

@pytest.fixture
def mock_api_keys():
    return {
        "openai": "test-key",
        "anthropic": "test-key",
        "google": "test-key"
    }

pyproject.toml (Dynamic Dependencies)

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "your-project-name"
version = "0.1.0"
description = "Your project description"
readme = "README.md"
requires-python = ">=3.9"
license = {text = "MIT"}
dynamic = ["dependencies"]

[tool.setuptools.dynamic]
dependencies = {file = ["requirements.txt"]}

2. Codex Environment Configuration

Step 1: Create/Select Environment

  1. In ChatGPT → Codex → Settings → Environments
  2. Select your repository
  3. Switch from "Basic" to "Advanced" mode

Step 2: Container Settings

  • Container Image: universal (default)
  • Internet Access:
    • Setup phase: ON (required for installing dependencies)
    • Runtime: ON (required for pip/pytest to work)

Step 3: Domain Allowlist (CRITICAL)

Add these domains to allow package installation while blocking unwanted access:

pypi.org
files.pythonhosted.org
*.python.org
github.com
raw.githubusercontent.com
*.pythonhosted.org
astral.sh

Step 4: Setup Script

#!/usr/bin/env bash
set -euo pipefail

# Install uv if not present (Codex might not have it)
if ! command -v uv &> /dev/null; then
    echo "Installing uv..."
    curl -LsSf https://astral.sh/uv/install.sh | sh
    export PATH="$HOME/.cargo/bin:$PATH"
fi

# Create virtual environment with uv
uv venv

# Activate the virtual environment
source .venv/bin/activate

# Install all dependencies with uv
uv pip install -r requirements.txt

# Install development dependencies
uv pip install pytest pytest-asyncio pytest-cov

# Pre-compile any large models/resources if needed
python -c "import nltk; nltk.download('punkt', quiet=True)" 2>/dev/null || true

# Verify critical imports
python -c "import streamlit, pandas, openai, anthropic, google.generativeai"
echo "✓ Core dependencies installed successfully"

# Make sure pytest can find the project modules
export PYTHONPATH="${PYTHONPATH:-}${PYTHONPATH:+:}${PWD}"

# Disable any API keys during testing
export OPENAI_API_KEY=""
export ANTHROPIC_API_KEY=""
export GOOGLE_API_KEY=""

echo "✓ Environment configured for safe testing (API keys disabled)"

3. Test Configuration

Mark API-Dependent Tests

import pytest

@pytest.mark.requires_api
def test_that_needs_real_api():
    """This test will be skipped in CI/Codex"""
    pass

def test_normal_unit_test():
    """This test will run normally"""
    pass

Integration Test Example

# tests/integration/test_example.py
import pytest

class TestIntegration:
    @pytest.mark.requires_api
    @pytest.mark.asyncio
    async def test_llm_generation(self):
        """Test that requires real API keys"""
        # This will be skipped
        pass
    
    def test_data_processing(self):
        """Test that doesn't need APIs"""
        # This will run normally
        pass

4. Common Issues and Solutions

Issue: "ModuleNotFoundError: No module named 'pandas'"

Cause: Dependencies not installed or virtual environment not activated Solution: Ensure setup script runs successfully and includes all dependencies

Issue: "PYTHONPATH: unbound variable"

Cause: Using undefined variable with set -u Solution: Use export PYTHONPATH="${PYTHONPATH:-}${PYTHONPATH:+:}${PWD}"

Issue: Network/Proxy Errors (403 Forbidden)

Cause: Internet access disabled or domains not allowlisted Solution: Enable internet access and add required domains to allowlist

Issue: Tests try to use real API keys

Cause: API keys present in environment Solution: Export empty API keys in setup script

5. Pre-Flight Checklist

Before running Codex on your project:

  • Create AGENTS.md with testing instructions
  • Create tests/conftest.py with pytest configuration
  • Mark all API-dependent tests with @pytest.mark.requires_api
  • Ensure requirements.txt includes all dependencies
  • Update pyproject.toml to use dynamic dependencies
  • Configure Codex environment with proper setup script
  • Enable internet access with domain allowlist
  • Test locally that pytest -v -m "not requires_api" works

6. Quick Setup Template

For new Python/Streamlit projects, use this template:

# 1. Create project structure
mkdir -p tests/unit tests/integration
touch tests/conftest.py
touch AGENTS.md

# 2. Copy the templates from above into:
#    - AGENTS.md
#    - tests/conftest.py
#    - pyproject.toml (update project name)

# 3. In Codex settings:
#    - Use the setup script template
#    - Enable internet with domain allowlist
#    - Save configuration

# 4. Test with a simple request like:
#    "Update the default system prompt in llm_judge.py"

7. Key Principles

  1. Safety First: Always disable API keys during testing
  2. Mark API Tests: Use @pytest.mark.requires_api consistently
  3. Document Everything: AGENTS.md is your contract with Codex
  4. Internet Access: Required for modern Python development, use allowlists
  5. Test Locally: Ensure pytest -v -m "not requires_api" works before using Codex

Save this guide and follow it for all future Python projects with Codex!