02 installation - the-omics-os/lobster-local GitHub Wiki

Comprehensive Installation Guide

This guide covers all installation methods for Lobster AI, from quick setup to advanced development configurations.

⚠️ Breaking Changes in v0.4.0: Lobster now requires explicit provider configuration via lobster init. Auto-detection from .env files has been removed.

Existing users: Run lobster init to create the new provider_config.json file. Your API keys in .env will continue to work.

Prerequisites
Installation Methods
Platform-Specific Instructions
Verification
Uninstalling Lobster AI
Development Installation
Optional Dependencies
Docker Deployment
Troubleshooting

Prerequisites

System Requirements

Minimum Requirements:

Python: 3.11+ (3.12+ recommended)
Memory: 4GB RAM (8GB+ recommended for large datasets)
Storage: 2GB free space (more for data analysis)
Network: Internet connection for API access and data downloads

Recommended Setup:

Python: 3.12+
Memory: 16GB+ RAM
Storage: 10GB+ free space
CPU: Multi-core processor for parallel analysis

Package Manager Recommendations

Lobster AI automatically detects and uses the best available package manager:

uv (fastest, recommended): Install uv
pip3 (macOS default)
pip (fallback)

We recommend installing uv for significantly faster package installation and dependency resolution.

Required API Keys

Easy Setup: Lobster requires explicit provider configuration (v0.4.0+). Run the interactive setup wizard to configure:

lobster init

The wizard creates two files:

provider_config.json - Provider/model selection (safe to commit to git)
.env - API keys and secrets (never commit to git)

Choose ONE of the following LLM providers:

Anthropic API Key (Recommended for most users)

⚠️ Important: Rate Limits - Anthropic applies conservative rate limits to new accounts. For production use or heavy workloads, we recommend AWS Bedrock. If you encounter rate limit errors, see Troubleshooting Guide.
- Visit Anthropic Console
- Create account and generate API key
- The wizard will prompt you to enter: ANTHROPIC_API_KEY=sk-ant-...
- Recommended for: Quick testing, development with small datasets
- Not recommended for: Production deployments, large-scale analysis
AWS Bedrock Access (Production/Enterprise)

✅ Best for production - AWS Bedrock provides enterprise-grade rate limits and reliability. Recommended for heavy workloads and production deployments.
- AWS account with Bedrock access
- Create IAM user with Bedrock permissions
- The wizard will prompt you to enter:
```
AWS_BEDROCK_ACCESS_KEY=...
AWS_BEDROCK_SECRET_ACCESS_KEY=...
```
- Recommended for: Production deployments, large-scale analysis, enterprise use
- Benefits: Higher rate limits, better reliability, enterprise SLA
Ollama (Local, Zero Cost)

🏠 Privacy-first - Run models locally on your hardware. No API keys, no internet required after model download.
- Install Ollama: https://ollama.com/
- Pull a model: ollama pull llama3:70b-instruct
- The wizard will auto-detect Ollama and configure it
- Recommended for: Privacy-sensitive data, offline work, unlimited usage
- Benefits: Zero cost, 100% local, no rate limits, offline capable
NCBI API Key (Optional)
- Visit NCBI E-utilities
- Enhances literature search capabilities
- The wizard offers to add this optionally

Advanced Users: You can skip the wizard by manually creating both config files (see Manual Configuration section below).

Pre-Installation Check

Before installing Lobster AI, run the system checker to verify your environment:

# Clone repository
git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local

# Run system checker
python3 check-system.py

The checker will:

✅ Verify Python 3.11+ is installed
✅ Detect missing system dependencies (Linux)
✅ Check for Docker availability
✅ Recommend best installation method for your platform
✅ Provide installation commands for missing dependencies

Platform-Specific Recommendations:

macOS: Native installation (Make-based)
Linux: Native with system dependencies
Windows: Docker Desktop (most reliable)

Installation Methods

Lobster AI can be installed in two ways:

Global Installation - Command available system-wide
Local Installation - Command available in virtual environment only

Comparison: Global vs Local

Aspect	Global Installation	Local Installation
Access	From anywhere in terminal	Only in activated venv
Command	`uv tool install lobster-ai`	`pip install lobster-ai`
Isolation	Separate per tool	Separate per project
Best For	CLI usage, quick analysis	Development, project-specific
Uninstall	`uv tool uninstall lobster-ai`	`pip uninstall lobster-ai`

Choose based on your workflow:

CLI power user? → Global installation
Multiple projects with different versions? → Local installation
Developer contributing to Lobster? → Local from source

Method 1: Quick Install (Recommended)

The easiest and most reliable installation method:

# Clone repository
git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local

# One-command installation
make install

What this does:

Verifies Python 3.11+ installation
Creates virtual environment at .venv
Installs all dependencies automatically
Sets up configuration files
Provides activation instructions

Method 2: Development Install

For contributors and developers:

git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local

# Install with development dependencies
make dev-install

Additional features:

Testing framework (pytest, coverage)
Code quality tools (black, isort, pylint, mypy)
Pre-commit hooks
Documentation tools

Method 3: Manual Installation

For full control over the installation process:

git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local
# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install uv (recommended for faster installation)
# https://docs.astral.sh/uv/getting-started/installation/

# Install Lobster AI with uv (recommended)
uv pip install -e .

# Alternative: pip install -e .

Method 4: Global Installation

Install the lobster command globally (Unix/macOS):

# First, install locally
make install

# Then install globally
make install-global

This creates a symlink in /usr/local/bin/lobster allowing you to run lobster from anywhere.

Method 5: PyPI Installation (Recommended)

The lobster-ai package is available on PyPI with two installation modes:

5A. Global Installation with uv tool (Recommended for CLI Use)

Best for: System-wide access, quick analysis, CLI usage

# Install uv if not already installed
# https://docs.astral.sh/uv/getting-started/installation/

# Install Lobster globally
uv tool install lobster-ai

# Verify installation
lobster --version
which lobster  # Should show: ~/.local/bin/lobster

# Configure
lobster init

# Start using
lobster chat

Benefits:

✅ Accessible from any directory
✅ Clean uninstall (one command)
✅ Isolated environment per tool
✅ Automatic PATH management
✅ No virtual environment needed

Note: Ensure ~/.local/bin is in your PATH:

# Check PATH
echo $PATH | grep -q ".local/bin" && echo "✅ In PATH" || echo "❌ Not in PATH"

# Add to PATH if needed (add to ~/.zshrc or ~/.bashrc)
export PATH="$HOME/.local/bin:$PATH"

5B. Local Installation with Virtual Environment

Best for: Project-specific installation, multiple versions, development

# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install with uv (faster)
uv pip install lobster-ai

# Or use pip
pip install lobster-ai

# Verify installation (only works when venv activated)
lobster --version

# Configure
lobster init

# Start using
lobster chat

Benefits:

✅ Project-specific versions
✅ Doesn't affect system Python
✅ Easy to remove (delete directory)
✅ Standard Python workflow

To use later:

# Always activate virtual environment first
source .venv/bin/activate
lobster chat

5C. Alternative: pipx (Similar to uv tool)

For users who already use pipx:

# Install pipx if not already installed
# https://pipx.pypa.io/stable/installation/

# Install Lobster globally with pipx
pipx install lobster-ai

# Uninstall
pipx uninstall lobster-ai

Note: pipx and uv tool are similar - choose whichever you prefer.

Configuration (All Methods):

After installation, run the configuration wizard to set up your API keys:

# Launch interactive configuration wizard
lobster init

# The wizard will guide you through:
# 1. Choose LLM provider (Anthropic, AWS Bedrock, or Ollama)
# 2. Enter your API keys securely (input is masked)
# 3. Optionally add NCBI API key for enhanced literature search
# 4. Configuration saved to TWO files:
#    - provider_config.json (provider/model selection - safe to commit)
#    - .env (API keys/secrets - never commit)

Additional configuration commands:

# Test API connectivity
lobster config test

# View current configuration (secrets masked)
lobster config show

# Reconfigure (creates backup of existing .env)
lobster init --force

# Non-interactive mode (for CI/CD)
lobster init --non-interactive --anthropic-key=sk-ant-xxx

The wizard creates two configuration files:

.lobster_workspace/provider_config.json - Provider/model selection
.env - API keys and secrets

Security Note:

✅ provider_config.json can be safely committed to git (no secrets)
❌ .env must NEVER be committed (contains API keys)

No manual file editing required!

Manual Configuration (Advanced)

If you prefer to skip the wizard, create both configuration files manually:

File 1: .lobster_workspace/provider_config.json (provider selection - safe to commit)

{
  "global_provider": "anthropic",
  "anthropic_model": "claude-sonnet-4-20250514",
  "profile": "production"
}

Available providers: anthropic, bedrock, ollama, gemini Available profiles: development, production, ultra, godmode

File 2: .env (API keys - never commit, add to .gitignore)

# Anthropic API
ANTHROPIC_API_KEY=sk-ant-api03-your-key-here

# AWS Bedrock
AWS_BEDROCK_ACCESS_KEY=your-access-key
AWS_BEDROCK_SECRET_ACCESS_KEY=your-secret-key

# Google Gemini
GOOGLE_API_KEY=your-google-api-key

# Ollama (local)
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_DEFAULT_MODEL=llama3:70b-instruct

# Optional: NCBI
NCBI_API_KEY=your-ncbi-key

Security Best Practices:

# Add .env to .gitignore
echo ".env" >> .gitignore

# Verify .env is not tracked
git check-ignore .env  # Should output: .env

# provider_config.json CAN be committed (no secrets)
git add .lobster_workspace/provider_config.json

Note: For development or contributing to Lobster AI, use Method 1 (Quick Install) or Method 3 (Manual Installation) to install from source.

Platform-Specific Instructions

macOS

Homebrew Setup (Recommended):

# Install Python 3.12+
brew install [email protected]

# Optional: Install uv for faster package management
brew install uv

git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local
make install

Issues with System Python: If using system Python causes issues:

# Use Homebrew Python explicitly
/opt/homebrew/bin/python3.12 -m venv .venv
source .venv/bin/activate

# Install with uv (recommended) or pip
uv pip install -e .
# Alternative: pip install -e .

Installation Choice:

Global: Use uv tool install lobster-ai (accessible anywhere)
Local: Use virtual environment method (project-specific)

See Installation Methods for detailed comparison.

Linux (Ubuntu/Debian)

⚠️ IMPORTANT: System Dependencies Required

Ubuntu/Debian require system libraries for compilation. Install these BEFORE running make install:

Quick Install (Recommended - All Dependencies):

# Run the automated installer
git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local
./install-ubuntu.sh

The installer script will:

Check for Python 3.12+
Detect missing system packages
Offer to install them automatically
Run make install when ready

Manual Installation:

# 1. Install ALL system dependencies (REQUIRED)
sudo apt update
sudo apt install -y \
    build-essential \
    python3.12-dev \
    python3.12-venv \
    pkg-config \
    libhdf5-dev \
    libxml2-dev \
    libxslt1-dev \
    libffi-dev \
    libssl-dev \
    libblas-dev \
    liblapack-dev \
    git

# 2. Clone and install
git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local
make install

Why These Packages Are Required:

build-essential: C/C++ compilers (gcc, g++, make)
python3.12-dev: Python header files for building extensions
libhdf5-dev: HDF5 file format support (required for AnnData)
libblas-dev, liblapack-dev: Linear algebra libraries (required for NumPy/SciPy)
libxml2-dev, libxslt1-dev: XML parsing (required for web scraping)
libffi-dev, libssl-dev: Cryptography and SSL support

Ubuntu Version Notes:

Ubuntu 22.04 LTS: Requires PPA for Python 3.12

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.12 python3.12-venv python3.12-dev

Ubuntu 24.04 LTS: Python 3.12+ available in default repositories

CentOS/RHEL/Fedora:

# Install Python 3.12+
sudo dnf install python3.12 python3.12-devel

# Install development tools and libraries
sudo dnf groupinstall "Development Tools"
sudo dnf install hdf5-devel libxml2-devel libxslt-devel \
                 openssl-devel libffi-devel blas-devel lapack-devel

# Clone and install
git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local
make install

Installation Choice:

Global: Use uv tool install lobster-ai (accessible anywhere)
Local: Use virtual environment method (project-specific)

See Installation Methods for detailed comparison.

Windows

⚠️ Native Windows installation is experimental. Docker Desktop is strongly recommended for Windows users.

Option 1: Docker Desktop (Recommended)

Docker provides the most reliable experience on Windows:

# 1. Install Docker Desktop for Windows
# Download from: https://www.docker.com/products/docker-desktop/

# 2. Clone repository
git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local

# 3. Configure API keys
copy .env.example .env
notepad .env
# Add: ANTHROPIC_API_KEY=sk-ant-api03-your-key-here

# 4. Run Lobster
docker-compose run --rm lobster-cli

Option 2: Native Installation (Experimental)

Prerequisites:

Python 3.12+ from python.org
Git for Windows from git-scm.com
(Optional) Visual Studio Build Tools if compilation errors occur

# 1. Clone repository
git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local

# 2. Run automated installer (PowerShell)
.\install.ps1

# If execution policy blocks the script:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
.\install.ps1

# Alternative: Batch file
install.bat

# 3. Configure API keys
notepad .env

# 4. Activate and run
.\.venv\Scripts\Activate.ps1
lobster chat

Option 3: Windows Subsystem for Linux (WSL)

WSL 2 provides a full Linux environment with excellent performance:

# 1. Install WSL 2
wsl --install

# 2. Install Ubuntu from Microsoft Store

# 3. Open Ubuntu terminal and follow Linux installation instructions
git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local
./install-ubuntu.sh

Common Windows Issues:

Compiler errors: Install Visual Studio Build Tools or use Docker
Permission denied: Run PowerShell as Administrator or use Docker
Python not found: Reinstall Python with "Add to PATH" checked
Long path errors: Enable long path support in Windows Registry or use Docker

Installation Choice:

Global: Use uv tool install lobster-ai (accessible anywhere)
Local: Use virtual environment method (project-specific)

See Installation Methods for detailed comparison.

Python Version Considerations

Python 3.12+ Requirements:

pyproject.toml specifies: >=3.12
Makefile enforces: >=3.12
Recommendation: Use Python 3.12+ for best performance

Installing Specific Python Version:

# macOS with pyenv
brew install pyenv
pyenv install 3.12.0
pyenv local 3.12.0

# Ubuntu with deadsnakes PPA
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt install python3.12 python3.12-venv

Verification

Test Installation

After installation, verify everything works:

# Activate environment
source .venv/bin/activate  # Linux/macOS
# .venv\Scripts\activate   # Windows

# Test CLI
lobster --help

# Test imports
python -c "import lobster; print('✅ Lobster imported successfully')"

# Run verification script
python verify_installation.py

Configuration Wizard

After installation, run the configuration wizard to set up your API keys:

# Launch interactive configuration wizard
lobster init

Expected wizard output:

╭────────────────────────────────────────────────────────────╮
│  🦞 Welcome to Lobster AI!                                 │
│                                                            │
│  Let's set up your LLM provider.                          │
╰────────────────────────────────────────────────────────────╯

Select your LLM provider:
  1 - Anthropic API - Best quality, quick setup
  2 - AWS Bedrock - Production, enterprise use
  3 - Ollama (Local) - Privacy, zero cost, offline

Choose provider [1]: 1

🔑 Anthropic API Configuration
Get your API key from: https://console.anthropic.com/

Enter your API key: ********************************

📚 NCBI API Key (Optional)
Enhances literature search capabilities.

Add NCBI API key? [y/N]: n

╭────────────────────────────────────────────────────────────╮
│  ✅ Configuration saved!                                   │
│                                                            │
│  Files created:                                            │
│  • .env (API keys - never commit to git)                  │
│  • .lobster_workspace/provider_config.json (versioned)    │
│                                                            │
│  Next step: Run lobster chat                              │
╰────────────────────────────────────────────────────────────╯

Configuration management commands:

# Test API connectivity
lobster config test

# View current configuration (secrets masked)
lobster config show

# Reconfigure (creates timestamped backup of existing .env)
lobster init --force

Check System Status

# After configuration, check status in chat
lobster chat

# In the chat interface, type:
/status

Expected output:

✅ System Status: Healthy
✅ Environment: Virtual environment active
✅ Dependencies: All packages installed
✅ Configuration: .env file present
✅ API Keys: Configured

Workspace Location

Lobster stores downloaded datasets, analysis results, plots, and exports in a workspace directory. By default, this is .lobster_workspace/ in your current working directory.

Default behavior:

# Running lobster creates workspace in current directory
cd /path/to/my/project
lobster chat
# Creates: /path/to/my/project/.lobster_workspace/

Override options:

CLI flag (highest priority):

lobster chat --workspace /custom/path/to/workspace
lobster query "analyze my data" --workspace ~/my_lobster_workspace

Environment variable:

# Add to ~/.bashrc or ~/.zshrc for persistence
export LOBSTER_WORKSPACE=/path/to/shared/workspace

# Or set per-session
LOBSTER_WORKSPACE=/tmp/test_workspace lobster chat

Resolution order: CLI --workspace flag > LOBSTER_WORKSPACE env var > current directory default

Workspace structure:

.lobster_workspace/
├── data/           # Downloaded and processed datasets (.h5ad files)
├── exports/        # Exported notebooks and reports
├── plots/          # Generated visualizations
├── cache/          # Temporary cache files
└── .session.json   # Session state

Best practices:

Use the same workspace directory across related analyses for data sharing
Set LOBSTER_WORKSPACE for team environments or CI/CD pipelines
Keep workspaces in version-controlled project directories for reproducibility

Verify API Connectivity

# Test API connectivity and validate configuration
lobster config test

This command will:

Check for .env file existence
Test LLM provider (Claude API or AWS Bedrock) connectivity
Test NCBI API if configured
Display detailed test results with ✅/❌ status for each service

Uninstalling Lobster AI

Remove Package

The uninstall process depends on how you installed Lobster:

Global Installation (uv tool)

# One command to remove everything
uv tool uninstall lobster-ai

# Verify removal
uv tool list | grep lobster  # Should output nothing
which lobster  # Should output nothing

✅ Clean: No leftover files, no manual cleanup needed.

Global Installation (pipx)

# One command to remove everything
pipx uninstall lobster-ai

# Verify removal
pipx list | grep lobster  # Should output nothing
which lobster  # Should output nothing

Local Installation (Virtual Environment)

# Activate virtual environment first
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Uninstall package
pip uninstall lobster-ai

# Deactivate virtual environment
deactivate

# Optional: Remove entire virtual environment
rm -rf .venv

Source Installation (make install-global)

cd /path/to/lobster

# Remove global symlink
make uninstall-global

# Optional: Remove virtual environment
make uninstall

Remove User Data (Optional)

⚠️ Warning: This permanently deletes all your analysis data!

Lobster stores user data separately from the package:

# View what will be removed
du -sh ~/.lobster ~/.lobster_workspace

# Remove command history, notebooks, workspaces
rm -rf ~/.lobster

# Remove cache, data, exports
rm -rf ~/.lobster_workspace

# Remove project configurations (in your project directories)
cd /path/to/your/project
rm .env  # API key configuration

What gets removed:

~/.lobster/: Command history, exported notebooks, saved workspaces
~/.lobster_workspace/: Download cache, intermediate data, exports
.env files: API key configurations (project-specific)

Keep user data if:

You plan to reinstall Lobster later
You want to preserve analysis history
You have important exported notebooks

Verify Complete Removal

# Check command removed
which lobster  # Should output: lobster not found

# Check package removed
pip list | grep lobster  # Should output nothing
uv tool list | grep lobster  # Should output nothing (if using uv tool)

# Check user data removed
ls ~/.lobster 2>/dev/null || echo "✅ Removed"
ls ~/.lobster_workspace 2>/dev/null || echo "✅ Removed"

Troubleshooting Uninstall

Problem: which lobster still shows a command after uninstall

Solution:

# Find the leftover executable
which lobster
ls -la $(which lobster)

# Remove manually (location varies by installation)
# For uv tool (shouldn't happen, but just in case)
rm ~/.local/bin/lobster

# For Homebrew Python
sudo rm /opt/homebrew/bin/lobster

# For system Python
sudo rm /usr/local/bin/lobster
# or
rm ~/Library/Python/3.x/bin/lobster

Problem: Package still shows in pip list after uninstall

Solution:

# Force remove
pip uninstall -y lobster-ai

# Or use uv
uv pip uninstall lobster-ai

Development Installation

Full Development Setup

git clone https://github.com/the-omics-os/lobster-local.git
cd lobster-local

# Development installation
make dev-install

# This installs:
# - All runtime dependencies
# - Testing framework (pytest, pytest-cov, pytest-xdist)
# - Code quality (black, isort, flake8, pylint, mypy)
# - Security tools (bandit)
# - Documentation (mkdocs)
# - Pre-commit hooks

Development Commands

# Run tests
make test

# Fast parallel testing
make test-fast

# Code formatting
make format

# Linting
make lint

# Type checking
make type-check

# Clean installation
make clean-install

Pre-commit Hooks

Development installation automatically sets up pre-commit hooks:

# Manual setup if needed
make setup-pre-commit

# Run on all files
pre-commit run --all-files

Optional Dependencies

These optional components enhance Lobster AI with advanced features. Install based on your analysis needs.

PyMOL (Protein Structure Visualization)

PyMOL enables 3D protein structure visualization and analysis (v0.2+).

Automated Installation (macOS):

cd lobster
make install-pymol

Manual Installation:

macOS

# Via Homebrew
brew install brewsci/bio/pymol

# Verify installation
pymol -c -Q

Linux (Ubuntu/Debian)

# Via apt
sudo apt-get update
sudo apt-get install pymol

# Verify installation
which pymol
pymol -c -Q

Linux (Fedora/RHEL)

# Via DNF
sudo dnf install pymol

# Verify installation
pymol -c -Q

Docker

PyMOL is pre-installed in the Docker image - no additional setup needed.

Usage:

# In Lobster chat
🦞 You: "Fetch protein structure 1AKE"
🦞 You: "Visualize 1AKE with PyMOL mode=interactive style=cartoon"
🦞 You: "Link protein structures to my RNA-seq data"

Troubleshooting: If PyMOL is not found, check installation:

which pymol
pymol --version

See Protein Structure Visualization Guide for complete usage details.

Docling (Advanced PDF Parsing)

Docling provides professional-grade PDF parsing for extracting methods from scientific publications (v0.2+).

Installation:

# Basic Docling
pip install docling

# Full installation with all features
pip install "docling[all]"

# With table extraction
pip install "docling[table]"

# With OCR support
pip install "docling[ocr]"

Verify Installation:

python -c "from docling.document_converter import DocumentConverter; print('✓ Docling installed')"

Benefits:

>90% Methods section detection (vs 30% with PyPDF2 fallback)
Table extraction from scientific papers
Formula recognition in publications
Better structure detection for complex PDFs

Usage:

# Docling is used automatically by ContentAccessService
🦞 You: "Extract methods from PMID:38448586"
🦞 You: "Read full publication PMID:35042229"

Fallback Behavior: If Docling is not installed, Lobster automatically falls back to PyPDF2 with reduced functionality.

Troubleshooting:

# Test Docling functionality
python -c "import docling; print(docling.__version__)"

# Check dependencies
pip list | grep docling

See Publication Intelligence Guide for technical details.

AWS Bedrock (Enhanced Setup)

Detailed AWS Bedrock configuration for production deployments.

Step 1: Create AWS Account

Visit AWS Console
Create account or sign in
Navigate to AWS Bedrock service

Step 2: Request Model Access

# Navigate to: AWS Bedrock → Model Access
# Request access to: Claude 3.5 Sonnet, Claude 3 Opus
# Approval typically takes 1-2 business days

Step 3: Create IAM User

# AWS Console → IAM → Users → Create User
# User name: lobster-ai-user
# Access type: Programmatic access

# Attach policy: AmazonBedrockFullAccess
# OR create custom policy (recommended):

Custom IAM Policy (Least Privilege):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:ListFoundationModels"
      ],
      "Resource": "*"
    }
  ]
}

Step 4: Configure Credentials

# Option 1: AWS CLI configuration (recommended)
aws configure
# Enter: Access Key ID, Secret Access Key, Region (us-east-1), Output format (json)

# Option 2: Environment variables
export AWS_BEDROCK_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
export AWS_BEDROCK_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_DEFAULT_REGION=us-east-1

# Option 3: .env file (for Lobster)
cat >> .env << EOF
AWS_BEDROCK_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
AWS_BEDROCK_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
AWS_DEFAULT_REGION=us-east-1
EOF

Step 5: Verify Access

# Test Bedrock connectivity
aws bedrock list-foundation-models --region us-east-1

# Test in Lobster
lobster chat
> /status
# Should show: "Model: AWS Bedrock (Claude)"

Troubleshooting AWS Bedrock:

# Check credentials
aws sts get-caller-identity

# Test model access
aws bedrock list-foundation-models --region us-east-1 | grep Claude

# Common issues:
# 1. Model access not approved → Wait for approval or request again
# 2. Wrong region → Bedrock availability varies by region
# 3. IAM permissions → Verify user has bedrock:InvokeModel permission

Regional Availability: AWS Bedrock Claude models are available in:

us-east-1 (US East, N. Virginia) - Recommended
us-west-2 (US West, Oregon)
eu-west-1 (Europe, Ireland)
ap-southeast-1 (Asia Pacific, Singapore)

See AWS Bedrock Regions for current availability.

Cloud Mode Configuration

Enable cloud processing for large-scale analyses (v0.2+).

Setup:

# 1. Request cloud API key
# Email: [email protected]
# Subject: "Lobster Cloud API Key Request"
# Include: Organization name, use case, expected usage

# 2. Configure API key
export LOBSTER_CLOUD_KEY="your-cloud-api-key-here"

# 3. Start Lobster in cloud mode
lobster chat

# 4. Verify cloud mode active
> /status
# Should show: "Cloud mode: active"

Benefits:

Scalable compute for datasets >100K cells
No local memory limits for large datasets
Faster processing with distributed infrastructure
Automatic resource management

Usage:

# Cloud mode is automatic when LOBSTER_CLOUD_KEY is set
🦞 You: "Download GSE123456 and analyze with cloud resources"
🦞 You: "Process this large dataset using cloud infrastructure"

# Switch back to local mode
unset LOBSTER_CLOUD_KEY
lobster chat

Cost Structure:

Free tier: 10 analyses/month
Pro tier: $6K-$18K/year (based on usage)
Enterprise: Custom pricing

Troubleshooting Cloud Mode:

# Check API key is set
echo $LOBSTER_CLOUD_KEY

# Test cloud connectivity
lobster chat
> /status

# Common issues:
# 1. API key not set → Export LOBSTER_CLOUD_KEY
# 2. Key expired → Request new key from [email protected]
# 3. Network timeout → Check firewall/proxy settings

See Configuration Guide for complete cloud setup details.

Docker Deployment

Lobster supports Docker for both CLI and FastAPI server modes. For comprehensive deployment guides, see Docker Deployment Guide.

Quick Start with Docker

Unix/macOS/Linux (using Makefile):

# 1. Build images
make docker-build

# 2. Run CLI interactively
make docker-run-cli

# 3. Or run FastAPI server
make docker-run-server

Windows (using PowerShell):

# 1. Build CLI image
docker build -t lobster:latest -f Dockerfile .

# 2. Run CLI interactively
docker run -it --rm \
  --env-file .env \
  -v ${PWD}/data:/app/data \
  -v lobster-workspace:/app/.lobster_workspace \
  lobster:latest chat

# 3. Or run FastAPI server
docker build -t lobster:server -f Dockerfile.server .
docker run -d --name lobster-api -p 8000:8000 --env-file .env lobster:server

Build Docker Images

Unix/macOS/Linux:

# Build both CLI and server images (using Makefile)
make docker-build

# Or manually
docker build -t lobster:latest -f Dockerfile .
docker build -t lobster:server -f Dockerfile.server .

Windows (PowerShell):

# Build CLI image
docker build -t lobster:latest -f Dockerfile .

# Build server image (optional, for FastAPI mode)
docker build -t lobster:server -f Dockerfile.server .

Note for Windows users: The make command is not available by default on Windows. Use the manual docker build commands shown above.

Run CLI with Docker

Unix/macOS/Linux:

# Using Makefile (recommended)
make docker-run-cli

# Or manually with environment file
docker run -it --rm \
  --env-file .env \
  -v $(pwd)/data:/app/data \
  -v lobster-workspace:/app/.lobster_workspace \
  lobster:latest chat

# Single query mode (automation)
docker run --rm \
  --env-file .env \
  -v $(pwd)/data:/app/data \
  lobster:latest query "download GSE12345"

Windows (PowerShell):

# Interactive chat mode
docker run -it --rm `
  --env-file .env `
  -v ${PWD}/data:/app/data `
  -v lobster-workspace:/app/.lobster_workspace `
  lobster:latest chat

# Single query mode (automation)
docker run --rm `
  --env-file .env `
  -v ${PWD}/data:/app/data `
  lobster:latest query "download GSE12345"

# With individual environment variables (if .env file not available)
docker run -it --rm `
  -e ANTHROPIC_API_KEY=your-key-here `
  -v ${PWD}/data:/app/data `
  -v lobster-workspace:/app/.lobster_workspace `
  lobster:latest chat

Windows Notes:

Use backtick (`) for line continuation in PowerShell
Use ${PWD} to reference current directory
Named volumes (like lobster-workspace) work the same on all platforms

Run FastAPI Server with Docker

Unix/macOS/Linux:

# Using Makefile (recommended)
make docker-run-server

# Or manually
docker run -d \
  --name lobster-api \
  -p 8000:8000 \
  --env-file .env \
  -v $(pwd)/data:/app/data \
  lobster:server

# Check server health
curl http://localhost:8000/health

# Stop server
docker stop lobster-api

Windows (PowerShell):

# Run server in detached mode
docker run -d `
  --name lobster-api `
  -p 8000:8000 `
  --env-file .env `
  -v ${PWD}/data:/app/data `
  lobster:server

# Check server health (PowerShell)
Invoke-WebRequest -Uri http://localhost:8000/health

# Or use curl if installed
curl http://localhost:8000/health

# View server logs
docker logs lobster-api

# Stop server
docker stop lobster-api

# Remove stopped container
docker rm lobster-api

Docker Compose

Unix/macOS/Linux:

# Run CLI interactively
make docker-compose-cli

# Start FastAPI server in background
make docker-compose-up

# View logs
docker-compose logs -f lobster-server

# Stop all services
make docker-compose-down

Windows (PowerShell):

# Run CLI interactively
docker-compose run --rm lobster-cli chat

# Start FastAPI server in background
docker-compose up -d lobster-server

# View logs
docker-compose logs -f lobster-server

# Stop all services
docker-compose down

docker-compose.yml supports both CLI and server modes. See Docker Deployment Guide for full configuration details.

Note: Docker Compose works identically on Windows, macOS, and Linux. The commands are the same across platforms.

Troubleshooting

Common Installation Issues

Python Version Problems

Error: Python 3.12+ is required

Solutions:

# Check Python version
python --version
python3 --version

# Install Python 3.12+
# macOS: brew install [email protected]
# Ubuntu: sudo apt install python3.12
# Windows: Download from python.org

# Use specific Python version
python3.12 -m venv .venv

Virtual Environment Issues

Error: Failed to create virtual environment

Solutions:

# Install venv module (Ubuntu/Debian)
sudo apt install python3.12-venv

# Clear existing environment
rm -rf .venv

# Create manually
python3 -m venv .venv --clear

# Alternative method
python3 -m venv .venv --without-pip
source .venv/bin/activate
curl https://bootstrap.pypa.io/get-pip.py | python

Dependency Installation Failures

Error: Failed building wheel for [package]

Solutions:

# Install development headers (Linux)
sudo apt install python3.12-dev build-essential

# Update pip and setuptools
pip install --upgrade pip setuptools wheel

# Clear pip cache
pip cache purge

# Install with no cache
pip install --no-cache-dir -e .

# Use uv for faster, more reliable installs
pip install uv
uv pip install -e .

Permission Errors

Error: Permission denied

Solutions:

# Don't use sudo with pip in virtual environment
# Instead, ensure virtual environment ownership
chown -R $USER:$USER .venv

# For global installation (Unix only)
sudo make install-global

Memory Issues During Installation

Error: Killed or memory-related errors

Solutions:

# Increase swap space (Linux)
sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

# Install with limited parallelism
pip install -e . --no-build-isolation

# Use development profile for lighter resource usage
export LOBSTER_PROFILE=development
make install

Runtime Issues

API Key Problems

Error: API key not found or invalid

Solutions:

# Check environment variables
echo $ANTHROPIC_API_KEY    # For Claude API
echo $AWS_BEDROCK_ACCESS_KEY  # For AWS Bedrock
source .env  # Load from file

# Test API connectivity
lobster config test

# Regenerate API keys if needed

Import Errors

Error: ModuleNotFoundError: No module named 'lobster'

Solutions:

# Ensure virtual environment is activated
source .venv/bin/activate

# Reinstall in development mode
pip install -e .

# Check PYTHONPATH
python -c "import sys; print(sys.path)"

Memory Issues During Analysis

Solutions:

# Use development profile (lighter than production)
export LOBSTER_PROFILE=development

# Reduce file size limits
export LOBSTER_MAX_FILE_SIZE_MB=100

# Monitor memory usage
htop  # Linux/macOS
# Task Manager on Windows

Getting Additional Help

Check System Health

lobster chat
/dashboard  # Comprehensive system overview
/status     # Quick status check

Enable Debug Mode

# Verbose logging
lobster chat --debug --verbose

# Show reasoning
lobster chat --reasoning

Log Files

# Check logs in workspace
ls .lobster_workspace/logs/

# Enable detailed logging
export LOBSTER_LOG_LEVEL=DEBUG

Community Support

GitHub Issues: Report bugs
Discord: Join community
Email: Direct support
Documentation: Full docs

Clean Reinstallation

If all else fails, perform a clean reinstallation:

# Remove everything
make uninstall      # Remove virtual environment
make clean          # Remove build artifacts
rm -rf .lobster_workspace  # Remove workspace (optional)

# Fresh installation
make clean-install

# Or manually
rm -rf .venv
git clean -fdx  # Warning: removes all untracked files
make install

Next Steps: Once installation is complete, see the Configuration Guide to set up API keys and customize your Lobster AI environment.

02 installation - the-omics-os/lobster-local GitHub Wiki

Comprehensive Installation Guide

Table of Contents

Prerequisites

System Requirements

Package Manager Recommendations

Required API Keys

Pre-Installation Check

Installation Methods

Comparison: Global vs Local

Method 1: Quick Install (Recommended)

Method 2: Development Install

Method 3: Manual Installation

Method 4: Global Installation

Method 5: PyPI Installation (Recommended)

5A. Global Installation with uv tool (Recommended for CLI Use)

5B. Local Installation with Virtual Environment

5C. Alternative: pipx (Similar to uv tool)

Manual Configuration (Advanced)

Platform-Specific Instructions

macOS

Linux (Ubuntu/Debian)

Windows

Python Version Considerations

Verification

Test Installation

Configuration Wizard

Check System Status

Workspace Location

Verify API Connectivity

Uninstalling Lobster AI

Remove Package

Global Installation (uv tool)

Global Installation (pipx)

Local Installation (Virtual Environment)

Source Installation (make install-global)

Remove User Data (Optional)

Verify Complete Removal

Troubleshooting Uninstall

Development Installation

Full Development Setup

Development Commands

Pre-commit Hooks

Optional Dependencies

PyMOL (Protein Structure Visualization)

macOS

Linux (Ubuntu/Debian)

Linux (Fedora/RHEL)

Docker

Docling (Advanced PDF Parsing)

AWS Bedrock (Enhanced Setup)

Cloud Mode Configuration

Docker Deployment

Quick Start with Docker

Build Docker Images

Run CLI with Docker

Run FastAPI Server with Docker

Docker Compose

Troubleshooting

Common Installation Issues

Python Version Problems

Virtual Environment Issues

Dependency Installation Failures

Permission Errors

Memory Issues During Installation

Runtime Issues

API Key Problems

Import Errors

Memory Issues During Analysis

Getting Additional Help

Check System Health

Enable Debug Mode

Log Files

Community Support

Clean Reinstallation