Network Isolation - mensfeld/code-on-incus GitHub Wiki

COI provides network isolation to protect your host and private networks from container access.

Requirements: Network isolation (restricted/allowlist modes) requires firewalld to be installed and running. COI uses firewalld direct rules to filter container traffic in the FORWARD chain. If firewalld is not available, you'll need to set [network] mode = "open" in config or install and configure firewalld.

Blocks local networks, allows internet:

coi shell  # Default behavior

• Blocks: RFC1918 private networks (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16)
• Blocks: Cloud metadata endpoints (169.254.0.0/16)
• Allows: All public internet (npm, pypi, GitHub, APIs, etc.)
• IPv6 disabled inside container to prevent bypassing IPv4 firewall rules

Only specific domains allowed:

[network]
mode = "allowlist"

• Requires configuration with allowed_domains list
• TTL-aware DNS refresh — Domain IPs are re-resolved based on actual DNS TTL values, not a fixed interval. Domains with short TTLs (e.g., CDNs, cloud services) refresh more frequently, preventing allowed domains from becoming unreachable when their IPs rotate. The refresh_interval_minutes config acts as a maximum cap; the actual interval is the minimum TTL across all resolved domains (with a 60-second floor to prevent excessive queries)
• Always blocks RFC1918 private networks
• IP caching for DNS failure resilience

No restrictions (trusted projects only):

[network]
mode = "open"

# ~/.coi/config.toml
[network]
mode = "restricted"  # restricted | open | allowlist

# Allowlist mode configuration
# Supports both domain names and raw IPv4 addresses
allowed_domains = [
    "8.8.8.8",             # Google DNS (REQUIRED for DNS resolution)
    "1.1.1.1",             # Cloudflare DNS (REQUIRED for DNS resolution)
    "registry.npmjs.org",  # npm package registry
    "api.anthropic.com",   # Claude API
    "platform.claude.com", # Claude Platform
]
refresh_interval_minutes = 30  # Maximum IP refresh interval; actual interval uses DNS TTL if shorter (0 to disable)

• Gateway IP is auto-detected and excluded from RFC1918 checks — COI automatically detects your network gateway IP and exempts it from private network blocking. This prevents false-positive alerts on routine DNS/NTP traffic that routes through the Incus bridge gateway. You don't need to add it manually.
• Public DNS servers required — 8.8.8.8 and 1.1.1.1 must be in the allowlist for DNS resolution to work.
• Firewall rule ordering — COI adds ALLOW rules first (for gateway, allowed domains/IPs), then REJECT rules (for RFC1918 ranges), then a default REJECT rule for allowlist mode.
• Supports both domain names (github.com) and raw IPv4 addresses (8.8.8.8)
• Subdomains must be listed explicitly (github.com ≠ api.github.com)
• Domains behind CDNs may have many IPs that change frequently
• DNS failures use cached IPs from previous successful resolution

By default, COI allows the host machine to access services running in containers. This works by adding an allow rule for the gateway IP (which represents the host) before the RFC1918 block rules. Since firewalld evaluates rules by priority, the gateway IP is allowed while other private IPs are still blocked.

For example, if a web server runs on port 3000 in the container:

# Inside container: Puma/Rails server listening on 0.0.0.0:3000
# From host: Access via container IP
curl http://<container-ip>:3000

For development environments where you want machines on your local network to access container services (e.g., accessing containers via tmux from multiple machines), add this to your config:

[network]
allow_local_network_access = true  # Allow all RFC1918, not just gateway

⚠️ Security Note: When allow_local_network_access = true, ALL RFC1918 private network traffic is allowed (no RFC1918 blocking). Use this only in trusted development environments where you need cross-machine access.

Default behavior: Only the host (gateway IP) can access container services. Other machines on your local network cannot, even if they're on the same subnet.

Note: Firewalld rules filter traffic at the FORWARD chain level. All traffic from the container to the gateway IP is permitted to allow host-to-container communication.

If you see "Connection refused" when trying to access container services:

1. Verify container service is listening: coi container exec <name> -- netstat -tlnp
2. Check container IP: coi list (shows IPv4 for running containers)
3. Ensure firewall allows traffic to the bridge network

Network isolation (restricted/allowlist modes) requires firewalld. If you see the error "firewalld is not available", you have two options:

# ~/.coi/config.toml
[network]
mode = "open"

This disables egress filtering but allows you to work immediately.

Firewalld provides the FORWARD chain filtering needed for network isolation. Follow these steps:

# 1. Install firewalld (Ubuntu/Debian)
sudo apt install firewalld

# 2. Enable and start firewalld
sudo systemctl enable --now firewalld

# 3. Enable masquerading for container NAT
sudo firewall-cmd --permanent --add-masquerade
sudo firewall-cmd --reload

# 4. Verify firewalld is running
sudo firewall-cmd --state

# 5. Allow COI to manage firewall rules (passwordless sudo)
echo "$USER ALL=(ALL) NOPASSWD: /usr/bin/firewall-cmd" | sudo tee /etc/sudoers.d/coi-firewalld
sudo chmod 440 /etc/sudoers.d/coi-firewalld

• Firewalld must be running for network isolation to work
• COI adds direct rules to the FORWARD chain to filter container traffic
• Rules are scoped by container IP address for precise filtering
• Rules are removed when containers are stopped/deleted

• COI gets the container's IP address from Incus
• Firewalld direct rules are added with priorities (lower = evaluated first)
• Restricted mode: Allow gateway, block RFC1918, allow all else
• Allowlist mode: Allow gateway, allow specific IPs, block RFC1918, block all else

COI automatically manages firewalld resources to prevent accumulation of stale configurations:

• Zone bindings cleanup: When containers are deleted (via coi kill or coi clean), their veth interface zone bindings are automatically removed from firewalld
• Cleanup on all termination paths: Firewall rules and veth zone bindings are cleaned up during normal exit, coi shutdown, coi kill, and when the security responder auto-kills a container
• Orphaned resource detection: coi clean --orphans scans for and removes:
  • Orphaned veth interfaces (no master bridge)
  • Orphaned firewall rules (for non-existent container IPs)
  • Orphaned firewalld zone bindings (veths registered in zones but no longer exist on the system)
  • Orphaned nftables monitoring rules and chains (when nftables monitoring is enabled)

This prevents firewalld from accumulating stale interface bindings over time, which could otherwise cause configuration bloat and potential conflicts.

Manual cleanup:

# Clean up all orphaned resources (veths, firewall rules, zone bindings, nft rules)
coi clean --orphans

# Dry run to see what would be cleaned
coi clean --orphans --dry-run