1. Introduction
2. System Architecture
3. Script Workflow
4. JWT Signature Verification
5. Caching Mechanism
6. File Output Structure
7. Usage Examples
8. Operational Considerations
9. Error Handling
10. Integration Patterns

The MDS (Metadata Service) snapshot update process is a critical component of the FIDO2/WebAuthn platform that ensures the local metadata repository remains synchronized with the authoritative FIDO Alliance Metadata Service (MDS3). This automated system periodically refreshes the local copy of FIDO authenticator metadata from the official FIDO Alliance endpoint (https://mds3.fidoalliance.org/) when changes are detected, maintaining the integrity and currency of authenticator trust data.

The update process implements a sophisticated verification pipeline that validates JWT signatures using embedded trust anchors, optimizes bandwidth through intelligent caching mechanisms, and maintains strict operational guarantees for reliability and security.

The MDS snapshot update system consists of several interconnected components that work together to maintain metadata synchronization:

graph TB
subgraph "Update Process"
Script[update_mds_snapshot.py]
Fetch[Remote Fetch]
Verify[JWT Verification]
Cache[Cache Management]
end
subgraph "Local Storage"
Blob[blob.jwt]
Verified[fido-mds3.verified.json]
Meta[fido-mds3.verified.json.meta.json]
end
subgraph "External Dependencies"
FIDO[FIDO Alliance MDS3]
TrustRoot[FIDO_METADATA_TRUST_ROOT_CERT]
end
Script --> Fetch
Fetch --> Verify
Verify --> Cache
Cache --> Blob
Cache --> Verified
Cache --> Meta
Fetch --> FIDO
Verify --> TrustRoot

Diagram sources

• update_mds_snapshot.py
• mds3.py

Section sources

• update_mds_snapshot.py
• config.py

The MDS snapshot update script follows a carefully orchestrated workflow that ensures data integrity while optimizing performance:

flowchart TD
Start([Script Execution]) --> Fetch[Fetch Remote BLOB]
Fetch --> Compare{Compare with Local}
Compare --> |Different| Parse[Parse & Verify JWT]
Compare --> |Same| Skip[Skip Update]
Parse --> Extract[Extract Payload]
Extract --> WriteBlob[Write Raw BLOB]
WriteBlob --> WriteVerified[Write Verified JSON]
WriteVerified --> UpdateCache[Update Cache State]
UpdateCache --> Success[Update Complete]
Skip --> NoChange[No Changes Made]
Success --> End([Exit])
NoChange --> End

Diagram sources

• update_mds_snapshot.py

1. Remote Fetch Phase: The script initiates an HTTP request to the FIDO Alliance MDS3 endpoint with a custom user agent header for identification and rate limiting compliance.

2. Comparison Logic: The script compares the newly fetched BLOB with the existing local version to determine if an update is necessary, avoiding unnecessary file operations.

3. JWT Parsing and Verification: When changes are detected, the script parses the JWT structure, verifies its cryptographic signature using the embedded trust root certificate, and extracts the verified payload.

4. File Writing: Both the raw JWT BLOB and the parsed, verified JSON representation are written to persistent storage.

5. Cache State Update: The script updates the cache metadata with current HTTP headers and timestamps for future optimization.

Section sources

• update_mds_snapshot.py

The script implements robust JWT signature verification using the embedded FIDO Metadata Trust Root Certificate:

sequenceDiagram
participant Script as update_mds_snapshot.py
participant Parser as parse_blob()
participant Crypto as Cryptographic Verification
participant Trust as Trust Root Certificate
Script->>Parser : parse_blob(blob, trust_root)
Parser->>Parser : Split JWT (header.payload.signature)
Parser->>Crypto : Extract X.509 Chain
Crypto->>Trust : Verify Certificate Chain
Trust-->>Crypto : Validation Result
Crypto->>Crypto : Extract Public Key
Crypto->>Crypto : Verify Signature
Crypto-->>Parser : Verified Payload
Parser-->>Script : MetadataBlobPayload

Diagram sources

• update_mds_snapshot.py
• mds3.py

The JWT verification process involves several critical steps:

1. JWT Structure Analysis: The script separates the JWT into its constituent parts: header, payload, and signature.

2. Certificate Chain Extraction: The X.509 certificate chain is extracted from the JWT header's x5c field.

3. Chain Validation: The certificate chain is validated against the embedded trust root certificate using standard X.509 validation procedures.

4. Public Key Extraction: The public key is extracted from the leaf certificate in the chain.

5. Signature Verification: The cryptographic signature is verified using the extracted public key and the appropriate COSE algorithm.

Section sources

• mds3.py
• update_mds_snapshot.py

The caching system optimizes subsequent update attempts by storing HTTP response headers and timestamps:

classDiagram
class CacheEntry {
+string last_modified
+string last_modified_iso
+string etag
+string fetched_at
+validate_cache() bool
+update_timestamp() void
}
class HTTPHeaders {
+string Last-Modified
+string ETag
+datetime fetched_at
}
class ConditionalRequest {
+add_headers(request) void
+check_response(response) bool
}
CacheEntry --> HTTPHeaders : stores
CacheEntry --> ConditionalRequest : enables

Diagram sources

• update_mds_snapshot.py
• metadata.py

The cache metadata is stored in a structured JSON format that captures essential HTTP headers:

1. ETag Support: The system utilizes ETags for strong entity validation, reducing bandwidth when content hasn't changed.

2. Conditional Requests: The Last-Modified header enables conditional GET requests that return 304 Not Modified when appropriate.

3. Timestamp Tracking: The fetched_at field helps detect stale cache entries and triggers forced updates when necessary.

Section sources

• update_mds_snapshot.py
• metadata.py

The MDS update process generates three distinct files that serve different purposes in the metadata ecosystem:

graph LR
subgraph "Output Files"
Raw[blob.jwt<br/>Raw JWT BLOB]
Verified[fido-mds3.verified.json<br/>Parsed JSON Snapshot]
Cache[fido-mds3.verified.json.meta.json<br/>Cache Metadata]
end
subgraph "File Purposes"
RawPurpose[Storage for future processing]
VerifiedPurpose[Direct consumption by applications]
CachePurpose[Optimization metadata]
end
Raw --> RawPurpose
Verified --> VerifiedPurpose
Cache --> CachePurpose

Diagram sources

• update_mds_snapshot.py

• Format: Binary JWT encoded data
• Purpose: Provides the original, unprocessed metadata for downstream processing systems
• Location: server/server/static/blob.jwt
• Retention: Stored alongside verified JSON for completeness

• Format: Structured JSON with verified metadata entries
• Purpose: Direct consumption by WebAuthn applications and verification systems
• Structure: Contains metadata statements, status reports, and authenticator information
• Location: server/server/static/fido-mds3.verified.json

• Format: JSON with HTTP header and timing information
• Purpose: Enable efficient subsequent update operations
• Content: Last-Modified headers, ETags, and fetch timestamps
• Location: server/server/static/fido-mds3.verified.json.meta.json

Section sources

• update_mds_snapshot.py
• config.py

The script can be executed manually for testing or development purposes:

# Execute the update script
python scripts/update_mds_snapshot.py

# Expected output when updates are available:
# Packaged metadata snapshot refreshed.

# Expected output when no updates are needed:
# Packaged metadata is already up to date; no changes made.

The script is designed for integration into continuous integration workflows:

# GitHub Actions example
name: Update MDS Metadata
on:
  schedule:
    - cron: '0 2 * * *'  # Daily at 2 AM UTC
  workflow_dispatch:  # Allow manual triggering

jobs:
  update-metadata:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.9'
          
      - name: Update MDS snapshot
        run: python scripts/update_mds_snapshot.py
        
      - name: Commit and push changes
        if: ${{ success() && steps.update.outputs.changed }}
        run: |
          git config user.name 'github-actions'
          git config user.email '[email protected]'
          git add server/server/static/
          git commit -m "Update MDS metadata snapshot"
          git push

The script can be invoked programmatically from other Python applications:

import subprocess
import sys

def update_mds_metadata():
    """Programmatically trigger MDS metadata update."""
    try:
        result = subprocess.run(
            ['python', 'scripts/update_mds_snapshot.py'],
            capture_output=True,
            text=True,
            check=True
        )
        
        if "Packaged metadata is already up to date" in result.stdout:
            print("Metadata is current - no updates needed")
            return False
        elif "Packaged metadata snapshot refreshed" in result.stdout:
            print("Metadata successfully updated")
            return True
        else:
            print(f"Unexpected output: {result.stdout}")
            return False
            
    except subprocess.CalledProcessError as e:
        print(f"Update failed: {e.stderr}")
        return False

Section sources

• update_mds_snapshot.py

The script implements several network-related considerations for reliable operation:

• Timeout Settings: 120-second timeout for HTTP requests to accommodate potential network latency
• User Agent: Custom user agent header for proper service identification
• Error Resilience: Graceful handling of network failures with appropriate exit codes

1. Certificate Validation: Strict X.509 certificate chain validation using embedded trust roots
2. Integrity Checking: Cryptographic signature verification prevents tampering
3. Access Control: Local file system permissions protect sensitive metadata

1. Bandwidth Conservation: Conditional requests reduce unnecessary data transfer
2. Idempotent Operations: Safe to run multiple times without adverse effects
3. Minimal Processing: Efficient comparison logic avoids unnecessary parsing

The system guarantees idempotent behavior:

• Safe Repeated Execution: Running the script multiple times produces consistent results
• Atomic Operations: File writes are performed atomically to prevent corruption
• State Preservation: Cache metadata survives script executions

Section sources

• update_mds_snapshot.py
• update_mds_snapshot.py

The script implements comprehensive error handling across multiple failure scenarios:

flowchart TD
Start([Script Start]) --> TryFetch{Try Fetch}
TryFetch --> |Success| TryCompare{Try Compare}
TryFetch --> |Network Error| LogError[Log Error]
TryCompare --> |Success| TryWrite{Try Write}
TryCompare --> |File Error| LogError
TryWrite --> |Success| Success[Update Complete]
TryWrite --> |IO Error| LogError
LogError --> Exit1[Exit 1]
Success --> Exit0[Exit 0]

Diagram sources

• update_mds_snapshot.py

• Timeout: HTTP requests exceeding 120-second timeout
• Connection Failure: Network unreachable or connection refused
• DNS Resolution: Unable to resolve FIDO Alliance endpoint

• Permission Denied: Insufficient permissions to write to target directories
• Disk Space: Insufficient storage space for new files
• Path Issues: Invalid or inaccessible file paths

• Invalid JWT: Malformed or corrupted JWT structure
• Signature Verification: Failed cryptographic signature validation
• JSON Parsing: Invalid JSON structure in metadata payload

The script provides structured error reporting:

# Network failure example
print(f"::error::Failed to download metadata BLOB: {exc}")

# File system error example
except OSError:
    pass  # Silent failure for cache writing

Section sources

• update_mds_snapshot.py
• update_mds_snapshot.py

The metadata system integrates seamlessly with server startup:

sequenceDiagram
participant Server as WebAuthn Server
participant Bootstrap as Bootstrap Process
participant Metadata as Metadata Loader
participant Cache as Cache System
Server->>Bootstrap : Initialize server
Bootstrap->>Metadata : Load cached snapshot
Metadata->>Cache : Check cache validity
Cache-->>Metadata : Cache status
Metadata-->>Bootstrap : Loaded metadata
Bootstrap-->>Server : Ready for requests

Diagram sources

• metadata.py

The system provides comprehensive testing capabilities:

# Test fixture for metadata operations
@pytest.fixture
def packaged_metadata_env(monkeypatch, tmp_path):
    """Setup temporary metadata environment for testing."""
    verified_path = tmp_path / "fido-mds3.verified.json"
    cache_path = tmp_path / "fido-mds3.verified.json.meta.json"
    
    # Initialize with test data
    # ...
    
    monkeypatch.setattr(metadata_module, "MDS_METADATA_VERIFIED_PATH", str(verified_path))
    monkeypatch.setattr(metadata_module, "MDS_METADATA_CACHE_PATH", str(cache_path))

The script supports monitoring through exit codes and logging:

• Exit Code 0: Successful update or no changes needed
• Exit Code 1: Error occurred during update process
• Logging: Structured error messages for operational visibility

Section sources

• test_metadata_bootstrap.py
• update_mds_snapshot.py