Server Metadata Management - FeitianTech/postquantum-webauthn-platform GitHub Wiki
Server Metadata Management
Table of Contents
- Introduction
- Caching Mechanism
- Session-Based Metadata System
- Metadata Merging Function
- Lifecycle Management
- Metadata Normalization and Validation
- Handling Official and User Metadata
- Security Considerations
Introduction
The server-side metadata management system in the WebAuthn platform provides a robust framework for handling FIDO Metadata Service (MDS) snapshots and user-uploaded metadata entries. This system enables both the use of official FIDO MDS data and custom metadata for testing purposes, with comprehensive caching, session management, and security features. The implementation supports development and testing workflows while maintaining the integrity of the authentication process.
Caching Mechanism
The system implements a multi-layer caching mechanism to optimize performance and reduce redundant processing of metadata blobs. The core caching functionality revolves around the _base_metadata_cache global variable in metadata.py, which stores parsed MetadataBlobPayload objects in memory to avoid repeated JSON parsing and deserialization.
The caching system follows a validation pattern based on file modification time (_base_metadata_mtime). When _load_base_metadata() is called, it first checks if the cached metadata is still valid by comparing the current modification time of MDS_METADATA_VERIFIED_PATH with the stored _base_metadata_mtime. If the file hasn't changed, the cached version is returned immediately, eliminating the need for disk I/O and JSON parsing.
For persistent caching across server restarts, the system maintains a cache entry file (MDS_METADATA_CACHE_PATH) that stores HTTP headers like last_modified and etag. This allows the system to quickly determine if a remote metadata update is available without downloading the entire payload. The load_metadata_cache_entry() and store_metadata_cache_entry() functions handle serialization and deserialization of this cache state.
Section sources
Session-Based Metadata System
The session-based metadata system allows users to upload and manage custom metadata entries through SessionMetadataItem objects. This system is centered around the SessionMetadataItem dataclass, which encapsulates metadata entries with additional metadata such as filename, upload timestamp, and original filename.
Each user session is identified by a unique session ID generated via ensure_metadata_session_id(), which creates a cryptographically secure token using secrets.token_urlsafe(32). The session ID is stored in both the Flask session and a persistent cookie (_SESSION_METADATA_COOKIE_NAME) with a one-year expiration period. This dual storage mechanism ensures session persistence across browser sessions while maintaining server-side session state.
User-uploaded metadata files are stored in a session-specific directory structure, with each file receiving a UUID-based filename to prevent naming conflicts. Associated metadata about the upload (original filename, upload timestamp) is stored in a companion .meta.json file. The list_session_metadata_items() function retrieves all metadata items for the current session, sorting them by modification time in descending order.
Section sources
Metadata Merging Function
The merge_metadata function combines user-uploaded metadata with the base MDS snapshot while preventing AAGUID conflicts. This function takes the base metadata (if available) and a list of SessionMetadataItem objects, then creates a combined metadata payload where user-provided entries take precedence over base entries with the same AAGUID.
The merging process uses the _extract_entry_aaguid() helper function to normalize AAGUID values by removing hyphens and converting to lowercase, ensuring consistent comparison. A seen_aaguids set tracks AAGUIDs encountered during processing to prevent duplicates. When a conflict is detected, the user-uploaded entry is included in the result while the base entry with the same AAGUID is excluded.
The merge order prioritizes custom entries by placing them before base entries in the combined entries list. This ordering ensures that when the metadata verifier processes entries, it will encounter and use the user-provided version before reaching any conflicting base entries. The function also handles legal header inheritance, using the first non-empty legal header from user-provided entries if the base metadata lacks one.
flowchart TD
Start([Start Merge]) --> CheckCustom["Process Custom Entries"]
CheckCustom --> InitSets["Initialize seen_aaguids set"]
InitSets --> ProcessCustom["For each session item entry"]
ProcessCustom --> ExtractAAGUID["Extract and normalize AAGUID"]
ExtractAAGUID --> CheckConflict{"AAGUID in seen_aaguids?"}
CheckConflict --> |No| AddCustom["Add to custom_entries<br>and seen_aaguids"]
AddCustom --> NextCustom["Next entry"]
NextCustom --> ProcessCustom
CheckConflict --> |Yes| SkipCustom["Skip entry"]
SkipCustom --> NextCustom
ProcessCustom --> CheckBase["Process Base Entries"]
CheckBase --> ProcessBase["For each base entry"]
ProcessBase --> ExtractBaseAAGUID["Extract and normalize AAGUID"]
ExtractBaseAAGUID --> CheckBaseConflict{"AAGUID in seen_aaguids?"}
CheckBaseConflict --> |No| AddBase["Add to base_entries"]
AddBase --> NextBase["Next entry"]
NextBase --> ProcessBase
CheckBaseConflict --> |Yes| SkipBase["Skip entry"]
SkipBase --> NextBase
ProcessBase --> Combine["Combine custom + base entries"]
Combine --> HandleLegal["Set legal header if needed"]
HandleLegal --> Return["Return merged MetadataBlobPayload"]
Return --> End([End])
Diagram sources
Lifecycle Management
The system implements comprehensive lifecycle management for metadata sessions, including automatic cleanup of inactive sessions and cookie-based session persistence. The cleanup process is governed by _maybe_cleanup_inactive_sessions(), which removes sessions that haven't been accessed for more than 14 days (_SESSION_METADATA_INACTIVE_AGE).
Cleanup operations are rate-limited by _SESSION_METADATA_CLEANUP_INTERVAL (6 hours) to prevent excessive filesystem operations. The cleanup process runs automatically whenever a session operation occurs (like listing or saving metadata items), ensuring that stale sessions are removed without requiring a separate background task.
Session persistence is implemented through a combination of Flask sessions and HTTP cookies. The _schedule_session_cookie() function sets a persistent cookie with appropriate security attributes: httponly to prevent JavaScript access, secure when served over HTTPS, and samesite set to "None" for cross-site requests when secure, otherwise "Lax". The _note_session_activity() function updates the last access time for a session whenever any metadata operation occurs, preventing premature cleanup.
stateDiagram-v2
[*] --> Active
Active --> Inactive : No activity for 14 days
Inactive --> Cleaned : Cleanup process
Cleaned --> [*]
Active --> Active : Activity detected
Inactive --> Active : Session reactivated
Active --> Cleaned : Manual deletion
Diagram sources
Metadata Normalization and Validation
The system provides robust normalization and validation functions to ensure incoming metadata JSON is properly formatted and secure. The expand_metadata_entry_payloads() function normalizes input by converting both single metadata entry objects and full MDS blob structures into a consistent list of individual entry payloads. This function validates that each entry is a valid JSON object and raises appropriate errors for malformed input.
The build_metadata_entry_components() function performs comprehensive validation and normalization of metadata entries. It ensures required fields have default values through _METADATA_STATEMENT_REQUIRED_DEFAULTS, normalizes arrays like statusReports, and validates data types. The function uses _clone_json_value() to create safe copies of JSON data, preventing potential mutation issues.
Input validation includes strict filename validation via _validate_session_metadata_filename(), which checks for path separators, leading dots, and proper file extensions. The system also validates AAGUIDs and other identifiers, normalizing them to a consistent format. Legal headers are stripped of whitespace, and metadata statements are validated to ensure they contain essential fields with appropriate data types.
flowchart TD
Start([Input JSON]) --> ValidateObject{"Is object?"}
ValidateObject --> |No| Error["Raise TypeError"]
ValidateObject --> |Yes| CheckEntries["Check for entries array"]
CheckEntries --> HasEntries{"Has entries array?"}
HasEntries --> |Yes| ProcessEntries["Process each entry"]
ProcessEntries --> ValidateEntry{"Is valid object?"}
ValidateEntry --> |No| EntryError["Raise ValueError"]
ValidateEntry --> |Yes| CloneEntry["Clone entry safely"]
CloneEntry --> ApplyLegal["Apply legal header if needed"]
ApplyLegal --> AddToExpanded["Add to expanded list"]
AddToExpanded --> NextEntry["Next entry"]
NextEntry --> ProcessEntries
HasEntries --> |No| SingleEntry["Treat as single entry"]
SingleEntry --> WrapAsList["Wrap in list"]
WrapAsList --> ApplyLegal
AddToExpanded --> Return["Return expanded list"]
Return --> End([Validated Output])
Diagram sources
Handling Official and User Metadata
The system handles both official FIDO MDS snapshots and user-provided metadata through a layered approach. Official metadata is loaded from a pre-verified snapshot file (MDS_METADATA_VERIFIED_PATH) that is updated through a CI/CD process rather than runtime downloads. The download_metadata_blob() function is explicitly disabled at runtime, raising a RuntimeError to prevent direct downloads, ensuring that only vetted metadata is used in production.
User-provided metadata is handled through the session system, allowing developers and testers to upload custom metadata for testing purposes. The maybe_store_uploaded_metadata_file() function provides an optional integration with GitHub to store uploaded metadata files in a repository, using content hashing to avoid duplicates. This allows teams to share test metadata while maintaining version control.
When processing authentication requests, the system combines both sources through the get_mds_verifier() function, which creates a unified attestation verifier that includes both base and session metadata. The metadata_entry_trust_anchor_status() function helps distinguish between trust-anchored (official) metadata and user-provided entries, allowing the system to apply appropriate trust levels to different metadata sources.
Section sources
Security Considerations
The system addresses several critical security considerations around input validation, session isolation, and secure storage of metadata files. Input validation is comprehensive, with multiple layers of checks on uploaded metadata JSON, filenames, and session identifiers. The system uses strict whitelisting for filename characters and prohibits path traversal attempts by validating that filenames don't contain directory separators.
Session isolation is enforced through unique session IDs and separate storage directories for each session. The system ensures that one user cannot access another user's metadata by validating the session ID against the current request context. Cookie security is prioritized with httponly, secure, and appropriate samesite attributes to prevent XSS and CSRF attacks.
Metadata files are stored with UUID-based filenames to prevent predictable file names and potential overwrites. The system implements proper access controls through the session metadata store, which validates session IDs before allowing file operations. Sensitive operations like file deletion include proper error handling to prevent information leakage through error messages.
The system also includes security-focused logging that records metadata upload attempts and failures without exposing sensitive content. The GitHub integration includes content hashing to detect and prevent duplicate uploads, reducing storage overhead and potential abuse.
Section sources