Custom Metadata Management System

Table of Contents

1. Introduction
2. System Architecture
3. UI Controls and Overlay
4. Server Endpoints
5. Custom Metadata Storage
6. Metadata Merging Process
7. Caching Mechanism
8. Error Handling and Validation
9. Testing and Simulation Scenarios
10. Security Considerations
11. Troubleshooting Guide

Introduction

The Custom Metadata Management System provides users with the ability to upload, manage, and utilize custom metadata entries within the FIDO Metadata Service (MDS) overlay. This system enables testing of authenticators not present in the official MDS dataset and simulation of various certification scenarios by allowing users to inject custom metadata entries that are seamlessly merged with the base MDS dataset.

The system operates entirely within browser sessions, ensuring that custom metadata remains isolated between different users and sessions while providing a powerful tool for testing and development purposes.

System Architecture

The custom metadata management system follows a layered architecture with clear separation between frontend UI controls, backend API endpoints, and persistent storage mechanisms.

graph TB
subgraph "Frontend Layer"
UI[MDS Overlay UI]
JS[JavaScript Controllers]
Constants[MDS Constants]
end
subgraph "API Layer"
Routes[Flask Routes]
Auth[Authentication]
Validation[Input Validation]
end
subgraph "Business Logic Layer"
Metadata[Metadata Manager]
Merge[Merge Engine]
Session[Session Manager]
end
subgraph "Storage Layer"
Local[Local Storage]
Cloud[Cloud Storage]
Cache[Memory Cache]
end
UI --> JS
JS --> Constants
JS --> Routes
Routes --> Auth
Routes --> Validation
Routes --> Metadata
Metadata --> Merge
Metadata --> Session
Session --> Local
Session --> Cloud
Merge --> Cache

Diagram sources

• mds.js
• general.py
• metadata.py

Section sources

• mds.js
• general.py
• metadata.py

UI Controls and Overlay

The custom metadata management interface is implemented as an overlay panel that integrates seamlessly with the main MDS interface. The overlay provides intuitive drag-and-drop functionality, file selection capabilities, and real-time feedback for upload operations.

Overlay Structure

The overlay consists of several key components:

classDiagram
class CustomMetadataPanel {
+HTMLElement root
+HTMLElement dropzone
+HTMLElement fileInput
+HTMLElement messageContainer
+HTMLElement fileList
+open() void
+close() void
+handleDrop(event) void
+handleFileSelection(files) void
}
class DropZone {
+HTMLElement element
+handleDragEnter(event) void
+handleDragLeave(event) void
+handleDrop(event) void
}
class FileInput {
+HTMLInputElement element
+handleInputChange(event) void
+validateFiles(files) boolean
}
class MessageSystem {
+HTMLElement container
+setMessage(message, type) void
+clearMessages() void
}
CustomMetadataPanel --> DropZone
CustomMetadataPanel --> FileInput
CustomMetadataPanel --> MessageSystem

Diagram sources

• mds.js
• mds-content.html

Drag-and-Drop Functionality

The system implements comprehensive drag-and-drop support with visual feedback:

• Visual Indicators: Drop zone highlights when files are dragged over
• File Validation: Automatic filtering of non-JSON files
• Multiple Selection: Support for uploading multiple metadata files simultaneously
• Accessibility: Full keyboard navigation and screen reader support

File Upload Process

The file upload process follows these steps:

1. File Selection: Users can either drag files onto the drop zone or use the file input
2. Validation: Files are validated for JSON format and structure
3. Processing: Each valid file is processed and stored in the session
4. Feedback: Real-time progress updates and completion notifications
5. Integration: Successfully uploaded files are immediately available in the metadata list

Section sources

• mds.js
• mds-content.html

Server Endpoints

The system exposes three primary REST endpoints for managing custom metadata:

Endpoint Specifications

Endpoint	Method	Purpose	Request Format	Response Format
/api/mds/metadata/custom	GET	List custom metadata entries	None	JSON array of metadata items
/api/mds/metadata/upload	POST	Upload custom metadata files	multipart/form-data with JSON files	Success/error response with uploaded items
/api/mds/metadata/custom/{filename}	DELETE	Delete specific metadata entry	Path parameter: stored filename	Deletion confirmation

GET /api/mds/metadata/custom

Retrieves all custom metadata entries for the current session:

Request Headers:

• Accept: application/json

Response Format:

{
  "items": [
    {
      "filename": "unique-filename.json",
      "originalFilename": "user-uploaded-file.json",
      "uploadedAt": "2024-01-15T10:30:00Z",
      "entry": {
        "aaguid": "12345678-1234-1234-1234-123456789abc",
        "metadataStatement": {
          "description": "Custom authenticator description",
          "authenticatorVersion": 1,
          "attestationTypes": ["basic"]
        }
      }
    }
  ]
}

POST /api/mds/metadata/upload

Uploads one or more JSON metadata files:

Request Format:

• Content-Type: multipart/form-data
• Body: Multiple files with name "files[]"

File Requirements:

• Must be valid JSON files
• Must have .json extension
• Must contain valid metadata structure

Success Response:

{
  "items": [
    {
      "filename": "generated-uuid.json",
      "originalFilename": "authenticator.json",
      "uploadedAt": "2024-01-15T10:30:00Z"
    }
  ],
  "errors": []
}

Error Response:

{
  "items": [],
  "errors": [
    "authenticator.json: Not a valid JSON file.",
    "invalid-format.xml: Unsupported file format."
  ]
}

DELETE /api/mds/metadata/custom/{filename}

Removes a specific custom metadata entry:

Path Parameters:

• filename: The stored filename of the metadata entry to delete

Success Response:

{
  "deleted": true
}

Error Responses:

• 404: Metadata entry not found
• 400: Invalid filename or session error
• 500: Server-side processing error

Section sources

• general.py
• mds-constants.js

Custom Metadata Storage

The system implements a dual-layer storage architecture that combines local filesystem storage with cloud backup capabilities, ensuring data persistence and reliability.

Storage Architecture

sequenceDiagram
participant Client as Browser Client
participant Server as Flask Server
participant Session as Session Store
participant Local as Local Storage
participant Cloud as Cloud Storage
Client->>Server : Upload Metadata File
Server->>Server : Validate File Format
Server->>Session : Create Session ID
Server->>Local : Store File Locally
Local-->>Server : Confirmation
Server->>Cloud : Backup to Cloud (Optional)
Cloud-->>Server : Confirmation
Server-->>Client : Upload Success Response

Diagram sources

• metadata.py
• session_metadata_store.py

Session-Based Isolation

Each browser session maintains its own isolated metadata storage:

• Session Cookies: Unique session identifiers stored in cookies
• Directory Structure: Separate directories for each session
• File Naming: UUID-based filenames to prevent conflicts
• Automatic Cleanup: Inactive sessions are automatically removed after 14 days

File Organization

Custom metadata files are organized as follows:

sessions/
├── session-id-1/
│   ├── metadata-uuid-1.json
│   ├── metadata-uuid-1.meta.json
│   ├── metadata-uuid-2.json
│   └── metadata-uuid-2.meta.json
├── session-id-2/
│   └── ...

Each metadata file has a corresponding metadata file containing:

• Original filename
• Upload timestamp
• Stored filename
• Additional metadata

Section sources

• metadata.py
• session_metadata_store.py

Metadata Merging Process

The custom metadata system implements a sophisticated merging algorithm that combines custom entries with the base MDS dataset while preserving existing entries and maintaining data integrity.

Merge Algorithm

flowchart TD
Start([Begin Merge Process]) --> LoadBase["Load Base MDS Dataset"]
LoadBase --> LoadCustom["Load Custom Metadata Items"]
LoadCustom --> ExtractAAGUIDs["Extract AAGUIDs from Custom Entries"]
ExtractAAGUIDs --> FilterDuplicates["Filter Duplicate AAGUIDs"]
FilterDuplicates --> CombineEntries["Combine Custom + Base Entries"]
CombineEntries --> PreserveLegal["Preserve Legal Header"]
PreserveLegal --> ValidateStructure["Validate Final Structure"]
ValidateStructure --> Complete([Merge Complete])
FilterDuplicates --> CheckDuplicate{"AAGUID Already Exists?"}
CheckDuplicate --> |Yes| SkipEntry["Skip Custom Entry"]
CheckDuplicate --> |No| AddEntry["Add to Custom Entries"]
SkipEntry --> NextEntry["Next Entry"]
AddEntry --> NextEntry
NextEntry --> MoreEntries{"More Entries?"}
MoreEntries --> |Yes| ExtractAAGUIDs
MoreEntries --> |No| CombineEntries

Diagram sources

• metadata.py
• mds.js

AAGUID Deduplication

The merging process ensures that custom entries don't conflict with existing base entries:

1. AAGUID Extraction: Extract authenticator AAGUIDs from both custom and base entries
2. Conflict Detection: Identify AAGUIDs present in both datasets
3. Priority Resolution: Custom entries take precedence over base entries
4. Preservation: Base entries are preserved for unique AAGUIDs

Legal Header Handling

The system intelligently manages legal headers during merging:

• Custom Priority: Custom metadata legal headers take precedence
• Fallback: Uses base MDS legal header if custom lacks one
• Consistency: Ensures legal header consistency across merged dataset

Section sources

• metadata.py
• mds.js

Caching Mechanism

The custom metadata system implements a multi-tier caching strategy to optimize performance and reduce server load while maintaining data freshness.

Cache Architecture

graph LR
subgraph "Browser Cache"
BC[Browser Memory Cache]
LS[Local Storage]
end
subgraph "Server Cache"
SC[Session Cache]
FC[File Cache]
end
subgraph "Persistence Layer"
FS[File System]
CS[Cloud Storage]
end
BC --> SC
LS --> FS
SC --> FC
FC --> FS
FS --> CS

Diagram sources

• mds.js
• metadata.py

Browser-Side Caching

The JavaScript implementation maintains several cache layers:

• Memory Cache: Active custom metadata items in memory
• Promise Cache: Prevents concurrent loading requests
• Session Cache: Stores fetched metadata for reuse
• Abort Signal: Manages request cancellation and cleanup

Server-Side Caching

The server maintains caches for:

• Base Metadata: Pre-loaded verified MDS dataset
• Session Metadata: Custom entries per session
• File Metadata: Information about stored files
• Verification Cache: Trust verification results

Cache Invalidation

Cache invalidation occurs under these conditions:

• Manual Refresh: User-initiated reload
• File Changes: New uploads or deletions
• Session Expiry: Automatic cleanup of inactive sessions
• Time-based: Periodic cache warming

Section sources

• mds.js
• metadata.py

Error Handling and Validation

The system implements comprehensive error handling and validation at multiple levels to ensure robust operation and provide meaningful feedback to users.

Frontend Validation

flowchart TD
FileInput[File Input] --> TypeCheck{JSON File?}
TypeCheck --> |No| FileTypeError["File Type Error"]
TypeCheck --> |Yes| ParseCheck{Valid JSON?}
ParseCheck --> |No| ParseError["JSON Parse Error"]
ParseCheck --> |Yes| StructureCheck{Valid Structure?}
StructureCheck --> |No| StructureError["Structure Error"]
StructureCheck --> |Yes| Upload[Proceed with Upload]
FileTypeError --> ErrorMessage["Display Error Message"]
ParseError --> ErrorMessage
StructureError --> ErrorMessage
ErrorMessage --> RetryPrompt[Allow Retry]

Diagram sources

• mds.js
• general.py

Backend Validation

The server performs comprehensive validation:

1. File Format Validation: JSON syntax and structure checks
2. Metadata Schema Validation: Compliance with FIDO MDS standards
3. Session Validation: Active session verification
4. File Size Limits: Reasonable file size constraints
5. Duplicate Prevention: AAGUID uniqueness checks

Error Response Formats

The system provides structured error responses:

Upload Errors:

{
  "items": [],
  "errors": [
    "authenticator.json: Not a valid JSON object.",
    "Missing required field: aaguid.",
    "Unsupported algorithm specification."
  ]
}

Deletion Errors:

{
  "error": "Metadata entry not found or inaccessible."
}

User Feedback System

The system provides real-time user feedback through:

• Toast Messages: Non-intrusive status updates
• Progress Indicators: Visual feedback during operations
• Error Notifications: Clear error descriptions
• Success Confirmations: Positive reinforcement for successful operations

Section sources

• general.py
• mds.js

Testing and Simulation Scenarios

The custom metadata system enables extensive testing and simulation capabilities for various authenticator scenarios and certification testing.

Authenticator Testing Scenarios

Unreleased Authenticators

Custom metadata allows testing of authenticators that haven't yet been included in the official MDS:

{
  "aaguid": "deadbeef-dead-beef-dead-beefdeadbeef",
  "metadataStatement": {
    "description": "Prototype Biometric Authenticator",
    "authenticatorVersion": 1000,
    "upv": [{"major": 1, "minor": 0}],
    "attestationTypes": ["basic"],
    "userVerificationDetails": [
      {
        "authenticatorAttachmentHint": ["external"],
        "userVerificationMethod": "presence"
      }
    ],
    "keyProtection": ["hardware"],
    "matcherProtection": ["hardware"],
    "attachmentHint": ["external"],
    "tcDisplay": ["any"],
    "attestationRootCertificates": []
  }
}

Certification Level Simulation

Test different certification scenarios:

• FIDO Certified L1: Basic functionality testing
• FIDO Certified L2: Advanced security features
• Not FIDO Certified: Development/testing phase
• Revoked: Security incident simulation

Algorithm Testing

Simulate various cryptographic algorithms:

{
  "aaguid": "test-algorithm-aaguid",
  "metadataStatement": {
    "description": "Algorithm Testing Authenticator",
    "authenticatorVersion": 1,
    "upv": [{"major": 1, "minor": 0}],
    "attestationTypes": ["basic"],
    "userVerificationDetails": [
      {
        "authenticatorAttachmentHint": ["internal"],
        "userVerificationMethod": "presence",
        "caDesc": {
          "alg": [-8, -7, -257],  // P-256, P-384, P-521 ECDSA
          "minPinLength": 4,
          "noAuth": false
        }
      }
    ]
  }
}

Development Workflow Integration

The system supports typical development workflows:

1. Initial Setup: Upload baseline authenticator metadata
2. Feature Testing: Add metadata for new features
3. Regression Testing: Maintain historical authenticator records
4. Performance Testing: Simulate various authenticator capabilities
5. Compatibility Testing: Test against different client implementations

Test Data Management

The system provides tools for managing test data:

• Batch Upload: Import multiple authenticator configurations
• Template System: Reuse common authenticator patterns
• Version Control: Track changes to authenticator metadata
• Export Capabilities: Export custom metadata for sharing

Section sources

• test_metadata_sessions.py
• metadata.py

Security Considerations

The custom metadata system implements several security measures to protect against malicious uploads and ensure data integrity.

Input Sanitization

All uploaded metadata undergoes rigorous sanitization:

• JSON Parsing: Safe JSON parsing with error handling
• Schema Validation: Compliance with FIDO MDS specifications
• Type Checking: Strict type validation for all fields
• Size Limits: Reasonable limits on file sizes
• Character Encoding: Proper UTF-8 validation

Session Isolation

Security through isolation:

• Per-Session Storage: Each browser session has separate metadata
• Cookie-Based Authentication: Secure session identification
• Automatic Cleanup: Inactive sessions are automatically removed
• Cross-Origin Protection: CSRF protection for all operations

Data Integrity

Measures to ensure data integrity:

• File Hashing: SHA-256 hashing for file integrity verification
• Backup Systems: Cloud backup with deduplication
• Transaction Safety: Atomic operations for uploads/deletes
• Audit Trails: Logging of all metadata operations

Access Control

Limited access model:

• Browser Session Only: Metadata accessible only within current session
• No Persistent Storage: No server-side persistence of custom metadata
• Temporary Nature: All custom metadata cleared on session termination
• Isolation Guarantees: Complete isolation between different users/sessions

Section sources

• metadata.py
• session_metadata_store.py

Troubleshooting Guide

Common issues and their solutions when working with the custom metadata management system.

Upload Issues

Problem: Files Not Recognized as JSON

Symptoms: "Not a JSON file" errors during upload Causes:

• Incorrect file extension (.json required)
• Malformed JSON syntax
• Encoding issues (UTF-8 required)

Solutions:

1. Verify file extension ends with .json
2. Validate JSON syntax using online validators
3. Ensure UTF-8 encoding without BOM
4. Check file size limitations

Problem: Upload Timeout

Symptoms: Upload hangs or times out Causes:

• Large file size
• Network connectivity issues
• Server resource constraints

Solutions:

1. Reduce file size (recommended < 1MB)
2. Check network connection
3. Retry upload with smaller files
4. Monitor browser developer console for errors

Display Issues

Problem: Custom Metadata Not Visible

Symptoms: Uploaded files don't appear in list Causes:

• Session mismatch
• Cache issues
• Browser compatibility problems

Solutions:

1. Refresh page to reload metadata
2. Clear browser cache and cookies
3. Try different browser
4. Check browser console for JavaScript errors

Problem: Formatting Issues

Symptoms: Metadata displays incorrectly Causes:

• Invalid metadata structure
• Missing required fields
• Encoding problems

Solutions:

1. Validate metadata against FIDO MDS schema
2. Ensure all required fields are present
3. Check for special characters in descriptions
4. Review metadata examples in documentation

Performance Issues

Problem: Slow Loading

Symptoms: Long delays when loading metadata Causes:

• Large number of custom entries
• Network latency
• Browser performance issues

Solutions:

1. Limit number of custom entries
2. Clear browser cache
3. Close unnecessary tabs
4. Use browser developer tools to identify bottlenecks

Session Management

Problem: Session Lost

Symptoms: Custom metadata disappears unexpectedly Causes:

• Browser cache cleared
• Cookie restrictions
• Session timeout

Solutions:

1. Check cookie settings in browser
2. Clear cache selectively
3. Reload page to restore session
4. Verify browser privacy settings

Section sources

• mds.js
• metadata.py