Product Requirements Document: Content Moderation - stephenhoward/pavillion GitHub Wiki

Overview

The Event Flagging System will allow calendar visitors, calendar owners, and administrators to report problematic events through a tiered review process that respects calendar ownership and administrative hierarchy. This feature supports community moderation while implementing safeguards against abuse and maintaining the autonomy of individual Pavillion instances.

Background and Strategic Fit

Communities need effective ways to maintain content quality and safety standards. Since Pavillion operates on a federation model with public viewing without registration requirements, we need a mechanism that:

  1. Allows anonymous visitors to contribute to community health
  2. Prevents abuse of the reporting system
  3. Respects calendar ownership and instance autonomy
  4. Enables appropriate escalation of moderation decisions
  5. Facilitates cross-instance communication about problematic content
  6. Supports calendar owners and administrators in making informed decisions

This feature aligns with our core project principles of accessibility (allowing anyone to report issues) and autonomy (providing tools for local governance of content with appropriate escalation paths).

User Personas

Calendar Visitor

  • Does not have an account
  • Encounters a problematic event
  • Wants to alert moderators without creating an account
  • Must complete email verification workflow

Calendar Owner

  • First line of moderation for their own calendar
  • Can also act as a reporter when viewing other calendars they follow
  • Already authenticated in the system
  • Needs tools to review reports on their own events or reposts
  • May need to communicate with remote calendar owners
  • Can delegate report review responsibilities to trusted editors

Calendar Editor

  • Account holder invited by a calendar owner to contribute to their calendar
  • Can create and edit events on the calendar
  • May be granted permission to review reports by the calendar owner
  • Shares responsibility for maintaining calendar community standards
  • Operates with delegated authority from the calendar owner
  • Cannot change report review permissions for other editors

Instance Administrator

  • Oversees all calendars on their instance
  • Reviews reports escalated from calendar owners
  • Makes final moderation decisions for the instance
  • Can flag content for calendar owner review before taking direct action
  • Maintains final oversight while respecting calendar owner autonomy

Feature Requirements

Reporting Flow

  1. Anonymous Visitor Reporting

    • Accessible via a discreet "Report Event" link on event pages
    • Collects information about:
      • Nature of the problem (dropdown with categories)
      • Description field for details
      • Reporter's email address
    • Clear explanation that email verification is required
    • System checks if the same email address has already reported this event and prevents duplicate reports
    • Email verification process with token generation and confirmation link
    • Report stored as pending until verified
    • After verification, report submitted to calendar owner's review queue

  2. Authenticated Calendar Owner Reporting

    • Same report form access via "Report Event" link
    • System recognizes logged-in user status
    • Pre-fills email with account email (non-editable)
    • No email verification required - report is submitted immediately
    • System still checks for duplicate reports from this user/email
    • Immediate confirmation of report submission
    • Report directly enters calendar owner's review queue

  3. Administrator Reporting

    • Same report form with additional options:
      • Option to flag as "admin concern" with priority indicator
      • Option to set review deadline before admin action
      • Additional context field for administrator-specific notes
    • No email verification required - report is submitted immediately
    • Report clearly marked in calendar owner's queue as admin-initiated
    • System notifies calendar owner through multiple channels (email, in-app)
    • If unaddressed within deadline, system automatically returns report to admin queue

Calendar Owner Review Interface

  1. Reports Dashboard

    • List of verified reports for events on their calendar
    • Special highlighting for admin-initiated reports
    • Prominent display of admin-set deadlines when applicable
    • Sorting and filtering options:
      • By event
      • By reporter (using email hash)
      • By report category
      • By date
      • By status (new, reviewed, resolved, dismissed)
      • By source (admin, user, anonymous)
    • Permission controls for delegating report review responsibilities to editors

  2. Report Grouping

    • Automatic grouping of reports about the same event
    • Visual indicators for patterns (multiple reports from same source or about same event)
    • Collapsible view to manage screen real estate
    • Special grouping for admin-initiated reports to ensure visibility

  3. Review Actions

    • View event details alongside report
    • For local events:
      • Edit, hide, or delete the event
      • Mark report as resolved
      • Add notes explaining decision (required for admin-initiated reports)
    • For reposted events:
      • Remove repost from their calendar
      • Forward report to remote calendar owner
      • Mark report as resolved
    • Dismiss report (automatically escalates to instance administrator)
    • Add notes explaining decision
    • Audit log showing which editor took which actions (if report review is delegated)

Editor Permissions Management

  1. Invitation Process

    • When inviting a new editor, calendar owners can enable/disable report review permission
    • Clear explanation of what report review entails during invitation process
    • Default setting configurable at calendar level (on or off by default)
    • Option to grant report review permission for specific event categories only

  2. Permission Management Interface

    • Dashboard for calendar owners to view all editors and their permissions
    • Toggle controls to enable/disable report review permissions at any time
    • Ability to revoke permissions without removing editor status
    • Batch operations for managing multiple editors simultaneously
    • Editor permission history for auditing purposes

  3. Editor Report Review Experience

    • Same review interface as calendar owners with clear indication of delegated authority
    • Actions taken by editors clearly attributed in audit logs
    • Optional notification to calendar owner when editors resolve or dismiss reports
    • Configurable thresholds for requiring owner approval on certain report types
    • Visual indicators showing which reports have been reviewed by editors vs. owners

Administrator Review Interface

  1. Escalated Reports Dashboard

    • List of reports dismissed by calendar owners or auto-escalated due to inactivity
    • Similar sorting and filtering as calendar owner interface
    • Additional filter for calendar source
    • Visual distinction between manually escalated and auto-escalated reports
    • Special section for admin-initiated reports that were not addressed by owners

  2. Report Creation Interface

    • Option to report events directly as an administrator
    • Setting deadlines for calendar owner review
    • Priority level indicators
    • Rich text formatting for detailed context and reasoning

  3. Review Actions

    • All actions available to calendar owners
    • Override calendar owner decisions
    • Instance-wide actions:
      • Block remote instances
      • Add instance-wide content policies
      • Set instance-wide auto-escalation timeframe (hours/days before unreviewed reports automatically escalate)
    • Final dismissal (closes report permanently)
    • Option to create a new admin-initiated report after reviewing existing reports

  4. Auto-Escalation Configuration

    • Set a single, instance-wide timeframe for automatic escalation of unreviewed reports
    • Option to send reminder notifications to calendar owners before auto-escalation occurs
    • Separate configuration for admin-initiated reports (typically shorter timeframe)

Cross-Instance Communication

  1. Calendar Owner to Calendar Owner

    • When acting on remote event, notification sent to remote calendar owner first
    • Message composer with report details pre-filled
    • Receipt confirmation when remote owner receives notification
    • Special indicator for admin-initiated reports that have been forwarded

  2. Escalation to Remote Administrator

    • If remote calendar owner dismisses report
    • Similar notification process but directed to remote administrator
    • Option for local administrator to block future content from source instance
    • Special message format for forwarding admin-initiated reports

Data Structure

  1. Report Record

    • Report ID (unique identifier)
    • Event ID (reference to reported event)
    • Calendar ID (reference to calendar containing the event)
    • Report category
    • Report description
    • Reporter information:
      • Email (hashed for privacy if anonymous visitor)
      • User ID (null for anonymous visitors, populated for authenticated users)
      • Reporter type flag (anonymous/authenticated/administrator)
    • Admin-specific fields:
      • Admin ID (if report was created by administrator)
      • Admin priority level (low/medium/high)
      • Admin review deadline
      • Admin notes (private notes visible only to administrators)
    • Submission timestamp
    • Last updated timestamp
    • Verification status (verified for authenticated users, pending/verified for anonymous)
    • Review status (owner review, admin review, resolved, dismissed)
    • Owner notes
    • Admin notes
    • Editor review information:
      • Reviewer ID (ID of editor who reviewed the report, if applicable)
      • Editor notes
      • Review timestamp
    • Verification token (for anonymous reporters only)
    • Auto-escalation deadline (calculated from submission time and instance-wide setting)
    • Escalation type (manual or automatic)
    • IP information:
      • Salted hash of full IP address (stored for 30 days)
      • Subnet information (first 3 octets of IPv4 or first 64 bits of IPv6, stored for 90 days)
      • Geographic region (non-specific, city/country level, stored indefinitely)
      • Automatic deletion after retention period through database lifecycle policies

  2. EventReport Junction

    • Enables many-to-many relationship between events and reports
    • Facilitates report grouping

  3. Report Escalation Record

    • Tracks the path of escalation
    • Timestamps for each review level
    • Review decisions at each level
    • Reviewer ID and role at each step

  4. Event Reporter Record

    • Composite unique key of event ID and reporter identifier (email hash or user ID)
    • Prevents duplicate reports from the same source for the same event
    • Timestamp of report submission

  5. Instance Configuration

    • System-wide auto-escalation timeframe
    • Admin report escalation timeframe
    • Notification timing for approaching deadlines
    • IP retention configuration (admins can reduce but not extend beyond default periods)

  6. Editor Permission Record

    • Calendar ID
    • Editor user ID
    • Report review permission flag
    • Granted timestamp
    • Granting user ID (calendar owner who granted permission)
    • Permission scope (all reports/specific categories)
    • Category restrictions (if applicable)

Technical Requirements

  1. Authentication Integration

    • Detection of authenticated users in the reporting flow
    • Different processing paths for authenticated vs. anonymous reports
    • Special processing for admin-initiated reports
    • Security measures to prevent authentication bypass

  2. Email System Integration

    • Secure token generation (for anonymous reporters)
    • Rate limiting for verification emails (maximum 3 per email address per 24 hours)
    • Proper email deliverability practices
    • Templates for verification, notification, and escalation emails
    • Priority indicators for admin-initiated report notifications

  3. Security Measures

    • Email hashing for privacy (for anonymous reporters)
    • Protection against enumeration attacks
    • CAPTCHA or similar challenge on anonymous report form to prevent automated abuse
    • Rate limiting by IP address
    • Prevention of duplicate reports based on appropriate identifier (email hash or user ID)
    • Tiered IP address retention and anonymization:
      • Full IP hash retained for 30 days for abuse pattern detection
      • Subnet information retained for 90 days for broad pattern analysis
      • Automatic deletion systems with tamper-proof audit logs
      • Geographic data generalized to city/country level only
      • Separation of IP data from personal identifiers where possible

  4. ActivityPub Implementation

    • Define message format for cross-instance report notifications
    • Special message format for admin-initiated reports
    • Implement proper authentication for cross-instance communications
    • Store record of sent notifications
    • Support for owner-to-owner and admin-to-admin communications

  5. Auto-Escalation System

    • Background job to periodically check for reports approaching escalation deadline
    • Notification service for upcoming escalations
    • Special handling for admin-initiated reports with deadlines
    • Simple, instance-wide escalation rules
    • Audit logging of all automatic escalations

User Experience

  1. Visitor Experience

    • Discreet but accessible reporting link
    • Simple, straightforward form
    • Clear expectations about verification process
    • Confirmation messages at each step

  2. Calendar Owner Experience

    • Dashboard showing reports only for their calendars
    • Clear highlighting for admin-initiated reports
    • Deadline indicators for reports requiring urgent attention
    • Quick actions for common moderation tasks
    • Clear escalation path to administrator
    • Notification system for new reports with priority for admin-initiated reports
    • Intuitive interface for managing editor report review permissions
    • Activity dashboard showing editor review actions

  3. Calendar Editor Experience

    • Clear indication of report review responsibilities
    • Same review interface as calendar owners with appropriate permission indicators
    • Visual distinction between reports they can and cannot review (if permissions are category-based)
    • Notification system for new reports requiring their attention
    • Easy access to calendar owner consultation if needed
    • Attribution of their review actions in the system

  4. Administrator Experience

    • Dashboard prioritizing escalated reports
    • Context of previous owner decisions
    • Efficient workflow for reviewing multiple reports
    • Tools for instance-wide moderation actions
    • Streamlined process for creating administrator reports
    • Tracking system for admin-initiated reports
    • Visibility into calendar owner and editor review activities

Error States and Edge Cases

  1. Prevention of Abuse

    • Email verification acts as primary barrier to automated abuse
    • Rate limiting prevents overwhelming the system
    • Calendar owner can flag suspicious reporting patterns
    • Administrator can block emails that repeatedly submit false reports
    • Duplicate report prevention ensures same email address cannot report same event multiple times
    • System presents appropriate error message when duplicate report is attempted

  2. Handling Report Floods

    • Grouping by event and reporter helps manage volume
    • Prioritization based on multiple reports of same event
    • Option to automatically escalate events that receive threshold number of reports
    • Special handling to maintain visibility of admin reports during high-volume periods

  3. Cross-Instance Communication Failures

    • Retry mechanism for failed ActivityPub notifications
    • Fallback contact methods for administrators
    • Local actions can proceed regardless of remote response
    • Special retry priority for admin-initiated reports

  4. Auto-Escalation Edge Cases

    • System downtime affecting scheduled escalations
    • Handling reports about events that are scheduled to occur before the auto-escalation deadline
    • Notification failures for approaching deadlines
    • Handling of unaddressed admin-initiated reports after deadline expiration

Success Metrics

  1. Effectiveness

    • Percentage of verified reports that lead to moderation action
    • Resolution rate at calendar owner level vs. escalation rate
    • Number of problematic events identified through visitor reports
    • Resolution rate of admin-initiated reports without requiring final admin action
    • Average time to resolution for admin-initiated vs. user-initiated reports

  2. Usability

    • Verification completion rate
    • Time to review reports at each level
    • Calendar owner and administrator satisfaction
    • Percentage of admin reports that result in calendar owner action

  3. Abuse Prevention

    • Ratio of verified to unverified reports
    • Number of blocked abuse attempts
    • False positive rate
    • Ratio of admin-initiated reports to other reports

Implementation Phases

Phase 1: Basic Reporting with Owner Review

  • Report form with email verification
  • Calendar owner review interface
  • Local-only actions
  • Simple escalation to administrator
  • Basic editor invitation with report review permission toggle

Phase 2: Enhanced Analysis and Communication

  • Report grouping and pattern detection
  • Cross-instance calendar owner notifications
  • Expanded administrator tools
  • Historical analytics
  • Basic administrator reporting functionality
  • Advanced editor permission management
  • Editor activity auditing and attribution

Phase 3: Complete Federation Features

  • Full escalation path across instances
  • Instance blocking capabilities
  • Federation status dashboard
  • Advanced reporting analytics
  • Complete administrator reporting system with deadlines and priority levels
  • Cross-instance editor permission compatibility
  • Fine-grained category-based permission controls for editors

Future Considerations

  1. Trusted Reporters

    • Option for calendar owners to whitelist certain email addresses to bypass verification
    • Reputation system for consistent helpful reporters
    • Special trust level for administrators from other instances

  2. Category-Based Workflows

    • Different review processes based on report category
    • Automatic routing of certain reports directly to administrators
    • Category-specific auto-escalation timeframes (e.g., urgent categories escalate faster)
    • Default categories for admin-initiated reports based on common patterns

  3. Appeals Process

    • Allow event creators to respond to reports
    • Documentation system for moderation decisions
    • Visitor notification of report outcomes (optional)
    • Special workflow for appealing actions taken on admin-initiated reports

  4. Customized Escalation Policies

    • Calendar-specific auto-escalation timeframes
    • Ability to temporarily suspend auto-escalation for specific calendars (e.g., when owner is on vacation)
    • Process for temporarily extending deadlines in exceptional circumstances
    • Dynamic escalation rules based on report volume or patterns
    • Custom escalation paths for different types of reports (user vs. admin)

  5. Event Content Change Detection

    • System to detect significant changes to event content
    • Allow re-reporting of previously reported events if content has changed significantly
    • Define thresholds for what constitutes "significant" change
    • Option for administrators to reset reporting eligibility for specific events
    • Special handling for changes made in response to admin-initiated reports

  6. Progressive Friction and Abuse Prevention

    • Reputation-based verification requirements:
      • Heightened verification for first-time reporters
      • Extra scrutiny for disposable email domains
      • Reduced friction for accounts with established history
    • Context-sensitive verification:
      • Dynamic increase in verification requirements when multiple new accounts target the same content
      • Automatic increase in verification difficulty during report spikes
      • Enhanced protection for imminent events
    • Tiered verification methods:
      • Escalation to secondary verification (SMS, enhanced CAPTCHA) when abuse suspected
      • Time delays between verification attempts
      • Privacy-preserving proof of unique identity options
    • Circuit breakers:
      • Temporary disabling of anonymous reporting for calendars experiencing report floods
      • Cooling-off periods after rejected reports
      • Automatic administrator alerts for suspected coordinated action
    • Cross-instance abuse coordination:
      • Sharing of abuse patterns across trusted instances
      • Federated reputation systems for reporting sources
      • Instance-level circuit breakers for cross-federation reporting