Analytics - nself-org/nchat GitHub Wiki

Analytics & Telemetry System (v0.5.0)

Overview

nChat v0.5.0 introduces a comprehensive analytics and telemetry system for tracking usage, performance, and engagement metrics. Built on TimescaleDB for efficient time-series data storage and querying.

Table of Contents

  1. Architecture
  2. Features
  3. Database Schema
  4. Configuration
  5. Usage
  6. API Endpoints
  7. Dashboard
  8. Privacy & Compliance
  9. Performance
  10. Troubleshooting

Architecture

Components

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  React Client   β”‚ ──track()──▢ Analytics Client
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                     β”‚
                                        β”‚ batch
                                        β–Ό
                            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                            β”‚  Analytics API     β”‚
                            β”‚  /api/analytics/*  β”‚
                            β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                        β”‚
                                        β–Ό
                            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                            β”‚  PostgreSQL +      β”‚
                            β”‚  TimescaleDB       β”‚
                            β”‚  (Hypertables)     β”‚
                            β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                        β”‚
                                        β–Ό
                            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                            β”‚  Hasura GraphQL    β”‚
                            β”‚  (Queries/Subs)    β”‚
                            β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                        β”‚
                                        β–Ό
                            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                            β”‚  Analytics         β”‚
                            β”‚  Dashboard         β”‚
                            β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Technology Stack

  • Database: PostgreSQL 15+ with TimescaleDB extension
  • Time-Series Storage: TimescaleDB hypertables with automatic partitioning
  • API: Next.js API routes
  • GraphQL: Hasura for real-time queries and subscriptions
  • Client: React hooks + Zustand for state management
  • Visualization: Recharts (lightweight, customizable)
  • Export: CSV, JSON, PDF support

Features

1. Usage Analytics

  • User Activity

    • Daily/Weekly/Monthly Active Users (DAU/WAU/MAU)
    • User retention cohorts
    • Session duration tracking
    • Feature adoption rates
  • Message Analytics

    • Message volume by channel/user/time
    • Average message length
    • Attachments vs text-only
    • Thread participation rates
  • Channel Analytics

    • Channel growth over time
    • Active vs inactive channels
    • Engagement rates by channel
    • Member participation

2. Performance Analytics

  • API Performance

    • Response times (p50/p95/p99)
    • Error rates by endpoint
    • Request volume by endpoint
    • Database query performance
  • WebSocket Metrics

    • Connection health
    • Message latency
    • Packet loss rates
    • Concurrent connections

3. Engagement Metrics

  • Reaction Analytics

    • Most used reactions
    • Reaction diversity
    • Top reactors
  • File Analytics

    • Upload volume and size
    • File type breakdown
    • Storage usage per user
  • Search Analytics

    • Popular search queries
    • No-results searches
    • Click-through rates

4. Business Metrics

  • User growth rate
  • Storage usage trends
  • Feature usage by tier
  • API call volume

Database Schema

Core Tables

1. nchat_analytics_events (Hypertable)

Primary time-series table for all analytics events.

CREATE TABLE nchat_analytics_events (
  id UUID PRIMARY KEY,
  timestamp TIMESTAMPTZ NOT NULL,
  event_name VARCHAR(100) NOT NULL,
  event_category VARCHAR(50) NOT NULL,
  user_id UUID REFERENCES nchat_users(id),
  session_id VARCHAR(100) NOT NULL,
  platform VARCHAR(20),
  properties JSONB DEFAULT '{}'
);

-- Convert to hypertable (1 day chunks)
SELECT create_hypertable('nchat_analytics_events', 'timestamp',
  chunk_time_interval => INTERVAL '1 day');

-- Retention: 90 days
SELECT add_retention_policy('nchat_analytics_events', INTERVAL '90 days');

2. nchat_analytics_user_activity (Hypertable)

Aggregated hourly user activity metrics.

CREATE TABLE nchat_analytics_user_activity (
  timestamp TIMESTAMPTZ NOT NULL,
  user_id UUID NOT NULL,
  messages_sent INTEGER DEFAULT 0,
  reactions_given INTEGER DEFAULT 0,
  files_uploaded INTEGER DEFAULT 0,
  engagement_score FLOAT DEFAULT 0,
  session_duration_seconds INTEGER DEFAULT 0,
  UNIQUE(user_id, timestamp)
);

SELECT create_hypertable('nchat_analytics_user_activity', 'timestamp',
  chunk_time_interval => INTERVAL '7 days');

3. nchat_analytics_channel_activity (Hypertable)

Aggregated hourly channel activity metrics.

CREATE TABLE nchat_analytics_channel_activity (
  timestamp TIMESTAMPTZ NOT NULL,
  channel_id UUID NOT NULL,
  messages_count INTEGER DEFAULT 0,
  active_users_count INTEGER DEFAULT 0,
  reactions_count INTEGER DEFAULT 0,
  average_response_time_seconds FLOAT DEFAULT 0,
  UNIQUE(channel_id, timestamp)
);

4. Continuous Aggregates (Materialized Views)

Automatically maintained aggregated views for fast queries:

-- Daily user summary
CREATE MATERIALIZED VIEW nchat_analytics_daily_user_summary
WITH (timescaledb.continuous) AS
SELECT
  time_bucket('1 day', timestamp) AS day,
  user_id,
  SUM(messages_sent) AS total_messages,
  AVG(engagement_score) AS avg_engagement_score
FROM nchat_analytics_user_activity
GROUP BY day, user_id;

-- Refresh every hour
SELECT add_continuous_aggregate_policy(
  'nchat_analytics_daily_user_summary',
  start_offset => INTERVAL '7 days',
  end_offset => INTERVAL '1 hour',
  schedule_interval => INTERVAL '1 hour'
);

Helper Functions

-- Get Daily Active Users
CREATE FUNCTION nchat_analytics_get_dau(start_date DATE, end_date DATE)
RETURNS TABLE (date DATE, active_users BIGINT);

-- Get Monthly Active Users
CREATE FUNCTION nchat_analytics_get_mau(start_date DATE)
RETURNS BIGINT;

-- Calculate Engagement Score
CREATE FUNCTION nchat_analytics_calculate_engagement_score(
  p_messages INT, p_reactions INT, p_files INT, p_threads INT
)
RETURNS FLOAT;

Configuration

AppConfig

Add to src/config/app-config.ts:

analytics: {
  enabled: boolean // Master toggle
  trackingMode: 'full' | 'privacy' | 'minimal'
  retentionDays: number // Data retention (default: 90)
  enableRealtime: boolean // Real-time dashboard updates
  enableExports: boolean // Allow CSV/PDF exports
  anonymizeUserData: boolean // Anonymize PII
  excludeTestUsers: boolean // Exclude test users from stats
  trackPerformance: boolean // Track API performance
  trackWebSocket: boolean // Track WebSocket metrics
  trackFeatureUsage: boolean // Track feature adoption
  trackErrors: boolean // Track errors (Sentry integration)
  allowUserOptOut: boolean // Allow users to opt-out
}

Environment Variables

# Required
NEXT_PUBLIC_ANALYTICS_ENABLED=true

# Optional
ANALYTICS_RETENTION_DAYS=90
TIMESCALEDB_ENABLED=true
ANALYTICS_BATCH_SIZE=10
ANALYTICS_FLUSH_INTERVAL=30000

Usage

Client-Side Tracking

Basic Event Tracking

import { useAnalytics } from '@/hooks/use-analytics';

function MyComponent() {
  const { track } = useAnalytics();

  const handleClick = () => {
    track('button_clicked', {
      buttonName: 'Submit',
      location: 'chat-input',
    });
  };

  return <button onClick={handleClick}>Submit</button>;
}

User Identification

import { useAnalytics } from '@/hooks/use-analytics'

function AuthComponent() {
  const { identify } = useAnalytics()

  useEffect(() => {
    if (user) {
      identify(user.id, {
        email: user.email,
        displayName: user.displayName,
        role: user.role,
        plan: 'free',
      })
    }
  }, [user])
}

Page View Tracking

import { useAnalytics } from '@/hooks/use-analytics'

function Page() {
  const { page } = useAnalytics()

  useEffect(() => {
    page('/chat', 'Chat - nChat')
  }, [])
}

Convenience Methods

const {
  trackMessageSent,
  trackMessageEdited,
  trackReactionAdded,
  trackFileUploaded,
  trackSearchPerformed,
  trackFeatureUsed,
  trackError,
} = useAnalytics()

// Track message sent
trackMessageSent(channelId, messageLength, hasAttachments)

// Track reaction
trackReactionAdded(messageId, 'πŸ‘')

// Track file upload
trackFileUploaded(fileId, fileName, fileSize, mimeType)

// Track search
trackSearchPerformed(query, resultCount)

// Track feature usage
trackFeatureUsed('voice-call', { duration: 120 })

// Track error
trackError('api_error', 'Failed to send message', { code: 500 })

Dashboard Consumption

useAnalyticsDashboard Hook

import { useAnalyticsDashboard } from '@/hooks/use-analytics-dashboard';

function AnalyticsDashboard() {
  const {
    summary,
    messageVolume,
    userActivity,
    isLoading,
    error,
    setDateRange,
    refresh,
  } = useAnalyticsDashboard();

  if (isLoading) return <Spinner />;
  if (error) return <Error message={error} />;

  return (
    <div>
      <h1>Analytics Dashboard</h1>
      <MetricCard value={summary.messages.total.value} label="Total Messages" />
      <MetricCard value={summary.users.activeUsers.value} label="Active Users" />
      <Chart data={messageVolume} />
    </div>
  );
}

Specialized Hooks

// Message analytics
const { volume, topMessages, stats } = useMessageAnalytics()

// User analytics
const { activity, growth, activeUsers } = useUserAnalytics()

// Channel analytics
const { activity, stats } = useChannelAnalytics()

// Date range filtering
const { dateRange, setPreset, setRange } = useDateRangeFilter()
setPreset('last30days')
setRange(new Date('2025-01-01'), new Date('2025-01-31'))

// Real-time updates (auto-refresh every 30 seconds)
useRealtimeAnalytics(30000)

API Endpoints

1. Track Events

POST /api/analytics/track
Content-Type: application/json

[
  {
    "name": "message_sent",
    "category": "messaging",
    "properties": {
      "channelId": "uuid",
      "messageLength": 42
    }
  }
]

Response: { success: true, tracked: 1 }

2. Get Dashboard Data

GET /api/analytics/dashboard?start=2025-01-01&end=2025-01-31&granularity=day

Response: {
  success: true,
  data: {
    summary: { ... },
    messageVolume: [...],
    userActivity: [...],
    ...
  }
}

3. Get Section Data

GET /api/analytics/messages
GET /api/analytics/users
GET /api/analytics/channels
GET /api/analytics/reactions

4. Export Data

POST /api/analytics/export
Content-Type: application/json

{
  "format": "csv",
  "sections": ["summary", "messages", "users"],
  "dateRange": { "start": "2025-01-01", "end": "2025-01-31" }
}

Response: {
  success: true,
  downloadUrl: "/api/analytics/download/export-123.csv"
}

Dashboard

Access

Navigate to /admin/analytics (admin/owner only).

Views

  1. Overview - High-level metrics and trends
  2. Messages - Message volume, top messages, patterns
  3. Users - User activity, growth, retention
  4. Channels - Channel engagement, top channels
  5. Reactions - Reaction analytics, top emojis
  6. Files - File uploads, storage usage
  7. Search - Search queries, no-results searches
  8. Reports - Scheduled reports, export history

Filters

  • Date Range: Today, Yesterday, Last 7/30/90 days, Custom
  • Granularity: Hour, Day, Week, Month, Year
  • Channels: Filter by specific channels
  • Users: Filter by specific users
  • Include Bots: Toggle bot inclusion

Exports

Supported formats:

  • CSV: Spreadsheet-friendly
  • JSON: API-friendly
  • PDF: Report-ready (with charts)
  • XLSX: Excel-compatible

Privacy & Compliance

GDPR Compliance

  • Data Minimization: Only collect necessary data
  • Anonymization: Option to anonymize user data
  • Right to Access: Users can export their data
  • Right to Erasure: Users can request deletion
  • Consent Management: Opt-in/opt-out support

Data Retention

  • Raw Events: 90 days (configurable)
  • Aggregated Data: 365 days
  • Continuous Aggregates: Permanent (summary only)
  • Automatic Cleanup: TimescaleDB retention policies

User Opt-Out

// Allow users to opt-out
const { reset } = useAnalytics()
reset() // Clears user identity and stops tracking

Performance

Optimizations

  1. Client-Side Batching: Events batched and sent every 30 seconds
  2. Server-Side Caching: Dashboard data cached for 5 minutes
  3. Continuous Aggregates: Pre-computed summaries for fast queries
  4. TimescaleDB Compression: Automatic compression of old data
  5. Index Optimization: Strategic indexes on high-traffic queries

Query Performance

  • DAU/MAU queries: <100ms
  • Dashboard load: <500ms
  • Event insertion: <10ms (batched)
  • Export generation: <5s for 30 days

Scalability

  • 10M+ events/day: Tested and optimized
  • 100k+ concurrent users: Supported
  • 10TB+ data: TimescaleDB handles efficiently

Troubleshooting

Issue: Events Not Tracking

Solution:

  1. Check analytics.enabled in AppConfig
  2. Verify NEXT_PUBLIC_ANALYTICS_ENABLED=true
  3. Check browser console for errors
  4. Verify analytics client initialized

Issue: Dashboard Slow

Solution:

  1. Reduce date range
  2. Increase cache TTL
  3. Check TimescaleDB compression status
  4. Verify continuous aggregates are refreshing

Issue: High Database Load

Solution:

  1. Enable continuous aggregates
  2. Increase retention policy frequency
  3. Check for missing indexes
  4. Enable TimescaleDB compression

Issue: Missing Data

Solution:

  1. Check retention policies (data may be expired)
  2. Verify hypertables are created
  3. Check Hasura permissions
  4. Verify cron jobs running (continuous aggregates)

Migration Guide

From v0.4.0 to v0.5.0

  1. Enable TimescaleDB:

    cd .backend
    nself db exec postgres "CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;"
  2. Run Analytics Migration:

    nself db migrate up
    # Applies analytics schema from migrations/
  3. Update AppConfig:

    analytics: {
      enabled: true,
      trackingMode: 'full',
      retentionDays: 90,
      // ...
    }
  4. Initialize Analytics Client:

    // Add to _app.tsx or root layout
    import { getAnalyticsClient } from '@/lib/analytics/analytics-client'
    
    useEffect(() => {
      getAnalyticsClient().initialize()
    }, [])

Best Practices

  1. Track Meaningful Events: Don't over-track; focus on business-critical events
  2. Use Batching: Batch events for better performance
  3. Respect Privacy: Anonymize PII, honor opt-outs
  4. Monitor Performance: Watch for slow queries
  5. Regular Exports: Export data regularly for backup
  6. Dashboard Access: Restrict to admins only
  7. Test Tracking: Use test users excluded from stats

Roadmap

v0.5.1

  • Enhanced PDF exports with charts
  • Email scheduled reports
  • Custom dashboard widgets

v0.6.0

  • Machine learning insights
  • Anomaly detection
  • Predictive analytics

Support

For issues or questions:


Version: v0.5.0 Last Updated: 2026-01-30 Contributors: nself team

⚠️ **GitHub.com Fallback** ⚠️