SEARCH ANALYTICS GUIDE - nself-org/nchat GitHub Wiki
Ι³Chat v0.9.1 - Complete Documentation
- Overview
- MeiliSearch Integration
- Search Features
- Analytics Dashboard
- Usage Tracking
- API Reference
- Real-time Indexing
- Export & Reporting
Ι³Chat provides comprehensive search and analytics capabilities powered by MeiliSearch and custom analytics aggregation.
- Full-text search across messages, files, users, and channels
- Real-time indexing via GraphQL subscriptions
- Advanced filtering with search operators
- Analytics dashboard with interactive charts
- Usage tracking with plan limit monitoring
- Export functionality (CSV, JSON)
- Performance optimized with caching and batch operations
βββββββββββββββββββ
β Frontend β
βββββββββββββββββββ€
β Search API β
β /api/search β
βββββββββββββββββββ€
β Search Service β
β search.service β
βββββββββββββββββββ€
β Sync Service β
β sync.service β
βββββββββββββββββββ€
β MeiliSearch β
β Port: 7700 β
βββββββββββββββββββ
Ι³Chat maintains 4 MeiliSearch indexes:
| Index | Purpose | Documents | Searchable Fields |
|---|---|---|---|
nchat_messages |
Message content | All messages | content, content_plain, author_name |
nchat_files |
File metadata | All files | name, description, original_name |
nchat_users |
User profiles | All users | username, display_name, email, bio |
nchat_channels |
Channel info | All channels | name, description, topic |
Each index is configured with:
- Filterable attributes: For precise filtering
- Sortable attributes: For ordering results
- Ranking rules: Custom relevance scoring
- Stop words: Common words to ignore
- Typo tolerance: Fuzzy matching for misspellings
// GET /api/search?q=hello
const response = await fetch('/api/search?q=hello')
const data = await response.json()// POST /api/search
const response = await fetch('/api/search', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
query: 'project update',
types: ['messages', 'files'],
channelIds: ['channel-id-1'],
dateFrom: '2024-01-01',
dateTo: '2024-12-31',
limit: 20,
offset: 0,
sortBy: 'date',
sortOrder: 'desc',
}),
})Ι³Chat supports Slack-style search operators:
| Operator | Example | Description |
|---|---|---|
from: |
from:john |
Filter by sender |
in: |
in:general |
Filter by channel |
has: |
has:link |
Messages with links |
has: |
has:file |
Messages with files |
has: |
has:image |
Messages with images |
before: |
before:2024-01-01 |
Before date |
after: |
after:2024-01-01 |
After date |
is: |
is:pinned |
Pinned messages |
is: |
is:starred |
Starred messages |
Example:
project from:alice in:engineering has:file after:2024-01-01
import { getSearchService } from '@/services/search'
const service = getSearchService()
const suggestions = await service.getSuggestions('proj', {
limit: 10,
types: ['messages', 'users', 'channels'],
})Search results include highlighted matches:
{
"id": "msg-123",
"content": "Let's discuss the project update",
"highlight": "Let's discuss the <mark>project</mark> update",
"score": 0.92
}The analytics dashboard consists of:
src/components/analytics/
βββ overview/
β βββ AnalyticsDashboard.tsx # Main dashboard
β βββ AnalyticsSummary.tsx # Summary cards
β βββ AnalyticsCards.tsx # Quick stats
β βββ AnalyticsHeader.tsx # Header with filters
β βββ UsageTrackingDashboard.tsx # Usage tracking
βββ charts/
β βββ MessageVolumeChart.tsx # Message trends
β βββ ActiveUsersChart.tsx # User activity
β βββ ChannelActivityChart.tsx # Channel stats
β βββ ReactionChart.tsx # Popular reactions
β βββ PeakHoursChart.tsx # Activity by hour
β βββ GrowthChart.tsx # User growth
β βββ FileUploadChart.tsx # File uploads
β βββ ResponseTimeChart.tsx # Response times
β βββ UserEngagementChart.tsx # Engagement metrics
βββ tables/
β βββ TopChannelsTable.tsx # Most active channels
β βββ TopUsersTable.tsx # Most active users
β βββ TopMessagesTable.tsx # Popular messages
β βββ InactiveUsersTable.tsx # Inactive users
βββ views/
β βββ MessageAnalytics.tsx # Message-specific view
β βββ UserAnalytics.tsx # User-specific view
β βββ ChannelAnalytics.tsx # Channel-specific view
β βββ FileAnalytics.tsx # File-specific view
β βββ SearchAnalytics.tsx # Search-specific view
β βββ BotAnalytics.tsx # Bot-specific view
βββ export/
βββ AnalyticsExport.tsx # Export functionality
- Total messages (24h, 7d, 30d, all-time)
- Active users (online now, today, this week)
- New channels created
- Storage usage
- Top channels by activity
- Messages over time (line chart)
- Messages by channel (bar chart)
- Messages by user (top 10)
- Message types breakdown (text, image, video, file)
- Peak hours analysis
- User growth (line chart)
- Active users trend (DAU/WAU/MAU)
- User roles breakdown (pie chart)
- Top contributors
- Inactive users (30+ days)
- Storage usage by type
- File uploads over time
- Largest files
- Files by channel
- File type distribution
- Channel activity heatmap
- Most active channels
- Channel types breakdown
- Member growth by channel
import { AnalyticsDashboard } from '@/components/analytics/overview/AnalyticsDashboard';
export default function AnalyticsPage() {
return <AnalyticsDashboard className="container mx-auto py-6" />;
}The dashboard supports filtering by:
- Date range: Last 7/30/90 days, custom range
- Granularity: Hour, day, week, month
- Channels: Filter by specific channels
- Users: Filter by specific users
- Include bots: Toggle bot activity
Ι³Chat tracks usage against plan limits:
| Metric | Free | Starter | Pro | Enterprise |
|---|---|---|---|---|
| Members | 50 | 100 | 500 | Unlimited |
| Channels | 15 | 50 | 200 | Unlimited |
| Storage | 10 GB | 50 GB | 500 GB | Unlimited |
| API Calls | 100k/mo | 500k/mo | 2M/mo | Unlimited |
| Video Minutes | 300/mo | 1k/mo | 5k/mo | Unlimited |
- Safe (0-74%): Green, healthy usage
- Warning (75-99%): Amber, approaching limit
- Critical (100%+): Red, limit reached
import { UsageTrackingDashboard } from '@/components/analytics/overview/UsageTrackingDashboard';
export default function UsagePage() {
const limits = {
plan: 'free',
members: { current: 45, limit: 50 },
channels: { current: 12, limit: 15 },
storage: { current: 8500, limit: 10240 },
apiCalls: { current: 95000, limit: 100000 },
videoMinutes: { current: 280, limit: 300 },
};
return (
<UsageTrackingDashboard
limits={limits}
onUpgrade={() => router.push('/billing/upgrade')}
onExport={(format) => console.log(`Exporting as ${format}`)}
/>
);
}Quick search across all types.
Query Parameters:
-
q(required): Search query -
limit(optional): Results per page (default: 20, max: 100) -
offset(optional): Pagination offset (default: 0)
Response:
{
"success": true,
"data": [...],
"pagination": {
"total": 100,
"page": 1,
"limit": 20,
"hasMore": true
}
}Advanced search with filters.
Request Body:
{
"query": "project update",
"types": ["messages", "files"],
"channelIds": ["channel-id-1"],
"userIds": ["user-id-1"],
"dateFrom": "2024-01-01",
"dateTo": "2024-12-31",
"limit": 20,
"offset": 0,
"sortBy": "date",
"sortOrder": "desc"
}Response:
{
"success": true,
"results": [...],
"totals": {
"messages": 50,
"files": 10,
"users": 5,
"channels": 2,
"total": 67
},
"query": "project update",
"types": ["messages", "files"],
"pagination": {...}
}Get index status and health.
Response:
{
"success": true,
"stats": {
"messages": {
"numberOfDocuments": 1000,
"isIndexing": false
},
...
},
"health": {
"messages": {
"healthy": true,
"pendingTasks": 0,
"failedTasks": 0
},
...
}
}Trigger reindexing (admin only).
Request Body:
{
"indexName": "messages",
"forceRebuild": true
}Clear all indexes (admin only).
Trigger full reindexing of all or specific indexes.
Request Body:
{
"indexNames": ["messages", "files"],
"forceRebuild": true,
"batchSize": 100
}Get reindexing progress.
Response:
{
"success": true,
"isReindexing": true,
"stats": {...},
"health": {...}
}Get aggregated dashboard data.
Query Parameters:
-
preset: Date range preset (last7days, last30days, last90days) -
start: Custom start date (ISO 8601) -
end: Custom end date (ISO 8601) -
granularity: Time granularity (hour, day, week, month) -
channels: Comma-separated channel IDs -
users: Comma-separated user IDs -
includeBots: Include bot activity (true/false)
Response:
{
"success": true,
"data": {
"summary": {...},
"messageVolume": [...],
"activeUsers": {...},
"channelActivity": [...],
...
},
"filters": {...},
"meta": {
"generatedAt": "2024-01-01T00:00:00Z",
"cached": false
}
}Get message-specific analytics.
Get user-specific analytics.
Export analytics data.
Query Parameters:
-
format: Export format (csv, json) -
sections: Comma-separated sections to export -
preset: Date range preset
GraphQL Subscription
β
RealtimeSyncService
β
SyncService
β
MessageIndexer
β
MeiliSearch
import { getRealtimeSyncService } from '@/services/search/realtime-sync'
import { io } from 'socket.io-client'
// Initialize realtime sync
const realtimeSync = getRealtimeSyncService({
enabled: true,
batchInterval: 1000,
maxBatchSize: 50,
debug: false,
})
// Connect to Socket.io
const socket = io('http://localhost:4000')
realtimeSync.connect(socket)
// Subscribe to sync events
const unsubscribe = realtimeSync.onSync((result, event) => {
console.log(`Indexed ${event.type}:`, result)
})
// Later: disconnect
realtimeSync.disconnect()
unsubscribe()The sync service listens for these events:
message:createdmessage:updatedmessage:deletedchannel:createdchannel:updatedchannel:deleteduser:createduser:updateduser:deletedfile:uploadedfile:deleted
import { getSyncService } from '@/services/search'
const syncService = getSyncService()
// Index a single message
await syncService.indexMessage(message, channel, author)
// Batch index messages
await syncService.batchIndexMessages([
{ message, channel, author },
{ message: message2, channel: channel2, author: author2 },
])
// Index other entities
await syncService.indexFile(file)
await syncService.indexUser(user)
await syncService.indexChannel(channel)import { exportFullReport } from '@/lib/analytics/analytics-export'
exportFullReport(data, {
format: 'csv',
sections: ['summary', 'messages', 'users'],
dateRange,
includeCharts: false,
})Generated file: analytics-report-2024-01-01.csv
exportFullReport(data, {
format: 'json',
sections: ['summary', 'messages', 'users'],
dateRange,
includeCharts: false,
})Generated file: analytics-report-2024-01-01.json
import { useAnalyticsStore } from '@/stores/analytics-store'
const { addScheduledReport } = useAnalyticsStore()
addScheduledReport({
id: 'weekly-report',
name: 'Weekly Analytics Report',
schedule: 'weekly',
format: 'csv',
sections: ['summary', 'messages', 'users'],
recipients: ['[email protected]'],
enabled: true,
})- Analytics aggregator: 5-minute cache for dashboard data
- Search results: Client-side caching via SWR
- Index stats: 1-minute cache for index health
- Message indexing: Batched every 1 second or 50 messages
- Bulk reindexing: 100 documents per batch
- API requests: Paginated with max 100 results
| Endpoint | Limit |
|---|---|
GET /api/search |
60/minute |
POST /api/search |
60/minute |
POST /api/search/reindex |
10/minute |
GET /api/analytics/dashboard |
30/minute |
Problem: Search returns no results.
Solutions:
- Check MeiliSearch is running:
docker ps | grep meilisearch - Verify indexes exist:
GET /api/search/index - Trigger reindex:
POST /api/search/reindex
Problem: New messages don't appear in search.
Solutions:
- Check Socket.io connection
- Verify GraphQL subscriptions are active
- Check console for sync errors
Problem: Dashboard shows no data.
Solutions:
- Check analytics collector is running
- Verify database connection
- Clear cache:
analyticsAggregator.clearCache()
Problem: Search takes too long.
Solutions:
- Reduce result limit
- Use specific types filter
- Add more MeiliSearch RAM
- Check index health
- Use specific types when possible to narrow results
- Implement pagination for large result sets
- Provide suggestions for better UX
- Show loading states during searches
- Highlight matches in results
- Use date range presets for common queries
- Implement caching for expensive aggregations
- Show comparison data for context
- Export large datasets instead of displaying all
- Monitor cache hit rates to optimize TTL
- Use batch operations for bulk indexing
- Monitor index health regularly
- Set up alerts for indexing failures
- Reindex periodically to maintain consistency
- Use real-time sync for immediate updates
Last Updated: 2026-02-03 Version: 0.9.1