Logging Framework, Monitoring and use guide - JU-DEV-Bootcamps/ERAS GitHub Wiki

We use Serilog as our logging framework for the ERAS-BE .NET projects. Serilog provides flexible, structured logging with multiple “sinks” (destinations) like files, console output, and external aggregators (like Seq, Splunk, Elastic Stack)

1. Install Serilog Packages

   • In your .csproj, add:

     <PackageReference Include="Serilog.AspNetCore" Version="7.*" />
     <PackageReference Include="Serilog.Sinks.Console" Version="4.*" />
     <PackageReference Include="Serilog.Sinks.File" Version="5.*" />

     If you need an aggregator (Seq, etc.), add the corresponding sink package (e.g., Serilog.Sinks.Seq)

2. Configure Serilog in Program.cs

   using Serilog;
   using Serilog.Events;
   
   var builder = WebApplication.CreateBuilder(args);
   
   var logger = new LoggerConfiguration()
       .MinimumLevel.Debug()  // or use environment-based logic
       .WriteTo.Console()
       .WriteTo.File("Logs/log-.log", rollingInterval: RollingInterval.Day)
       .CreateLogger();
   
   builder.Host.UseSerilog(logger);

3. Add Log Statements

   • Inject ILogger<T> into your controllers/services
   • Use _logger.LogInformation(), _logger.LogWarning(), _logger.LogError(), etc. in appropriate places

4. Environment-Based Log Levels

   • Set more verbose levels (Debug/Information) in Development and higher levels (Warning/Error) in Production

• A log aggregator (like Seq, Splunk, or Elastic) centralizes all logs from multiple containers/instances
• You can query, filter, and visualize logs for better observabilit

1. Run Seq (either locally via Docker or as a cloud-hosted instance).

   docker run -d --name seq -p 5341:80 datalust/seq:latest

2. Add the Seq Sink
   In your .csproj:

   <PackageReference Include="Serilog.Sinks.Seq" Version="5.*" />

3. Configure Seq in Program.cs

   var seqUrl = builder.Configuration.GetValue<string>("Seq:Url") 
                ?? "http://localhost:5341"; // or from ENV variable
   
   var logger = new LoggerConfiguration()
       .MinimumLevel.Debug()
       .WriteTo.Console()
       .WriteTo.Seq(seqUrl)
       .CreateLogger();

4. Provide Seq:ApiKey if needed.
5. Test: Run the app, generate logs, then view them at http://localhost:5341.

• For Splunk: Use Serilog.Sinks.Splunk.
• For Elastic Stack: Use Serilog.Sinks.Elasticsearch.

• Trace/Debug: Very fine-grained events (typically hidden in production)
• Information: General business flow (e.g., “User {UserId} logged in”, “Data imported successfully”)
• Warning: Potentially harmful situations (e.g., “Could not find user in cache, falling back to DB”)
• Error: Exceptions or failures that affect the current operation (but the service can recover)
• Critical: The system or a major subsystem is down or significantly impaired

Tip: Keep logs readable and meaningful—avoid excessive noise in production logs

1. Be Descriptive:
   • “User {UserId} attempted login” is better than “Login attempt.”
   • Include context: IDs, relevant parameters, or correlation IDs
2. Use Structured Logging:
   • LogInformation("User {UserId} logged in", userId)
   • Avoid string concatenation—structured logs are more query-friendly in aggregators
3. Never Log Sensitive Data:
   • Don’t log passwords, tokens, or PII in plain text

• Controllers:

  • Log the entry into an endpoint, key parameters, and possible errors or unusual conditions.
  • Rely on global exception middleware for unhandled exceptions.

• Services / Command Handlers:

  • Log major business events (e.g., order created, payment processed).
  • Log unexpected errors that might occur in business logic or external API calls.

• Repositories:

  • Generally, rely on EF Core logs or log only if you have complex operations that might fail in unusual ways.

• On Success:

  _logger.LogInformation("Successfully processed {Entity} for User {UserId}", entityName, userId);

• On Failure:

  _logger.LogError(ex, "Error processing {Entity} for User {UserId}", entityName, userId);

• Warnings:

  _logger.LogWarning("Unable to cache result for {UserId}. Falling back to DB.", userId);

public class StudentsController : ControllerBase
{
    private readonly ILogger<StudentsController> _logger;
    // ...
    public StudentsController(ILogger<StudentsController> logger)
    {
        _logger = logger;
    }

    [HttpPost]
    public IActionResult ImportStudents(...)
    {
        _logger.LogInformation("Importing students at {DateTime}", DateTime.UtcNow);
        try
        {
            // ... import logic
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to import students");
            return StatusCode(500);
        }
        return Ok();
    }
}

• Development: LogEventLevel.Debug or Information
• Production: LogEventLevel.Warning or Error
• Configure this via appsettings.{Environment}.json or in code:

  LogEventLevel minimum = builder.Environment.IsDevelopment() 
      ? LogEventLevel.Debug : LogEventLevel.Warning;