Logging Styleguide - green-ecolution/backend GitHub Wiki

Introduction

Effective logging is a key part of any software application, providing insight into the application's behavior and helping diagnose issues. This style guide outlines best practices for logging in the Green Ecolution backend application where a logger is implemented and stored in the context. The logger automatically includes essential request information such as request_id, user_id, request duration, and start time.

Logging Best Practices

General Logging Guidelines

Use Contextual Loggers: Always use the logger retrieved from the context using log := logger.GetLogger(ctx). This ensures that all logs contain the necessary request-specific metadata.
Log Levels: Choose the appropriate logging level to reflect the severity and purpose of the log entry.
- Use INFO for general application flow and expected behaviors.
- Use ERROR for issues that require attention or investigation.
- Use DEBUG for detailed information during development or troubleshooting.
- Use WARNING for potentially problematic situations that warrant attention but do not yet constitute errors.
Log Structure: Logs should be structured for easy parsing and analysis. Always include key-value pairs that provide context (e.g., "cluster_id"="1", "error"="some error message").
Avoid Sensitive Data: Ensure that no sensitive information, such as passwords or personal data, is logged.
Consistency: Follow a consistent log message format throughout the application. Include important information such as the action performed, the outcome, and any relevant identifiers.

log.Info("created new tree cluster", "cluster_id", cluster.ID, "user_id", userID)

This shows a clear message and structured key-value data, making it easy to search and analyze in logs.

Error Logging in Layers

Storage Layer

The MapError method in the storage layer is used to translate database errors into application-specific errors. This method should be used cautiously:

func (s *Store) MapError(err error, dbType any) error {
    if err == nil {
        return nil
    }
    rType := reflect.TypeOf(dbType)
    if rType.Kind() == reflect.Pointer {
        rType = rType.Elem()
    }

    var rName string
    switch rType.Kind() {
    case reflect.Struct:
        rName = rType.Name()
    case reflect.String:
        rName = dbType.(string)
    default:
        panic("unreachable")
    }

    if errors.Is(err, pgx.ErrNoRows) {
        return storage.ErrEntityNotFound(rName)
    }

    return err
}

Guidelines:
- Understand the Error: Ensure you understand the error being translated and its implications.
- Appropriate Use: Use MapError only when it makes sense to translate the error. For instance, when pgx.ErrNoRows is encountered, ensure that returning ErrEntityNotFound is the correct behavior for the application context.
- Default Handling: For unhandled cases, return the original error to avoid losing context.
- Don't call it twice: Make sure you only call this function once to fix an error. It is easy to write a helper function where the error is returned by the MapError function, which is then called again.

Service Layer

In the service layer, the MapError method is used to log and map errors based on the context and a provided error mask:

func MapError(ctx context.Context, err error, errorMask ErrorLogMask) error {
    log := logger.GetLogger(ctx)
    var entityNotFoundErr storage.ErrEntityNotFound
    if errors.As(err, &entityNotFoundErr) {
        if errorMask&ErrorLogEntityNotFound == 0 {
            log.Error("can't find entity", "error", err)
        }
        return NewError(NotFound, entityNotFoundErr.Error())
    }

    if errors.Is(err, ErrValidation) {
        if errorMask&ErrorLogValidation == 0 {
            log.Error("failed to validate struct", "error", err)
        }
        return NewError(BadRequest, err.Error())
    }

    log.Error("an error has occurred", "error", err)
    return NewError(InternalError, err.Error())
}

Guidelines:
- Error Masks: Use the errorMask parameter to suppress logging of expected errors. The mask is a bitmask, where each bit corresponds to a specific type of error logging. For example:
  - ErrorLogEntityNotFound: Suppresses logs for EntityNotFound errors when it is expected as part of normal application behavior.
  - ErrorLogValidation: Suppresses logs for validation errors when they are expected.
- Implementation: To apply the mask, perform a bitwise AND operation (errorMask&ErrorLogType == 0). If the result is 0, the error log is allowed; otherwise, it is suppressed.
- Error Context: Always log errors with enough context to aid debugging. Use structured logging to provide key details.
- Error Transformation: Ensure the mapped error aligns with the application's error model. For example, map validation errors to BadRequest and database not-found errors to NotFound.
- Don't call it twice: Ensure that MapError is not called multiple times for the same error. This can lead to redundant error transformations and duplicate logs, making debugging more difficult. Instead, centralize error mapping logic in one place.

Dos and Don’ts

Dos

Use Loggers with Context: Always retrieve the logger from the context to maintain consistency.
Be Intentional with Error Mapping: Clearly understand and document why an error is being mapped in a specific way.
Test Logging: Regularly verify that logs are generated as expected and provide the necessary information.
Document Error Flows: Clearly document which errors are logged and how they are transformed.

Don’ts

Avoid Over-Logging: Do not log every error at the service layer if it’s already logged at the storage layer.
Do Not Suppress Important Errors: Use error masks carefully to avoid unintentionally suppressing critical error logs.
Avoid Panics: Ensure the MapError methods handle edge cases gracefully without panicking.
ERROR Logs Should Only Indicate Errors: Only log at the ERROR level when an actual error occurs. For other scenarios, such as debugging or providing additional context, use the DEBUG level.