logging

---
name: logging
description: >
license: MIT
---

# Logging & Observability

## Core Principles

1. **Structured logging with Serilog** — Every log entry is a structured event with named properties, not a formatted string. This enables searching, filtering, and alerting. All setup (two-stage bootstrap, `AddSerilog()`, sinks, enrichers) lives in the **serilog** skill — that skill's `AddSerilog()`-over-`UseSerilog()` guidance is canonical.
2. **OpenTelemetry for distributed tracing** — Traces connect requests across services; metrics track system health over time. Full setup lives in the **opentelemetry** skill.
3. **Health checks for operational readiness** — Every service exposes `/health` endpoints for load balancers and orchestrators. Liveness and readiness are separate questions and separate endpoints.
4. **Correlation IDs for request tracing** — Every request gets a unique ID that flows through all log entries and downstream service calls, so one user complaint maps to one filtered log stream.

## Patterns

### How the Pieces Fit Together

| Concern | Owner | Skill |
|---------|-------|-------|
| Structured application logs | Serilog (`AddSerilog()`) | `serilog` |
| Request summary logging | `UseSerilogRequestLogging()` | `serilog` |
| Traces + metrics + OTLP export | OpenTelemetry SDK | `opentelemetry` |
| Health endpoints, correlation IDs, log-level strategy | This skill | `logging` |

Wire logging first (you need logs to debug the rest), then health checks, then tracing.

### Correlation IDs

```csharp
// Middleware to set correlation ID
public class CorrelationIdMiddleware(RequestDelegate next)
{
    private const string CorrelationIdHeader = "X-Correlation-Id";

    public async Task InvokeAsync(HttpContext context)
    {
        var correlationId = context.Request.Headers[CorrelationIdHeader].FirstOrDefault()
            ?? Guid.NewGuid().ToString();

        context.Items["CorrelationId"] = correlationId;
        context.Response.Headers[CorrelationIdHeader] = correlationId;

        using (LogContext.PushProperty("CorrelationId", correlationId))
        {
            await next(context);
        }
    }
}

// Program.cs — register early so every downstream log carries the ID
app.UseMiddleware<CorrelationIdMiddleware>();
```

Why middleware: pushing the property once at the pipeline edge attaches it to every log event in the request scope — no per-call-site plumbing. Propagate the same header on outgoing `HttpClient` calls via a `DelegatingHandler` (see the **httpclient-factory** skill).

### Health Checks

```csharp
// Program.cs
builder.Services.AddHealthChecks()
    .AddNpgSql(builder.Configuration.GetConnectionString("Default")!,
        name: "database", tags: ["ready"])
    .AddRedis(builder.Configuration.GetConnectionString("Redis")!,
        name: "redis", tags: ["ready"])
    .AddRabbitMQ(builder.Configuration.GetConnectionString("RabbitMq")!,
        name: "rabbitmq", tags: ["ready"]);

// Map endpoints
app.MapHealthChecks("/health/live", new HealthCheckOptions
{
    Predicate = _ => false // No dependency checks — just "am I running?"
});

app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("ready")
});
```

Why two endpoints: liveness failing means "restart me"; readiness failing means "stop sending traffic". Conflating them makes a slow database restart your app in a loop.

### Log-Level Strategy

| Level | Use for | Environment default |
|-------|---------|---------------------|
| Debug | Diagnostic detail, payload dumps (never PII in prod) | Development only |
| Information | Business events: order placed, job completed | Dev + staging |
| Warning | Recoverable anomalies: retry fired, fallback used | Everywhere — production default |
| Error | Failed operations that need attention | Everywhere |
| Fatal/Critical | App cannot continue | Everywhere |

Why Warning as the production default: Information-level request noise at scale costs real money in log storage and drowns the signals. Keep Information for genuine business events via namespace overrides (see the **serilog** skill's `MinimumLevel.Override` pattern).

## Anti-patterns

### Don't Log Sensitive Data

```csharp
// BAD — logging credentials
logger.LogInformation("User logged in: {Email} with password {Password}", email, password);

// GOOD — log identifiers, never secrets or PII at Information level
logger.LogInformation("User {UserId} logged in", userId);
```

### Don't Skip Health Check Tags

```csharp
// BAD — all checks run for liveness AND readiness
app.MapHealthChecks("/health");

// GOOD — separate liveness (am I running?) from readiness (can I serve traffic?)
app.MapHealthChecks("/health/live", new() { Predicate = _ => false });
app.MapHealthChecks("/health/ready", new() { Predicate = c => c.Tags.Contains("ready") });
```

### Don't Re-Implement What the Owning Skill Provides

```csharp
// BAD — hand-rolling Serilog bootstrap here from memory
builder.Host.UseSerilog(...);  // legacy API — the serilog skill forbids this

// GOOD — load the serilog skill and use its two-stage AddSerilog() bootstrap
builder.Services.AddSerilog((services, lc) => lc.ReadFrom.Configuration(builder.Configuration)...);
```

## Decision Guide

| Scenario | Recommendation |
|----------|---------------|
| Application logging setup | Load `serilog` — `AddSerilog()` two-stage bootstrap |
| Distributed tracing / metrics | Load `opentelemetry` — OTLP exporter |
| Custom business metrics | `IMeterFactory` + counters/histograms (`opentelemetry` skill) |
| Request tracing | Correlation ID middleware (this skill) |
| Container health | `/health/live` and `/health/ready` endpoints (this skill) |
| Log storage | Seq (development), Elastic/Grafana/OTLP backend (production) |
| Log levels | Debug in dev, Information in staging, Warning default in production |