Skip to main content

Observability

OpinionatedEventing emits structured logs, distributed traces, and metrics out of the box. All telemetry uses standard .NET abstractions — no Serilog, Seq, or OpenTelemetry package is required in the library itself.

Logging​

The library uses Microsoft.Extensions.Logging.ILogger<T>. As long as you configure a logging provider in your host, log output flows automatically.

What is logged​

ComponentLevelEvents
OutboxDispatcherWorkerDebugPoll cycle start/end, batch size
OutboxDispatcherWorkerInformationMessage dispatched successfully
OutboxDispatcherWorkerWarningDispatch attempt failed (retrying)
OutboxDispatcherWorkerErrorMessage dead-lettered after MaxAttempts
SagaTimeoutWorkerDebugTimeout check cycle
SagaTimeoutWorkerInformationSaga timed out, timeout handler invoked
SagaTimeoutWorkerErrorTimeout handler threw exception
Transport consumersDebugMessage received
Transport consumersInformationMessage handled successfully
Transport consumersErrorHandler threw unhandled exception

Configuring log levels​

In appsettings.json:

{
"Logging": {
"LogLevel": {
"Default": "Information",
"OpinionatedEventing": "Debug"
}
}
}

Distributed tracing​

The library uses System.Diagnostics.ActivitySource. All activity source names are under the opinionatedeventing prefix. Wire them up via the OpenTelemetry SDK:

using OpinionatedEventing.OpenTelemetry;

services.AddOpenTelemetry()
.WithTracing(tracing => tracing
.AddOpinionatedEventingInstrumentation()
.AddOtlpExporter());

Emitted spans​

Span nameWhenKey attributes
outbox.writeIPublisher writes to the outbox storemessaging.message.type, messaging.message.kind, messaging.message.correlation_id
outbox.dispatchOutboxDispatcherWorker calls ITransport.SendAsyncmessaging.message.id, messaging.message.type
consumeTransport consumer hands off to handler(s)messaging.message.type, messaging.message.kind, messaging.message.correlation_id
saga.stepA saga handler executessaga.type, saga.correlation_key, messaging.message.type

Each span carries W3C trace context propagated through the message envelope so traces span service boundaries.

Viewing traces locally​

When using Aspire, the dashboard includes a built-in trace viewer. Traces from all services are aggregated automatically — no additional exporter configuration is needed for local development.

Metrics​

The library uses System.Diagnostics.Metrics.Meter. Wire up the metrics provider:

using OpinionatedEventing.OpenTelemetry;

services.AddOpenTelemetry()
.WithMetrics(metrics => metrics
.AddOpinionatedEventingMetrics()
.AddOtlpExporter());

Available instruments​

InstrumentTypeDescription
opinionatedeventing.outbox.pendingGaugeCurrent number of pending (unprocessed) outbox messages
opinionatedeventing.outbox.processedCounterTotal messages successfully dispatched to the broker
opinionatedeventing.outbox.failedCounterTotal messages dead-lettered after MaxAttempts
opinionatedeventing.publish.durationHistogramTime (ms) to write a message to the outbox store
opinionatedeventing.dispatch.durationHistogramTime (ms) to dispatch a message to the broker
opinionatedeventing.consume.durationHistogramTime (ms) for the handler to process a message
opinionatedeventing.saga.activeGaugeCurrent number of active saga instances
opinionatedeventing.saga.timed_outCounterTotal sagas that have timed out

Alerting recommendations​

MetricAlert conditionMeaning
opinionatedeventing.outbox.pending> threshold for > 5 minDispatcher is stalled or the broker is unavailable
opinionatedeventing.outbox.failedRate > 0Messages are dead-lettering — investigate errors
opinionatedeventing.consume.durationp99 > SLAHandler is slow — risk of broker timeout and redelivery
opinionatedeventing.saga.activeGrowing without boundSagas are not completing — check timeouts

Health checks​

Health checks are the quickest way to surface observability in an operations dashboard:

services.AddHealthChecks()
.AddOpinionatedEventingHealthChecks(options =>
{
options.OutboxBacklogThreshold = 100; // pending messages before Degraded
options.SagaTimeoutBacklogThreshold = 10; // expired sagas before Degraded
});

app.MapHealthChecks("/health");
app.MapHealthChecks("/health/live", new HealthCheckOptions { Predicate = c => c.Tags.Contains("live") });
app.MapHealthChecks("/health/ready", new HealthCheckOptions { Predicate = c => c.Tags.Contains("ready") });

The built-in checks use two tags:

TagChecks
liveBroker connectivity
readyOutbox backlog, saga timeout backlog

Broker connectivity is a liveness check (live), not readiness: if the broker is unreachable the process is genuinely broken and a restart may help re-establish the connection. Backlog checks are readiness signals — a load balancer can stop routing HTTP traffic to the instance while the background workers drain the queue.

Readiness and broker consumers: readiness probes only control the Kubernetes Service endpoint, which governs HTTP traffic. The broker consumer workers are BackgroundService instances that pull messages directly from the broker, independent of any load balancer. By default, a degraded readiness check does not pause event consumption.

To automatically pause consumers when a dependency becomes unavailable, chain WithConsumerPause() and tag the relevant checks with "pause":

services.AddHealthChecks()
.AddOpinionatedEventingHealthChecks()
.AddNpgsql(connectionString, tags: ["pause"]) // pause consumers when DB is unreachable
.WithConsumerPause();

When any check tagged "pause" reports Degraded or Unhealthy, the consumer workers stop accepting new messages from the broker. They resume automatically once all "pause"-tagged checks recover to Healthy. This is opt-in — the default behaviour (always consuming) is preserved when WithConsumerPause() is not called.

The built-in backlog checks ("ready") intentionally do not carry the "pause" tag. Pausing consumers does not help drain the outbox or saga-timeout backlogs — those are drained by OutboxDispatcherWorker and SagaTimeoutWorker, which are never paused. Wiring backlog checks to consumer pause would cause oscillation: consumers pause → broker queue grows → backlogs eventually drain → consumers resume → new messages create more backlog → repeat.

Correlation and causation IDs​

Every message carries two identifiers that let you reconstruct the full causal chain across services:

  • CorrelationId — Set at the entry point (e.g., an HTTP request ID) and propagated through every event and command in the chain. All spans and log entries include this.
  • CausationId — The Id of the message that caused this one to be sent. Null for root messages. This is how you reconstruct the parent-child relationship between messages.

These IDs are accessible in handlers via IMessagingContext:

public class OrderNotificationHandler(IMessagingContext context) : IEventHandler<OrderPlaced>
{
public Task HandleAsync(OrderPlaced @event, CancellationToken ct)
{
// context.CorrelationId — same across the entire order flow
// context.CausationId — ID of the message that triggered this handler
return Task.CompletedTask;
}
}

Automatic logging scope​

MessageHandlerRunner calls ILogger.BeginScope before invoking handlers, pushing CorrelationId, CausationId, and MessageType as structured properties into the ambient logging scope:

{
"CorrelationId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"CausationId": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
"MessageType": "MyApp.Orders.OrderPlaced, MyApp"
}

Any ILogger<T> used inside a handler automatically inherits these properties — no manual logging required. Logging providers that support structured scopes (OpenTelemetry, Serilog, Application Insights) pick them up and include them on every log entry emitted during handler execution.