Updated README.md

Added more telemetry tooling
2026-06-22 18:58:32 +02:00
parent b5b2925837
commit 1560e282b5
33 changed files with 897 additions and 129 deletions
@@ -30,6 +30,27 @@ It features a rich fluent API for routing log events to dozens of destinations -
 - **Memory flow** - Store recent logs in a circular in-memory buffer for quick diagnostics or fallback output.
 - **Lazy initialization** - Flows are only initialized when first used, reducing startup overhead.

+
+### Telemetry & Observability
+
+EonaCat.LogStack includes built-in telemetry primitives for production monitoring:
+
+- **Metrics registry** - Low-overhead counters and snapshots for logging throughput.
+- **Telemetry snapshots** - Capture service name, environment, runtime/process metrics, log counters and trace activity.
+- **Runtime health data** - Export operational status for dashboards and monitoring systems.
+- **Tracing support** - Activity/span integration for distributed request correlation.
+- **Custom exporters** - Build exporters around telemetry snapshots for your monitoring backend.
+
+Example:
+```csharp
+var options = new TelemetryOptions
+{
+    Enabled = true,
+    IncludeRuntimeMetrics = true,
+    IncludeTraceMetrics = true
+};
+```
+
 ## Supported Targets

 - .NET Standard 2.1
@@ -1496,6 +1517,55 @@ await using var logger = new LogBuilder("HighVolumeApp")
    .Build();
 ```

+
+## OpenTelemetry-style Telemetry
+
+EonaCat.LogStack includes a dependency-free telemetry layer inspired by OpenTelemetry concepts.
+
+### Distributed tracing
+
+```csharp
+var source = new TelemetryActivitySource();
+using var span = source.StartScope("ProcessOrder", telemetry =>
+{
+    telemetry.Attributes["order.id"] = "123";
+});
+```
+
+Supported tracing concepts:
+- Trace IDs and Span IDs
+- Parent span relationships
+- Span duration measurement
+- Span attributes
+- Span events
+- Status tracking
+
+### Metrics
+
+```csharp
+var meter = new TelemetryMeter();
+meter.Count("requests.total");
+meter.Record("request.duration", 25.5);
+var p95 = meter.GetHistogram("request.duration")?.Percentile(0.95);
+```
+
+Telemetry features:
+- Counters
+- Histograms
+- Percentiles
+- Runtime snapshots
+- Resource attributes
+- Baggage/context propagation
+- HTTP telemetry ingestion
+
+### Resource metadata
+
+```csharp
+var resource = new TelemetryResource()
+    .Add("service.name", "OrderService")
+    .Add("deployment.environment", "production");
+```
+
 ## Diagnostics

 ### Real-Time Metrics
@@ -2531,6 +2601,55 @@ services.AddEonaCatLogging("ProductionApp", builder =>
        .BoostWithUser();
 });
 ```
+
+## OpenTelemetry-style Telemetry
+
+EonaCat.LogStack includes a dependency-free telemetry layer inspired by OpenTelemetry concepts.
+
+### Distributed tracing
+
+```csharp
+var source = new TelemetryActivitySource();
+using var span = source.StartScope("ProcessOrder", telemetry =>
+{
+    telemetry.Attributes["order.id"] = "123";
+});
+```
+
+Supported tracing concepts:
+- Trace IDs and Span IDs
+- Parent span relationships
+- Span duration measurement
+- Span attributes
+- Span events
+- Status tracking
+
+### Metrics
+
+```csharp
+var meter = new TelemetryMeter();
+meter.Count("requests.total");
+meter.Record("request.duration", 25.5);
+var p95 = meter.GetHistogram("request.duration")?.Percentile(0.95);
+```
+
+Telemetry features:
+- Counters
+- Histograms
+- Percentiles
+- Runtime snapshots
+- Resource attributes
+- Baggage/context propagation
+- HTTP telemetry ingestion
+
+### Resource metadata
+
+```csharp
+var resource = new TelemetryResource()
+    .Add("service.name", "OrderService")
+    .Add("deployment.environment", "production");
+```
+
 ## Diagnostics

 ```csharp
@@ -2792,4 +2911,335 @@ The logger is registered as a Singleton in the DI container, so it will be autom
 ```csharp
 var loggerFactory = app.Services.GetRequiredService<ILoggerFactory>();
 await loggerFactory.DisposeAsync();
-```
+```
+
+## Telemetry Enhancements
+
+The telemetry subsystem provides production observability features:
+
+- **Runtime health snapshots** - expose processed events, failures, active spans, uptime and custom health checks.
+- **Telemetry exporters** - plug in exporters for Prometheus, OpenTelemetry collectors, dashboards or custom monitoring platforms.
+- **Telemetry snapshots** - collect metrics, resource information and health state into a single export model.
+- **Custom instrumentation support** - extend telemetry with application-specific counters and tracing data.
+
+Example:
+
+```csharp
+var snapshot = new TelemetrySnapshot
+{
+    Metrics = logger.GetMetrics(),
+    Health = new TelemetryHealth
+    {
+        IsHealthy = true,
+        Checks = { ["storage"] = "ok" }
+    }
+};
+
+await exporter.ExportAsync(snapshot);
+```
+
+
+## Telemetry
+
+EonaCat.LogStack includes an enhanced telemetry layer focused on operational intelligence:
+
+- **Adaptive telemetry engine** - dynamically tracks signals and adjusts analysis based on runtime behavior.
+- **Built-in anomaly detection** - detects unusual latency, throughput, and value deviations without external analysis systems.
+- **Zero-dependency telemetry snapshots** - capture complete runtime state without requiring a collector.
+- **Signal intelligence** - calculates count, min, max, averages, and runtime trends for every signal.
+- **Telemetry-first logging correlation** - combines logs, metrics, traces, and context into a single pipeline.
+- **Runtime diagnostics** - exposes health, performance, and flow statistics from the same engine.
+- **Self-contained observability** - designed to work without mandatory agents or external telemetry infrastructure.
+
+Example:
+
+```csharp
+var telemetry = new AdaptiveTelemetryEngine();
+
+telemetry.Record("request.duration.ms", 42);
+telemetry.Record("cache.hit.rate", 0.98);
+
+var snapshot = telemetry.Snapshot();
+```
+
+
+
+---
+
+# Additional Usage Guide
+
+This section expands on practical usage patterns, production configuration, telemetry, diagnostics, and operational guidance.
+
+## Complete Application Startup Example
+
+A typical production application can configure logging once during startup and reuse the same logger instance:
+
+```csharp
+await using var logger = new LogBuilder("InventoryService")
+    .WithMinimumLevel(LogLevel.Information)
+    .WithTimestampMode(TimestampMode.Utc)
+
+    .WriteToConsole(useColors: true)
+
+    .WriteToFile(
+        directory: "./logs",
+        filePrefix: "inventory",
+        maxFileSize: 250 * 1024 * 1024,
+        compression: CompressionFormat.GZip,
+        batchSize: 100)
+
+    .WriteToElasticSearch(
+        elasticSearchUrl: "https://elastic.example.com:9200",
+        indexName: "inventory")
+
+    .BoostWithApplication("InventoryService", "3.0.0")
+    .BoostWithEnvironment("Production")
+    .BoostWithMachineName()
+    .BoostWithProcessId()
+    .BoostWithCorrelationId()
+
+    .Build();
+
+logger.Information("Service started");
+
+await RunApplicationAsync();
+
+await logger.FlushAsync();
+```
+
+## Telemetry Explained
+
+Telemetry in EonaCat.LogStack provides operational insight into the logging pipeline itself.
+
+Telemetry can answer questions such as:
+
+- How many log events were created?
+- Are any events being dropped?
+- Which flow is slow?
+- Are remote sinks failing?
+- How much memory is the process using?
+- How long are operations taking?
+
+### Enable Telemetry
+
+```csharp
+var telemetry = new TelemetryOptions
+{
+    Enabled = true,
+    IncludeRuntimeMetrics = true,
+    IncludeTraceMetrics = true
+};
+```
+
+### Runtime Metrics
+
+Runtime metrics include information such as:
+
+- Process uptime
+- Memory usage
+- Garbage collection information
+- Thread information
+- CPU/runtime counters where available
+
+Example:
+
+```csharp
+var snapshot = logger.GetTelemetrySnapshot();
+
+Console.WriteLine(snapshot.Runtime.ProcessId);
+Console.WriteLine(snapshot.Runtime.MemoryUsage);
+Console.WriteLine(snapshot.Runtime.Uptime);
+```
+
+## Flow Metrics
+
+Every flow maintains statistics.
+
+Example:
+
+```csharp
+var diagnostics = logger.GetDiagnostics();
+
+foreach (var flow in diagnostics.FlowStats)
+{
+    Console.WriteLine(flow.Name);
+    Console.WriteLine($"Processed: {flow.Processed}");
+    Console.WriteLine($"Dropped: {flow.Dropped}");
+    Console.WriteLine($"Errors: {flow.ErrorCount}");
+}
+```
+
+Useful production alerts:
+
+```csharp
+if (diagnostics.TotalDropped > 0)
+{
+    logger.Warning(
+        "Logs are being dropped",
+        ("Dropped", diagnostics.TotalDropped));
+}
+```
+
+## Distributed Tracing
+
+Correlation IDs connect logs from the same request or operation.
+
+Enable it:
+
+```csharp
+new LogBuilder("Api")
+    .BoostWithCorrelationId()
+    .Build();
+```
+
+A request might then produce:
+
+```
+TraceId=abc123
+Request started
+
+TraceId=abc123
+Database query executed
+
+TraceId=abc123
+Request completed
+```
+
+This makes debugging distributed applications easier.
+
+## Custom Telemetry Export
+
+Telemetry snapshots can be forwarded to custom monitoring systems.
+
+Example:
+
+```csharp
+var snapshot = logger.GetTelemetrySnapshot();
+
+SendToMonitoringSystem(new
+{
+    Service = snapshot.ServiceName,
+    Environment = snapshot.Environment,
+    Metrics = snapshot.Metrics
+});
+```
+
+## Logging HTTP Requests
+
+Example request logging:
+
+```csharp
+logger.Information(
+    "HTTP request completed",
+    ("Method", "GET"),
+    ("Path", "/api/products"),
+    ("StatusCode", 200),
+    ("DurationMs", 42));
+```
+
+Output:
+
+```
+HTTP request completed
+Method=GET
+Path=/api/products
+StatusCode=200
+DurationMs=42
+```
+
+## Recommended Production Layout
+
+A common production setup:
+
+```
+Application
+ |
+ +--> Console (developer visibility)
+ |
+ +--> Local File (fallback)
+ |
+ +--> Elasticsearch (search)
+ |
+ +--> Audit Flow (compliance)
+ |
+ +--> Slack/Teams (alerts)
+ |
+ +--> Diagnostics (monitoring)
+```
+
+Example:
+
+```csharp
+await using var logger = new LogBuilder("Payments")
+
+    .WriteToConsole()
+
+    .WriteToFile("./logs")
+
+    .WriteToAudit(
+        directory: "./audit",
+        auditLevel: AuditLevel.WarningAndAbove)
+
+    .WriteToThrottled(
+        inner: new SlackFlow(slackUrl),
+        minimumLevel: LogLevel.Error)
+
+    .WriteDiagnostics(
+        snapshotInterval: TimeSpan.FromMinutes(1))
+
+    .BoostWithCorrelationId()
+    .Build();
+```
+
+## Shutdown Handling
+
+Always flush buffered logs during shutdown:
+
+```csharp
+try
+{
+    await Application.RunAsync();
+}
+finally
+{
+    await logger.FlushAsync();
+    await logger.DisposeAsync();
+}
+```
+
+## Debugging Logging Problems
+
+Check diagnostics first:
+
+```csharp
+var stats = logger.GetDiagnostics();
+
+Console.WriteLine(
+    $"Written: {stats.TotalLogged}");
+
+Console.WriteLine(
+    $"Dropped: {stats.TotalDropped}");
+```
+
+If events are missing:
+
+1. Check minimum log levels.
+2. Check flow filters.
+3. Check dropped counters.
+4. Check remote destination connectivity.
+5. Enable retry/failover.
+
+## Summary
+
+EonaCat.LogStack combines:
+
+- High performance logging
+- Structured events
+- Multi-target flows
+- Runtime telemetry
+- Distributed tracing support
+- Diagnostics
+- Resilience features
+- Security-focused audit logging
+
+The library is designed to scale from small console tools to distributed production services.