EonaCat.LogStack/README.md

# EonaCat.LogStack

**EonaCat.LogStack** flow-based logging library for .NET, designed for zero-allocation logging paths and superior memory efficiency. 
It features a rich fluent API for routing log events to dozens of destinations - from console and file to Slack, Discord, Redis, Elasticsearch, and beyond.

---

## Features

- **Flow-based architecture** - route log events to one or many output destinations simultaneously
- **Booster system** - enrich every log event with contextual metadata (machine name, process ID, thread info, memory, uptime, correlation IDs, and more)
- **Pre-build modifiers** - intercept and mutate log events before they are written
- **Zero-allocation hot path** - `AggressiveInlining` throughout, `StringBuilder` pooling, and `ref`-based builder pattern
- **Async-first** - all flows implement `IAsyncDisposable` and `FlushAsync`
- **Resilience built-in** - retry, failover, throttling, and rolling buffer flows
- **Tamper-evident audit trail** - SHA-256 hash-chained audit files
- **Encrypted file logging** - AES-encrypted log files with a built-in decrypt utility
- **Compression** - GZip-compressed rolled log files
- **Category routing** - split logs into separate files per category or log level
- **Diagnostics** - live counters (total logged, total dropped, per-flow stats)

---

## Supported Targets

- .NET Standard 2.1
- .NET 8.0
- .NET Framework 4.8

---

## Installation

```bash
dotnet add package EonaCat.LogStack
```

---

## Quick Start

```csharp
await using var logger = LogBuilder.CreateDefault("MyApp");

logger.Information("Application started");
logger.Warning("Low memory warning");
logger.Error(ex, "Unexpected error occurred");
```

`CreateDefault` creates a logger writing to both the console and a `./logs` directory, enriched with machine name and process ID.

---

## Fluent Configuration

Build a fully customised logger using `LogBuilder`:

```csharp
await using var logger = new LogBuilder("MyApp")
    .WithMinimumLevel(LogLevel.Debug)
    .WithTimestampMode(TimestampMode.Utc)
    .WriteToConsole(useColors: true)
    .WriteToFile("./logs", filePrefix: "app", maxFileSize: 50 * 1024 * 1024)
    .WriteToSlack("https://hooks.slack.com/services/...")
    .BoostWithMachineName()
    .BoostWithProcessId()
    .BoostWithCorrelationId()
    .Build();
```

---

## Logging Methods

```csharp
logger.Trace("Verbose trace message");
logger.Debug("Debug detail");
logger.Information("Something happened");
logger.Warning("Potential problem");
logger.Warning(ex, "Warning with exception");
logger.Error("Something failed");
logger.Error(ex, "Error with exception");
logger.Critical("System is going down");
logger.Critical(ex, "Critical failure");

// With structured properties
logger.Log(LogLevel.Information, "User logged in",
    ("UserId", 42),
    ("IP", "192.168.1.1"));
```

---

## Available Flows
### Flows can be extended with custom implementations of `IFlow`, but here are the built-in options:

| Flow | Method | Description |
|------|--------|-------------|
| Console | `WriteToConsole()` | Colored console output |
| File | `WriteToFile()` | Batched, rotated, compressed file output |
| Encrypted File | `WriteToEncryptedFile()` | AES-encrypted log files |
| Memory | `WriteToMemory()` | In-memory ring buffer |
| Audit | `WriteToAudit()` | Tamper-evident hash-chained audit trail |
| Database | `WriteToDatabase()` | ADO.NET database sink |
| HTTP | `WriteToHttp()` | Generic HTTP endpoint (batched) |
| Webhook | `WriteToWebhook()` | Generic webhook POST |
| Email | `WriteToEmail()` | HTML digest emails via SMTP |
| Slack | `WriteToSlack()` | Slack incoming webhooks |
| Discord | `WriteToDiscord()` | Discord webhooks |
| Microsoft Teams | `WriteToMicrosoftTeams()` | Teams incoming webhooks |
| Telegram | `WriteToTelegram()` | Telegram bot messages |
| SignalR | `WriteToSignalR()` | Real-time SignalR hub push |
| Redis | `RedisFlow()` | Redis Pub/Sub + optional List persistence |
| Elasticsearch | `WriteToElasticSearch()` | Elasticsearch index |
| Splunk | `WriteToSplunkFlow()` | Splunk HEC |
| Graylog | `WriteToGraylogFlow()` | GELF over UDP or TCP |
| Syslog UDP | `WriteToSyslogUdp()` | RFC-5424 Syslog over UDP |
| Syslog TCP | `WriteToSyslogTcp()` | RFC-5424 Syslog over TCP (with optional TLS) |
| TCP | `WriteToTcp()` | Raw TCP (with optional TLS) |
| UDP | `WriteToUdp()` | Raw UDP datagrams |
| SNMP Trap | `WriteToSnmpTrap()` | SNMP v2c traps |
| Zabbix | `WriteToZabbixFlow()` | Zabbix trapper protocol |
| EventLog | `WriteToEventLogFlow()` | Remote event log forwarding |
| Rolling Buffer | `WriteToRollingBuffer()` | Circular buffer with trigger-based flush |
| Throttled | `WriteToThrottled()` | Token-bucket rate limiting + deduplication |
| Retry | `WriteToRetry()` | Automatic retry with exponential back-off |
| Failover | `WriteToFailover()` | Primary/secondary failover |
| Diagnostics | `WriteDiagnostics()` | Periodic diagnostic snapshots |
| Status | `WriteToStatusFlow()` | Service health monitoring |

---

## Available Boosters

Boosters enrich every log event with additional properties before it reaches any flow.

```csharp
new LogBuilder("MyApp")
    .BoostWithMachineName()       // host name
    .BoostWithProcessId()         // PID
    .BoostWithThreadId()          // managed thread ID
    .BoostWithThreadName()        // thread name
    .BoostWithUser()              // current OS user
    .BoostWithApp()               // app name and base directory
    .BoostWithApplication("MyApp", "2.0.0")  // explicit name + version
    .BoostWithEnvironment("Production")
    .BoostWithOS()                // OS description
    .BoostWithFramework()         // .NET runtime description
    .BoostWithMemory()            // working set in MB
    .BoostWithUptime()            // process uptime in seconds
    .BoostWithProcStart()         // process start time
    .BoostWithDate()              // current date (yyyy-MM-dd)
    .BoostWithTime()              // current time (HH:mm:ss.fff)
    .BoostWithTicks()             // current timestamp ticks
    .BoostWithCorrelationId()     // Activity.Current correlation ID
    .BoostWithCustomText("env", "prod")   // arbitrary key/value
    .Boost("myBooster", () => new Dictionary<string, object?> { ["key"] = "val" })
    ...
```

---

## Pre-Build Modifiers

Modifiers run after boosters and can mutate or cancel a log event before it is dispatched to flows:

```csharp
logger.AddModifier((ref LogEventBuilder builder) =>
{
    builder.WithProperty("RequestId", Guid.NewGuid().ToString());
});
```

---

## Resilience Patterns

### Retry with exponential back-off
```csharp
.WriteToRetry(
    primaryFlow: new HttpFlow("https://logs.example.com"),
    maxRetries: 5,
    initialDelay: TimeSpan.FromMilliseconds(200),
    exponentialBackoff: true)
```

### Primary / secondary failover
```csharp
.WriteToFailover(
    primaryFlow: new ElasticSearchFlow("https://es-prod:9200"),
    secondaryFlow: new FileFlow("./fallback-logs"))
```

### Token-bucket throttling with deduplication
```csharp
.WriteToThrottled(
    inner: new SlackFlow(webhookUrl),
    burstCapacity: 10,
    refillPerSecond: 1.0,
    deduplicate: true,
    dedupWindow: TimeSpan.FromSeconds(60))
```

### Rolling buffer - flush context on error
```csharp
.WriteToRollingBuffer(
    capacity: 500,
    triggerLevel: LogLevel.Error,
    triggerTarget: new FileFlow("./error-context"))
```

---

## Encrypted File Logging

```csharp
.WriteToEncryptedFile("./secure-logs", password: "s3cr3t")
```

To decrypt later:
```csharp
LogBuilder.DecryptFile(
    encryptedPath: "./secure-logs/log.enc",
    outputPath:    "./secure-logs/log.txt",
    password:      "s3cr3t");
```

---

## Audit Trail

The audit flow produces a tamper-evident file where every entry is SHA-256 hash-chained. Deleting or modifying any past entry invalidates all subsequent hashes.

```csharp
.WriteToAudit(
    directory: "./audit",
    auditLevel: AuditLevel.WarningAndAbove,
    includeProperties: true)
```

Verify integrity at any time:
```csharp
bool intact = AuditFlow.Verify("./audit/audit.audit");
```

---

## Log Message Template

Both `ConsoleFlow` and `FileFlow` accept a customisable template string:

```
[{ts}] [{tz}] [Host: {host}] [Category: {category}] [Thread: {thread}] [{logtype}] {message}{props}
```

| Token | Description |
|-------|-------------|
| `{ts}` | Timestamp (yyyy-MM-dd HH:mm:ss.fff) |
| `{tz}` | Timezone (UTC or local name) |
| `{host}` | Machine name |
| `{category}` | Logger category |
| `{thread}` | Managed thread ID |
| `{pid}` | Process ID |
| `{logtype}` | Log level label (INFO, WARN, ERROR, …) |
| `{message}` | Log message text |
| `{props}` | Structured properties as key=value pairs |
| `{newline}` | Line break |

---

## Diagnostics

```csharp
var diag = logger.GetDiagnostics();
Console.WriteLine($"Logged: {diag.TotalLogged}, Dropped: {diag.TotalDropped}");
```

---

## Flushing and Disposal

```csharp
// Flush all pending events
await logger.FlushAsync();

// Dispose (flushes automatically)
await logger.DisposeAsync();
```

---

## Events

```csharp
logger.OnLog += (sender, msg) =>
{
    // Fired for every log event that passes filters
    Console.WriteLine($"[Event] {msg.Level}: {msg.Message}");
};
```

---

## Custom Flows

Implement `IFlow` (or extend `FlowBase`) to create your own destination:

```csharp
public class MyFlow : FlowBase
{
    public MyFlow() : base("MyFlow", LogLevel.Trace) { }

    public override Task<WriteResult> BlastAsync(LogEvent logEvent, CancellationToken ct = default)
    {
        // Write logEvent somewhere
        return Task.FromResult(WriteResult.Success);
    }

    public override Task FlushAsync(CancellationToken ct = default) => Task.CompletedTask;
}

// Register with:
new LogBuilder("App").WriteTo(new MyFlow()).Build();
```



# EonaCat.LogStack.Server

A lightweight, multi-transport log server for the EonaCat LogStack ecosystem.

## Quick start

```csharp
// Minimal - UDP on port 5555
var server = new Server();
await server.Start();

// Full control via ServerOptions
var server = new Server(new ServerOptions
{
    UseTcp            = true,
    UseUdp            = true,
    UseHttp           = true,       // enables POST /ingest + GET /metrics
    Port              = 5555,
    HttpPort          = 5556,
    MinimumLevel      = ServerLogLevel.Information,   // drop Debug/Trace
    RateLimitPerSecond = 100,       // per remote endpoint
    LogRetentionDays  = 30,
    MaxLogDirectorySize = 10L * 1024 * 1024 * 1024,  // 10 GB
    LogsRootDirectory = "logs",
});

server.LogWritten += line => Console.WriteLine("[written] " + line);
server.LogDropped += line => Console.WriteLine("[dropped] " + line);

await server.Start();
```

## Transports

| Transport | Default | Notes |
|-----------|---------|-------|
| TCP | enabled | Streams until connection closes |
| UDP | enabled | Max 65 507 bytes per packet |
| HTTP | disabled | Enable via `UseHttp = true` |

TCP and UDP can run simultaneously on the same port.

## HTTP endpoints (when `UseHttp = true`)

| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/ingest` | Accept a JSON log entry or array |
| `GET`  | `/metrics` | Return live server metrics as JSON |

### POST /ingest - single entry
```json
{ "level": "info", "message": "Hello world", "source": "MyApp", "host": "srv-01" }
```

### POST /ingest - batch
```json
[
  { "level": "warn",  "message": "Disk at 80%",  "source": "monitor" },
  { "level": "error", "message": "DB timeout",   "source": "api", "exception": "TimeoutException…" }
]
```

### GET /metrics response
```json
{
  "totalReceived": 12345,
  "totalWritten": 12300,
  "totalDropped": 45,
  "totalBytes": 4096000,
  "activeTcpConnections": 3,
  "uptimeSeconds": 3600,
  "startedAt": "2026-03-27T08:00:00Z"
}
```

## Structured JSON parsing

If the incoming payload is valid JSON the server parses it and formats each
entry before writing:

```
[2026-03-27T09:15:00Z] [ERROR] [MyApp] host=srv-01 trace=abc123 Something went wrong
  EXCEPTION: System.TimeoutException: The operation timed out.
```

Recognised JSON fields:

| Field | Aliases | Description |
|-------|---------|-------------|
| `timestamp` | - | ISO-8601 timestamp |
| `level` | `Level`, `severity`, `Severity` | Log level string |
| `message` | `Message` | Log message |
| `source` | `application` | App / service name |
| `host` | - | Hostname |
| `traceId` | - | Distributed trace ID |
| `exception` | - | Exception string |

Plain-text payloads are written as-is and always bypass the level filter.

## Log level filtering

```csharp
MinimumLevel = ServerLogLevel.Warning  // only Warning / Error / Critical are stored
```

Levels in order: `Trace → Debug → Information → Warning → Error → Critical`

## Rate limiting

```csharp
RateLimitPerSecond = 100  // per remote IP:port, 0 = disabled
```

Dropped messages are counted in `Metrics.TotalDropped` and raise the `LogDropped` event.

## Metrics

```csharp
var m = server.GetMetrics();
Console.WriteLine($"Written={m.TotalWritten} Dropped={m.TotalDropped} Uptime={m.Uptime}");
```

## Log file layout

```
logs/
  20260327/
    EonaCatLogs.log          ← active file (≤ 200 MB)
    EonaCatLogs_1.log        ← rolled over
  20260326/
    EonaCatLogs.log
```

Daily directories older than `LogRetentionDays` are deleted automatically.
The total directory is also capped at `MaxLogDirectorySize`.

## Events

```csharp
server.LogWritten += line => NotifyDashboard(line);
server.LogDropped += line => Metrics.Increment("dropped");
```

## Graceful shutdown

```csharp
Console.CancelKeyPress += (_, e) => { e.Cancel = true; server.Stop(); };
```

`Stop()` prints a throughput summary and disposes all listeners cleanly.


## Dependency Injection (DI) registration methods

### 1. Basic Registration (Simplest)

Register with default settings:

```csharp
services.AddEonaCatLogging();
```

This registers:
- `ILoggerFactory` - For creating category-specific loggers
- `Microsoft.Extensions.Logging.ILoggerFactory` - For Microsoft.Extensions.Logging compatibility
- `ILogger` - For injecting the default logger

### 2. Registration with Log Level and Timestamp Mode

```csharp
services.AddEonaCatLogging(
    minimumLevel: LogLevel.Information,
    timestampMode: TimestampMode.Local);
```

### 3. Registration with Pre-built EonaCatLogStack

If you've already created an `EonaCatLogStack` instance:

```csharp
var logStack = new EonaCatLogStack("MyApp");
logStack.AddFlow(new ConsoleFlow());

services.AddEonaCatLogging(logStack);
```

### 4. Registration with Configuration Callback (Recommended)

Configure the logger directly with an `Action<EonaCatLogStack>`:

```csharp
services.AddEonaCatLogging(logStack =>
{
    logStack.AddFlow(new ConsoleFlow());
    logStack.AddFlow(new FileFlow("./logs"));
    logStack.AddBooster(new MachineNameBooster());
});
```

### 5. Registration with LogBuilder (Most Fluent - Recommended)

Use the fluent `LogBuilder` API for the most intuitive configuration:

```csharp
services.AddEonaCatLogging("MyApplication", builder =>
{
    builder
        .WithMinimumLevel(LogLevel.Information)
        .WithTimestampMode(TimestampMode.Local)
        .WriteToConsole()
        .WriteToFile("./logs")
        .BoostWithMachineName()
        .BoostWithProcessId()
        .BoostWithCorrelationId();
});
```

Or with the default "Application" category:

```csharp
services.AddEonaCatLogging(builder =>
{
    builder
        .WriteToConsole()
        .WriteToFile("./logs");
});
```

### 6. Registration with Factory Method (Advanced)

For advanced scenarios where you need access to the service provider:

```csharp
services.AddEonaCatLoggingFactory("MyApplication", builder =>
{
    builder
        .WriteToConsole()
        .WriteToFile("./logs");
});
```

## Using in Your Application

### Injecting ILoggerFactory

```csharp
public class MyService
{
    private readonly ILoggerFactory _loggerFactory;

    public MyService(ILoggerFactory loggerFactory)
    {
        _loggerFactory = loggerFactory;
    }

    public void DoSomething()
    {
        var logger = _loggerFactory.CreateLogger("MyService");
        logger.Log(LogLevel.Information, "Doing something");
    }
}
```

### Injecting ILogger

```csharp
public class MyService
{
    private readonly ILogger _logger;

    public MyService(ILogger logger)
    {
        _logger = logger;
    }

    public void DoSomething()
    {
        _logger.Log(LogLevel.Information, "Doing something");
    }
}
```

### Using with Microsoft.Extensions.Logging.ILogger

```csharp
public class MyService
{
    private readonly Microsoft.Extensions.Logging.ILogger _logger;

    public MyService(Microsoft.Extensions.Logging.ILogger logger)
    {
        _logger = logger;
    }

    public void DoSomething()
    {
        _logger.LogInformation("Doing something");
    }
}
```

## ASP.NET Core / Razor Pages Integration

In your `Program.cs`:

```csharp
var builder = WebApplication.CreateBuilder(args);

// Add EonaCat LogStack to the service collection
builder.Services.AddEonaCatLogging("WebApplication", logBuilder =>
{
    logBuilder
        .WithMinimumLevel(LogLevel.Information)
        .WriteToConsole(useColors: true)
        .WriteToFile("./logs")
        .BoostWithCorrelationId()
        .BoostWithThreadId();
});

// Rest of your configuration...
var app = builder.Build();

// Configure HTTP request pipeline...
app.Run();
```

### In Razor Page Code-Behind

```csharp
public class IndexModel : PageModel
{
    private readonly ILogger _logger;

    public IndexModel(ILogger logger)
    {
        _logger = logger;
    }

    public void OnGet()
    {
        _logger.Log(LogLevel.Information, "Index page loaded");
    }
}
```

## Features

### Multiple Output Destinations (Flows)

```csharp
services.AddEonaCatLogging(builder =>
{
    builder
        .WriteToConsole()                          // Console output
        .WriteToFile("./logs")                     // File output
        .WriteToSlack("https://hooks.slack.com/...")  // Slack
        .WriteToDiscord("https://discordapp.com/...")  // Discord
        .WriteToElasticSearch("http://localhost:9200") // Elasticsearch
        .WriteToEmail("smtp.gmail.com", 587, ...)      // Email
        .WriteToMicrosoftTeams("https://...");         // Teams
});
```

### Enriching Log Events (Boosters)

```csharp
services.AddEonaCatLogging(builder =>
{
    builder
        .BoostWithMachineName()      // Adds machine name
        .BoostWithProcessId()        // Adds process ID
        .BoostWithThreadId()         // Adds thread ID
        .BoostWithCorrelationId()    // Adds correlation ID (for distributed tracing)
        .BoostWithMemory()           // Adds memory usage
        .BoostWithOS()               // Adds OS info
        .BoostWithUser()             // Adds username
        .BoostWithCustomText("Environment", "Production"); // Custom properties
});
```

### Log Level Filtering

```csharp
services.AddEonaCatLogging(builder =>
{
    builder
        .WithMinimumLevel(LogLevel.Warning)  // Only log warnings and above
        .WriteToConsole(minimumLevel: LogLevel.Information)  // More verbose for console
        .WriteToFile("./logs", minimumLevel: LogLevel.Error); // Only errors to file
});
```

## Best Practices

1. **Use LogBuilder for Configuration**: The fluent LogBuilder API is the most readable and maintainable approach.

2. **Register Early**: Register logging in `Program.cs` before other services that depend on logging.

3. **Use Appropriate Log Levels**: 
   - `Trace` - Very detailed diagnostic info
   - `Debug` - Debug-level diagnostic info
   - `Information` - General informational messages
   - `Warning` - Warning messages
   - `Error` - Error messages
   - `Critical` - Critical failures

4. **Inject Specific Types**: Prefer injecting `ILoggerFactory` to create category-specific loggers rather than injecting a single shared logger.

5. **Use Categories**: Create loggers with meaningful category names:
   ```csharp
   var logger = loggerFactory.CreateLogger("MyApp.Services.UserService");
   ```

6. **Enable Correlation IDs**: For distributed tracing scenarios:
   ```csharp
   builder.BoostWithCorrelationId()
   ```

## Configuration Examples

### Minimal Setup (Console Only)

```csharp
services.AddEonaCatLogging(b => b.WriteToConsole());
```

### Development Environment

```csharp
services.AddEonaCatLogging(builder =>
{
    builder
        .WithMinimumLevel(LogLevel.Debug)
        .WriteToConsole(useColors: true)
        .WriteToFile("./logs")
        .BoostWithMachineName()
        .BoostWithThreadId();
});
```

### Production Environment

```csharp
services.AddEonaCatLogging("ProductionApp", builder =>
{
    builder
        .WithMinimumLevel(LogLevel.Information)
        .WriteToFile("./logs", minimumLevel: LogLevel.Information)
        .WriteToElasticSearch("https://elastic.company.com")
        .WriteToSlack("https://hooks.slack.com/...")
        .BoostWithCorrelationId()
        .BoostWithMachineName()
        .BoostWithUser();
});
```
## Diagnostics

```csharp
public class DiagnosticsService
{
    private readonly ILoggerFactory _loggerFactory;

    public DiagnosticsService(ILoggerFactory loggerFactory)
    {
        _loggerFactory = loggerFactory;
    }

    public void PrintDiagnostics()
    {
        var diagnostics = _loggerFactory.GetDiagnostics();
        Console.WriteLine($"Total Logged: {diagnostics.TotalLoggedCount}");
        Console.WriteLine($"Total Dropped: {diagnostics.TotalDroppedCount}");
    }
}
```

## Disposing of the Logger

The logger is registered as a Singleton in the DI container, so it will be automatically disposed when the application shuts down. You can also manually access and dispose it:

```csharp
var loggerFactory = app.Services.GetRequiredService<ILoggerFactory>();
await loggerFactory.DisposeAsync();
```