ServerLogger

Browsing, searching, and retaining server logs is often a tedious manual task that hinders effective debugging. ServerLogger is a drop-in Hex package for Elixir/Phoenix applications that captures your Erlang :logger events into PostgreSQL — with per-level retention policies, automatic monthly partitioning, and auto-pruning — so you never have to grep through log files or worry about unbounded disk growth again. It includes an optional Phoenix LiveDashboard page for real-time log viewing with level filtering, full-text search, sorting, pagination, and auto-refresh.

THIS IS BETA SOFTWARE! Be very cautious enabling this software in your application. While designed to be performant, we have not done comprehensive profiling so this code may have a noticable impact on your application's performance. Additionally, make sure you have plenty of disk space for the log records in your database.

Features

Zero-overhead buffering — Log events are stored as raw tuples in ETS by the caller process; formatting is deferred to flush time
Batch inserts — Configurable flush interval and max buffer size; bulk INSERT into PostgreSQL
Monthly partitions — Automatic partition creation and rotation; old partitions are dropped when all levels expire
Per-level retention — Configure lifetime per log level (debug: 1 day, error: 90 days, etc.), including "never save" and "keep forever"
Crash recovery — ETS table is owned by the supervisor; unflushed entries survive BufferServer restarts
LiveDashboard integration — Optional page with filtering, sorting, search, pagination, auto-refresh, and metrics

Example

Installation

Add server_logger to your dependencies:

def deps do
  [
    {:server_logger, "~> 0.2.0"}
  ]
end

Setup

1. Configure

Configure the server logger using config/config.exs.

# minimal required setup
config :server_logger, repo: MyApp.Repo

Here's a more comprehensive setup. See Configuration Reference for details. Individual settings can be updated per environment.

config :server_logger,
  repo: MyApp.Repo,
  enabled: true,
  buffer_flush_interval_ms: 1_000,
  buffer_max_size: 2_000,
  lifetime_days: [
    debug: nil,        # nil = never save
    info: 7,
    warning: 30,
    error: 90,
    critical: 0        # 0 = keep forever
  ],
  memory_limits: [
    max_message_size_mb: 8
  ]

2. Generate and run the migration

mix server_logger.gen.migration
mix ecto.migrate

3. Add to your supervision tree

# lib/my_app/application.ex
def start(_type, _args) do
  children = [
    MyApp.Repo,
    # ... other children ...
    ServerLogger.Supervisor
  ]

  Supervisor.start_link(children, strategy: :one_for_one)
end

4. (Optional) Add LiveDashboard page

# lib/my_app_web/router.ex
live_dashboard "/dashboard",
  additional_pages: [
    server_logs: ServerLogger.Dashboard.Page
  ]

Configuration Reference

Key	Type	Default	Description
`repo`	module	required	Your Ecto Repo module
`enabled`	boolean	`true`	Enable/disable the entire system. Useful for disabling in test or prod based on your system's needs
`buffer_flush_interval_ms`	integer	`1_000`	Periodic interval in ms to flush logs in ETS buffer to the database
`buffer_max_size`	integer	`2_000`	Max ETS entries before a flush automatically triggers
`logging_enabled`	`:stdio` \| `:stderr` \| `nil`	`nil`	ServerLogger's own diagnostic logging
`prune_interval_ms`	integer	`21_600_000` (6h)	How often the pruner checks for expired logs.
`lifetime_days`	keyword	see below	Per-level retention policy
`memory_limits`	keyword	`[max_message_size_mb: 8]`	Large log messages will be truncated down to this size

Lifetime days

nil — Never save logs of this level
0 — Save and keep forever (never prune)
Positive number — Prune after N days (e.g., 0.5 for 12 hours)

Defaults:

lifetime_days: [
  debug: 1,
  info: 7,
  warning: 30,
  error: 90,
  critical: 0
]

Architecture

┌─────────────┐     ┌───────────┐     ┌──────────────┐
│ Your App    │────▸│  :logger  │────▸│   Handler    │
│ Logger.info │     │  (Erlang) │     │ (caller pid) │
└─────────────┘     └───────────┘     └──────────────┘
                                              │
                                       ┌──────▼─────┐
                                       │ ETS Buffer │
                                       │  (public)  │
                                       └──────┬─────┘
                                              │
                                    ┌─────────▼────────┐
                                    │   BufferServer   │
                                    │ (periodic flush) │
                                    └─────────┬────────┘
                                              │
  ┌───────────────────┐            ┌──────────▼──────────┐            ┌────────────────┐
  │   PrunerServer    │───prune───▸│       PostgreSQL    │◂───query───│ Dashboard.Page │
  │ (cleanup records) │            │ (partitioned table) │            │   (LiveView)   │
  └───────────────────┘            └─────────────────────┘            └────────────────┘

AI Disclosure

This library was vibe engineered with the assistance of Anthropic's Claude 4.6 Opus. Just the code, not the architecture diagram... See docs/implementation-plans for implementation reference prompts.

License

MIT