python-logging-best-practices

Python Logging Best Practices

Overview

Unified reference for Python logging patterns optimized for machine readability (Claude Code analysis) and operational reliability.

MANDATORY Best Practices

1. Log Rotation (ALWAYS CONFIGURE)

Prevent unbounded log growth - configure rotation for ALL log files:

# Loguru pattern (recommended for modern scripts)
from loguru import logger

logger.add(
    log_path,
    rotation="10 MB",      # Rotate at 10MB
    retention="7 days",    # Keep 7 days
    compression="gz"       # Compress old logs
)

# RotatingFileHandler pattern (stdlib-only)
from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    log_path,
    maxBytes=100 * 1024 * 1024,  # 100MB
    backupCount=5                 # Keep 5 backups (~500MB max)
)

2. JSONL Format (Machine-Readable)

Use JSONL (.jsonl) for logs that Claude Code or other tools will analyze:

# One JSON object per line - jq-parseable
{"timestamp": "2026-01-14T12:45:23.456Z", "level": "info", "message": "..."}
{"timestamp": "2026-01-14T12:45:24.789Z", "level": "error", "message": "..."}

File extension: Always use .jsonl (not .json or .log)

Validation: cat file.jsonl | jq -c .

Terminology: JSONL is canonical. Equivalent terms: NDJSON, JSON Lines.

When to Use Which Approach

Complete Loguru + platformdirs Pattern

Cross-platform log directory handling with structured JSONL output:

#!/usr/bin/env python3
# /// script
# requires-python = ">=3.11"
# dependencies = ["loguru", "platformdirs"]
# ///

import json
import sys
from pathlib import Path
from uuid import uuid4

import platformdirs
from loguru import logger


def json_formatter(record) -> str:
    """JSONL formatter for Claude Code analysis."""
    log_entry = {
        "timestamp": record["time"].strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z",
        "level": record["level"].name.lower(),
        "component": record["function"],
        "operation": record["extra"].get("operation", "unknown"),
        "operation_status": record["extra"].get("status", None),
        "trace_id": record["extra"].get("trace_id"),
        "message": record["message"],
        "context": {k: v for k, v in record["extra"].items()
                   if k not in ("operation", "status", "trace_id", "metrics")},
        "metrics": record["extra"].get("metrics", {}),
        "error": None
    }

    if record["exception"]:
        exc_type, exc_value, _ = record["exception"]
        log_entry["error"] = {
            "type": exc_type.__name__ if exc_type else "Unknown",
            "message": str(exc_value) if exc_value else "Unknown error",
        }

    return json.dumps(log_entry)


def setup_logger(app_name: str = "my-app"):
    """Configure Loguru for machine-readable JSONL output."""
    logger.remove()

    # Console output (JSONL to stderr)
    logger.add(sys.stderr, format=json_formatter, level="INFO")

    # Cross-platform log directory
    # macOS: ~/Library/Logs/{app_name}/
    # Linux: ~/.local/state/{app_name}/log/
    log_dir = Path(platformdirs.user_log_dir(
        appname=app_name,
        ensure_exists=True
    ))

    # File output with rotation
    logger.add(
        str(log_dir / f"{app_name}.jsonl"),
        format=json_formatter,
        rotation="10 MB",
        retention="7 days",
        compression="gz",
        level="DEBUG"
    )

    return logger


# Usage
setup_logger("my-app")
trace_id = str(uuid4())

logger.info(
    "Operation started",
    operation="my_operation",
    status="started",
    trace_id=trace_id
)

logger.info(
    "Operation complete",
    operation="my_operation",
    status="success",
    trace_id=trace_id,
    metrics={"duration_ms": 150, "items_processed": 42}
)

Semantic Fields Reference

Related Resources

• launchagent-log-rotation skill - RotatingFileHandler for daemons
• platformdirs reference - Cross-platform directories
• loguru patterns - Advanced loguru configuration
• migration guide - From print() to structured logging

Anti-Patterns to Avoid

1. Unbounded logs - Always configure rotation
2. print() for logging - Use structured logger
3. Bare except - Catch specific exceptions, log them
4. Silent failures - Log errors before suppressing
5. Hardcoded paths - Use platformdirs for cross-platform