sls

sls stands for Simple Logging Service. It's a server with an API that aggregates and broadcasts logs to other services.

Install

go get -u egt.run/sls

Usage

API

POST /log
Header:
    Body-Content: application/json

Body:
    [
        "any content at all",
        "separated into elements",
        "in a json array",
    ]

sls takes logs from the request body and broadcasts them to various targets. Targets are anything that implement the LogTarget interface:

type LogTarget interface {
    Name() string
    io.WriteCloser
}

Two LogTarget implementations are provided out-of-the-box:

• disk appends logs to a file on disk, automatically rotating the file every 24 hours and retaining files for a limited time. It's important to have enough HDD space for those log files. Writes to disk are threadsafe. Log files will be written to a given directory with the format YYYYMMDD.log.
• stackdriver sends logs asynchronously on a best-effort basis to Google's Stackdriver logging service.

Ordering: it's assumed that logs batched together in a single API call are from the same request or are related to the same action. Multiple requests (e.g. logs from several different HTTP requests) can be included in a single API call, but to ensure proper ordering, do not split up a request across API calls. A simple buffer that flushes to the API is not sufficient if ordering needs to be preserved.

Timestamps: sls does not add timestamps to logs. If needed, servers doing the logging should include them.

Auth: sls assumes that connections are initiated by trusted parties over a LAN.

!!! DO NOT EXPOSE SLS ON A PUBLIC PORT !!!

Configuration

Create config.json (or use the -c flag to specify a custom file), and fill it out with the following:

{
    "port": 9000,
    "targets": {
        "disk": {
            "dir": "/var/log",
            "retain_days": 30
        },
        "stackdriver": {
            "project_id": "your-project",
            "credential_file": "/path/to/service-file/google.json"
        }
    }
}

If you want to use only a particular target, omit the others from the config file.