log
github.com/regorov/logwriter
go get github.com/regorov/logwriter
log

github.com/regorov/logwriter

Golang log package extension. Automates logging routine: split files, archive, etc.

go get github.com/regorov/logwriter
Readme

logwriter

Golang package logwriter automates routine related to logging into files.

GoDoc Build Status Coverage Status

Initial version finished. Stabilization, testing and benchmarkign are going.

Concepts

Hot and Cold Log Files

There is a single hot log file. Usually file name is similar to daemon/service name and located in /var/log/servicename/. There are cold log files. In accordance to rules specified by logwriter.Config, logwriter freezes content of hot file by moving content to new cold file.

Using sync.Mutex

If you don't need buffering (logwriter.Config.BufferSize==0) you can believe that file write executes synchronously.

Error Handling

There is set of background routines what can face critital i/o errors. Specify callback function f(error) in NewLogWriter() what will be called every time then background routines catch an error. Possible cases:

  • No free space at config.ColdPath
  • No free space at config.HotPath
  • Permission denied
  • ...

You error handler function calls in sync mode and blocks log writing. Please, do not do long operations there.

Stop! It's not a *unix way

Oh nooo. Not everyone develops Facebook (c) or smth similar daily :)

Features

  • Folders for hot and cold log files configurable
  • Using fixed name of file with latest log items
  • Support module running mode
    • Production - writes into the file only
    • Debug - writes into file and os.Stdout
  • Support hot file freezing rules:
    • By max file size
    • Every time.Duration
    • Every midnight
    • Manually
    • Freeze when your application starts
  • File write buffering
    • Configurable buffer size
    • Flush buffer every time.Duration
    • Flush buffer manually
  • Update configuration on the fly
  • Cold log files compression
  • Log items re-ordering before persisting
  • Log items re-ordering on freezing stage
  • Cold files cleaning
  • Cold log files round robin
  • Tracing option. Saving some of log items in separate .trc files
  • Ability to freeze hot file several times per second

Tasks

  • Add benchmarks
  • Add tests
  • Add examples

Examples

Using standard log package

package main

import (
    "github.com/regorov/logwriter"
    "log"
    "time"
)

func main() {
    cfg := &logwriter.Config{
        BufferSize:       0,                  // no buffering
        FreezeInterval:   1 * time.Hour,      // freeze log file every hour
        HotMaxSize:       100 * logwriter.MB, // 100 MB max file size
        CompressColdFile: true,               // compress cold file
        HotPath:          "/var/log/mywebserver",
        ColdPath:         "/var/log/mywebserver/arch",
        Mode:             logwriter.ProductionMode, // write to file only
    }

    lw, err := logwriter.NewLogWriter("mywebserver",
        cfg,
        true, // freeze hot file if exists
        nil)

    if err != nil {
        panic(err)
    }

    logger := log.New(lw, "mywebserver", log.Ldate|log.Ltime)
    logger.Println("Module started")

    if err := lw.Close(); err != nil {
        // Error handling
    }

    return
}

Using github.com/Sirupsen/logrus

package main

import (
  "time"
  "github.com/Sirupsen/logrus"
  "github.com/regorov/logwriter"
)

func errHandler(err error) {

    // send SMS or Smth
    return
}

func main() {

    lw, err := logwriter.NewLogWriter("mywebserver",
                                      &logwriter.Config{
                                          BufferSize: 1024 * 1024, // 1 MB
                                          BufferFlushInterval : 3 * time.Second, // flush buffer every 3 sec
                                          FreezeInterval : 1 * time.Hour, // create new log every hour
                                          HotMaxSize : 100 * 1024 * 1024, // or when hot file size over 100 MB
                                          HotPath: "/var/log/myweb",
                                          ColdPath: "/var/log/myweb/arch",
                                          Mode: logwriter.ProductionMode,
                                        },
                                      false, // do not freeze hot file if exists
                                      errHandler))
    if err != nil {
        // Error handling
    }

    var log = logrus.New()
    log.Out = lw

    log.WithFields(logrus.Fields{"animal": "walrus",
                                 "size":   10,
    }).Info("A group of walrus emerges from the ocean")

    if err := lw.Close(); err != nil {
        // Error handling
    }

    return
}

License

MIT

GitHub Stars

11

LAST COMMIT

4yrs ago

MAINTAINERS

0

CONTRIBUTORS

2

OPEN ISSUES

0

OPEN PRs

0
VersionTagPublished
v0.0.0-20171127083154-22424588c902
3yrs ago
No alternatives found
No tutorials found
Add a tutorial