Is anyone who is using go language in production faced problem with rolling of log files?
Is there any possibility to roll log file thread safely?
Are there any frameworks that make this?
Thank you.
Here's an internal solution. You can consider having a single goroutine responsible for logging and log rotation with a channel where it receives logs from other goroutines. Whenever it is time to rotate, the goroutine will rotate the log, while the incoming logs from other goroutines are enqueued on the channel. After log rotation is completed, it will then dequeue whatever is on the channel and continue logging.
Something along the lines of:
type Log struct {
log string
// other info you might want
}
// This will be your goroutine
func Logging(LogChannel chan Log) {
// assume logger creates/opens a file and prepares it for writing
logger := New(logger)
for {
// collect a log. Will also block until a log is available on the channel
log <- LogChannel
if timeToRotate() {
RotateLogFile(logger)
}
// write log
logger.Log(log)
}
}
EDIT: The previous code had the blocking call after checking if the log file rotated. It is better to put it before the timeToRotate
function call to prevent the possibility of writing the log after the log file has rotated
What I usually do is not use log files directly. Either you write to a network logger, like scribe or kafka, etc; Or preferably, you write to stdout/stderr, and have the service runner deal with writing to logfiles, forwarding to a network logger, or rotating the files.
I tend to take the 12 Factor approach of logging to std error and letting something else deal with the rotation.
Then, you have the system rotate the logs for you, as is mentioned here
i've had good success with https://github.com/natefinch/lumberjack it's lightweight and piggy backs on golang's standard and memory efficient "log" pkg.
very simple to use, just add one line of code to set it up, and use log as normal:
log.SetOutput(&lumberjack.Logger{
Filename: "/var/log/myapp/foo.log",
MaxSize: 500, // megabytes
MaxBackups: 3,
MaxAge: 28, //days
})