对一个通道使用多个接收器

I am trying write a REST service in golang using go-json-rest

The purpose of the service is just to convert the received data to CSV and log it. Since the load may be heavy, I would like to do the logging using goroutines. Currently I have created four LogWorkers(goroutine) Each goroutine will log the CSV into separate files.

When I execute the code, the log is always triggered from the last goroutine. I see a single file created in my log folder which is from fourth routine.

Here is my server code

package main

import (
    "github.com/ant0ine/go-json-rest/rest"
    "log"
    "net/http"
    "strconv"
    "time"
)

const workerCount = 4
var evChannel = make(chan Event)
var workers = make([]*LogWorker, workerCount)
const maxLogFileSize = 100 // In MB
const maxLogFileBackups = 30
const maxLogFileAge = 5
const logFileName = "/home/sam/tmp/go_logs/event_"

func main() {
    // Initialize workers
    // Four workers is being created
    for i := 0; i < workerCount; i++ {
        var fileName = logFileName + strconv.Itoa(i)
        workers[i] = NewLogWorker(fileName, maxLogFileSize, maxLogFileBackups, maxLogFileAge)
        go workers[i].Work(evChannel)
    }    

    // Initialize REST API
    api := rest.NewApi()
    //api.Use(rest.DefaultDevStack...)
    api.Use(rest.DefaultCommonStack...)
    router, err := rest.MakeRouter(
        rest.Post("/events", StoreEvents),
    )
    if err != nil {
        log.Fatal(err)
    }
    api.SetApp(router)
    log.Fatal(http.ListenAndServe(":4545", api.MakeHandler()))
}

func StoreEvents(w rest.ResponseWriter, r *rest.Request) {
    event := Event{}
    err := r.DecodeJsonPayload(&event)
    if err != nil {
        rest.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    // TODO : Add validation if needed
    // Add code to parse the request and add further information to event 
    // log.Println()
    select {
        case evChannel <- event:
        case <- time.After(5 * time.Second):
      // throw away the message, so sad
    }    
    // evChannel <- event
    //log.Println(Csv(event))
    w.WriteHeader(http.StatusOK)
}

here is my worker code

package main

import (
    "gopkg.in/natefinch/lumberjack.v2"
    "log"
    "fmt"
)

type LogWorker struct {
    FileName string
    MaxSize int // In megabytes
    MaxBackups int // No of backups per worker
    MaxAge int // maximum number of days to retain old log files
}

func NewLogWorker(fileName string, maxSize int, maxBackups int, maxAge int) (lw *LogWorker) {
    return &LogWorker {fileName, maxSize, maxBackups, maxAge}
}

func (lw *LogWorker) Work(evChannel chan Event) {
    fmt.Println(lw.FileName)
    log.SetOutput(&lumberjack.Logger {
        Filename:   lw.FileName,
        MaxSize:    lw.MaxSize,
        MaxBackups: lw.MaxBackups,
        MaxAge:     lw.MaxAge,
    })
    log.SetFlags(0)

    for {
        event := <- evChannel
        log.Println(Csv(event))
    }
}

Please note that event is a struct which contains some string fields. Already there is a similar question in SO. When I tried to execute the goroutine in playground, it still prints the value from last go routine. The answer provided has some wait.Done. As my worker needs to run continuously I don't I think I can use it.

Please help me to find why my all goroutines (LogWorkers) are not used?

You are setting the log package's default global logger's output in each goroutine.

You probably want to do something more like:

func (lw *LogWorker) Work(evChannel chan Event) {
    fmt.Println(lw.FileName)
    lg := log.New(&lumberjack.Logger {
        Filename:   lw.FileName,
        MaxSize:    lw.MaxSize,
        MaxBackups: lw.MaxBackups,
        MaxAge:     lw.MaxAge,
    }, "", 0)

    for {
        event := <- evChannel
        lg.Println(Csv(event))
    }
}

this will give you a logger per goroutine.

In your version, you likely just had the last one to execute (probably last goroutine spawned, but not guaranteed)

And to improve a bit more, you probably also want your for loop written as:

for event := range evChannel {
    lg.Println(Csv(event))
}

This way, it will terminate the goroutine when the channel is closed rather than spin on empty values coming out of a closed channel. See here for reference