I'm testing the concurrency of incrementing itemID
in the handler func below, and sometimes the increment skips a value (example: 4, 6, 7, ... skipped id 5).
func proxyHandler() http.Handler {
var itemID int32
return http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
proxy := httputil.NewSingleHostReverseProxy(url)
proxy.ModifyResponse = func(res *http.Response) error {
item := Item{
ID: int(atomic.AddInt32(&itemID, 1)),
}
items.Add(item)
return nil
}
proxy.ServeHTTP(rw, req)
})
}
I solved it by using Mutex:
func proxyHandler() http.Handler {
itemID := 0
mux := sync.Mutex{}
return http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
proxy := httputil.NewSingleHostReverseProxy(url)
proxy.ModifyResponse = func(res *http.Response) error {
mux.Lock()
itemID++
item := Item{
ID: itemID,
}
items.Add(item)
mux.Unlock()
return nil
}
proxy.ServeHTTP(rw, req)
})
}
I'd like to understand why atomic add didn't work as I was expecting, that is, generating a sequential value without gaps.
atomic.AddInt32()
is perfectly fine for concurrent use by multiple goroutines. That is why it is in the atomic
package. Your issue is with Items.Add()
which you have pointed out in the comments as not having any lock protection.
This is a rough definition for a safe Items.Add()
type Items struct {
items []Item
lock sync.Mutex
}
func (i *Items) Add(item Item) {
i.lock.Lock()
defer i.lock.Unlock()
i.items = append(i.items, item)
}
With the above definition of Items
, you can now use your initial code with atomic.AddInt32()
. However, I would like to point out that you must not read Items
while other threads are appending to it. Even reads must be synchronized.