I have a map and want to manually further sharding, the simplified code is
const (
dictShardNum = 16
dictShardSize = 1 << 28
)
type shard struct {
mu sync.Mutex
m map[int64]uint32
}
type dict struct {
shards []shard
}
func newDict() *dict {
shards := make([]shard, 0, dictShardNum)
for i := 0; i < dictShardNum; i++ {
shards = append(shards, shard{ m: make(map[int64]uint32) })
}
return &dict{ shards }
}
func (d *dict) insert(n int64) uint32 {
shardNum := int(n % dictShardNum)
shard := d.shards[shardNum]
shard.mu.Lock()
defer shard.mu.Unlock()
tempID, ok := shard.m[n]
if !ok {
tempID = uint32(len(shard.m) + shardNum*dictShardSize)
shard.m[n] = tempID // fatal error: concurrent map writes
}
return tempID
}
When running I got fatal error: concurrent map writes at that line, but I did lock the mutex, not sure what's wrong with my code
import "sync"A Mutex is a mutual exclusion lock. The zero value for a Mutex is an unlocked mutex.
A Mutex must not be copied after first use.
Your code doesn't compile!
Playground: https://play.golang.org/p/6AwS0vOZfeP
25:18: undefined: n
30:24: undefined: n
33:11: undefined: n
If I change v int64 to n int64:
A Mutex must not be copied after first use.
$ go vet mutex.go
./mutex.go:26:11: assignment copies lock value to shard: command-line-arguments.shard contains sync.Mutex
$
Playground: https://play.golang.org/p/jExE-m11ny5
package main
import (
"sync"
)
const (
dictShardNum = 16
dictShardSize = 1 << 28
)
type shard struct {
mu sync.Mutex
m map[int64]uint32
}
type dict struct {
shards []shard
}
/// a newDict function
func (d *dict) insert(n int64) uint32 {
shardNum := int(n % dictShardNum)
shard := d.shards[shardNum]
shard.mu.Lock()
defer shard.mu.Unlock()
tempID, ok := shard.m[n]
if !ok {
tempID = uint32(len(shard.m) + shardNum*dictShardSize)
shard.m[n] = tempID // fatal error: concurrent map writes
}
return tempID
}
func main() {}
Vet examines Go source code and reports suspicious constructs
Flag: -copylocks
Locks that are erroneously passed by value.
I think the answer is related to copy mutex values. The dict should be
type dict struct {
shards []*shard
}
All shards are accessed via pointer then it won't have any issue.