Go's append() function only allocates new slice data, when the capacity of the given slice is not sufficient (see also: https://stackoverflow.com/a/28143457/802833). This can lead to unexpected behavior (at least for me as a golang newbie):
package main
import (
"fmt"
)
func main() {
a1 := make([][]int, 3)
a2 := make([][]int, 3)
b := [][]int{{1, 1, 1}, {2, 2, 2}, {3, 3, 3}}
common1 := make([]int, 0)
common2 := make([]int, 0, 12) // provide sufficient capacity
common1 = append(common1, []int{10, 20}...)
common2 = append(common2, []int{10, 20}...)
idx := 0
for _, k := range b {
a1[idx] = append(common1, k...) // new slice is allocated
a2[idx] = append(common2, k...) // no allocation
idx++
}
fmt.Println(a1)
fmt.Println(a2) // surprise!!!
}
output:
[[10 20 1 1 1] [10 20 2 2 2] [10 20 3 3 3]]
[[10 20 3 3 3] [10 20 3 3 3] [10 20 3 3 3]]
https://play.golang.org/p/8PEqFxAsMt
So, what ist the (idomatic) way in Go to force allocation of new slice data or more precisely to make sure that the slice argument to append() remains unchanged?
Thanx for your feedback.
So the solution to gain control of the memory allocation is to do it explicitely (which remembers me that Go is a more a system language than other (scripting) langs):
package main
import (
"fmt"
)
func main() {
a1 := make([][]int, 3)
a2 := make([][]int, 3)
b := [][]int{{1, 1, 1}, {2, 2, 2}, {3, 3, 3}}
common1 := make([]int, 0)
common2 := make([]int, 0, 12) // provide sufficient capacity
common1 = append(common1, []int{10, 20}...)
common2 = append(common2, []int{10, 20}...)
idx := 0
for _, k := range b {
a1[idx] = append(common1, k...) // new slice is allocated
a2[idx] = make([]int, len(common2), len(common2)+len(k))
copy(a2[idx], common2) // copy & append could probably be
a2[idx] = append(a2[idx], k...) // combined into a single copy step
idx++
}
fmt.Println(a1)
fmt.Println(a2)
}
output:
[[10 20 1 1 1] [10 20 2 2 2] [10 20 3 3 3]]
[[10 20 1 1 1] [10 20 2 2 2] [10 20 3 3 3]]
You might maintain a wrong idea of how slices work in Go.
When you append elements to a slice, the call to append() returns a new slice. If reallocation did not happen, both slice values — the one you called append() on and the one it returned back — share the same backing array but they will have different lengths; observe:
package main
import "fmt"
func main() {
a := make([]int, 0, 10)
b := append(a, 1, 2, 3)
c := append(a, 4, 3, 2)
fmt.Printf("a=%#v
b=%#v
c=%#v
", a, b, c)
}
outputs:
a=[]int{}
b=[]int{4, 3, 2}
c=[]int{4, 3, 2}
So, len(a) == 0, len(b) == 3, len(c) == 3, and the second call to append() owerwrote what the first one did because all the slices share the same underlying array.
As to reallocation of the backing array, the spec is clear:
If the capacity of s is not large enough to fit the additional values, append allocates a new, sufficiently large underlying array that fits both the existing slice elements and the additional values. Otherwise, append re-uses the underlying array.
From this, it follows that:
append() never copies the underlying storage if the capacity of the slice being appeneded to is sufficient.That is, given a slice s to which you want to append N elements, the reallocation won't be done iff cap(s) - len(s) ≥ N.
Hence I suspect your problem is not about unexpected reallocation results but rather about the concept of slices as implemented in Go. The code idea to absorb is that append() returns the resulting slice value, which you're supposed to be using after the call unless you fully understand the repercussions.
I recommend starting with this to fully understand them.