I was playing with different ways to initialise/declare arrays in golang. I got different behaviours/results.
go version go1.3 darwin/amd64
func main() {
a := [100000000]int64{}
var i int64
for i = 0; i < 100000000; i++ {
a[i] = i
}
}
Produces a 763MB binary. It crashes after a few seconds when I run it with this message.
runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow
func main() {
var a [100000000]int64
var i int64
for i = 0; i < 100000000; i++ {
a[i] = i
}
}
Produces a 456KB binary. It runs in less than one second.
Can anybody help me understand why those differences (and other I may have missed) are there? Thanks!
edit:
I built the two different snippets and run the compiled versions, so the compilation time wasn't added. The first time I run version1 is extremely slow in comparison though. Here is the output.
go build version1.go
go build version2.go
These are the execution outputs
first run
time ./version1
runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow
runtime stack:
runtime.throw(0x2fb42a8e)
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/panic.c:520 +0x69
runtime.newstack()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/stack.c:770 +0x486
runtime.morestack()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/asm_amd64.s:228 +0x61
goroutine 16 [stack growth]:
main.main()
/Users/ec/repo/offers/lol/version1.go:3 fp=0x2b7b85f50 sp=0x2b7b85f48
runtime.main()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:247 +0x11a fp=0x2b7b85fa8 sp=0x2b7b85f50
runtime.goexit()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:1445 fp=0x2b7b85fb0 sp=0x2b7b85fa8
created by _rt0_go
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/asm_amd64.s:97 +0x120
goroutine 17 [runnable]:
runtime.MHeap_Scavenger()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/mheap.c:507
runtime.goexit()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:1445
./version1 0.00s user 0.10s system 1% cpu 7.799 total
second run
runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow
runtime stack:
runtime.throw(0x2fb42a8e)
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/panic.c:520 +0x69
runtime.newstack()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/stack.c:770 +0x486
runtime.morestack()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/asm_amd64.s:228 +0x61
goroutine 16 [stack growth]:
main.main()
/Users/ec/repo/offers/lol/version1.go:3 fp=0x2b7b85f50 sp=0x2b7b85f48
runtime.main()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:247 +0x11a fp=0x2b7b85fa8 sp=0x2b7b85f50
runtime.goexit()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:1445 fp=0x2b7b85fb0 sp=0x2b7b85fa8
created by _rt0_go
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/asm_amd64.s:97 +0x120
goroutine 17 [runnable]:
runtime.MHeap_Scavenger()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/mheap.c:507
runtime.goexit()
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:1445
./version1 0.00s user 0.10s system 98% cpu 0.102 total
first run
time ./version2
./version2 0.16s user 0.26s system 99% cpu 0.429 total
second run
time ./version2
./version2 0.17s user 0.25s system 97% cpu 0.421 total
In Version 1, you're declaring a literal array of [100000000]int64{} which the compiler immediately allocates.
Version 2, you're only declaring the type of a as [100000000]int64.
When you only have a variable declaration, the contents aren't known at that point during compilation. In version 2, the compiler knows that a is of type [100000000]int64, but the memory isn't allocated until runtime.
When you use a literal, that exact memory representation is written into the binary. It works the same as if you declared a string literal vs a variable of type string; the string literal will be written in place, while a variable declaration is only a placeholder.
Even though the current compiler (go 1.3) allows a to escape to the heap, the literal data is expected to live in the stack frame. You can see this in the assembly output (frame size is 800000016):
TEXT "".func1+0(SB),$800000016-0
If you actually need a larger literal than what can fit in the stack, you can place it in a global variable. The following executes just fine:
var a = [100000000]int64{1}
func func1() {
var i int64
for i = 0; i < 100000000; i++ {
a[i] = i
}
}
I did have to initialize at least one value in a here, because it seems that the compiler can elide this literal, if it's equal to the zero value.
It's not actually an answer but maybe someone can find this useful.
In my case this happens when I've try to json.Marshal following structure:
type Element struct {
Parent *Element
Child []*Element
}
Where Parent points to element which has current element in Child.
So I just mark "Parent" as json:"-" to be ignored in marshalling.