Coming from a python background, and just starting with Go, I found myself looking for the equivalent of the map() and reduce() functions in Go. I didn't find them, so fell back on for loops. For example, this is what I used instead of a map(), where mapFunction is defined elsewhere:
data := make([]byte, 1024)
count, err := input.Read(data) // error handling removed from this snippet
for i:=0; i<count; i++ {
data[i] = mapFunction(data[i])
}
and this is what I used instead of a reduce(), where there are 2 state variables that I'm using to keep track of quoting of fields in a CSV as the code moves through each item in the slice:
data := make([]byte, 1024)
count, err := input.Read(data) // error handling removed from this snippet
for i:=0; i<count; i++ {
data[i], stateVariable1, stateVariable2 =
reduceFunction(data[i], stateVariable1, stateVariable2)
}
Here are my questions:
Thanks!
ps - the full code is here: https://github.com/dbro/csvquote/blob/go/csvquote.go
In short:
A bit longer.
Volker has given a good answer, but it doesn't play to one of Go's main strengths, which is its concurrency. A map/reduce type of operation may be parallelized (premature optimization aside) through the use of a 'server farm' strategy. This involves dividing the work to be done into work packets that are sent to separate workers (i.e. goroutines). Map/Reduce is a generic way of doing this and requires higher order functions and immutable data structures.
Go is flexible enough to allow a bespoke parallel decomposition even though it isn't a functional language. Although there's no immutability, it allows aliasing to be avoided through the use of copy semantics, thereby eliminating race conditions when values are exchanged between goroutines, which is effectively as good. Put simply: use structs directly instead of pointers to structs when sharing. (And to help, there's a new race detector in Go1.1).
The server farm pattern is a good way of achieving high parallelization efficiencies because it is self-balancing. This contrasts with geometric decompositions (i.e. sharing a grid of data by clumping zones and allocating them to processors) and with algorithmic decompositions (i.e. allocating different stages in a pipeline to different processors), both of which suffer from potentially unbalanced load. Go is capable of expressing all three kinds.