Is it possible to implement something akin to yield from python in go? That is, I want a producer/consumer paradigm where the consumer can ignore values produced by the producer at any time without manifesting a memory leak.
In python I could have a producer that produces infinite values and the client can consume as many as they wish.
def producer():
i = 0
while True:
yield i
i += 1
def consumer():
for i in producer():
if i == 2:
return
The common advice for this is for the producer to return a channel that the client uses in a range expression, and the producer can put its values into the channel. This is fine as long as the client reads all the values in the channel, but if the client breaks control from the range loop early the producer channel will be left hanging forever using up memory.
package main
func yield() chan int {
out := make(chan int)
go func(){
out <- 1
out <- 2
close(out)
}()
return out
}
func test(){
for _ = range yield() {
// return
}
}
func main(){
for {
test()
}
}
If the return in test is uncommented this code will use memory without bound, because the channel returned by yield has a value that is waiting to be read and apparently neither the channel nor the goroutine can be garbage collected safely.
Some possible solutions (neither of which I like) are
Change out in yield to be a buffered channel with exactly the amount of space as values that will be produced. In that case yield could be
func yield() chan int { out := make(chan int, 2) out <- 1 out <- 2 close(out) return out }
But this is suboptimal because I don't know in general how many items yield will produce. This style is similar to just returning an array.
yield method use a timeout for putting values into the channel, and if the timeout is reached then close the channel early. This makes assumptions about the execution time of the client, which seems like a bad idea.