I have a case where I am reading data from 2 different locations (ES and REDIS), I need to read a single value from the fastest source, thus I am firing 2 goroutines, one to get the data from ES, the other to get it from REDIS.
once data has been fetched from one of the goroutines, the other goroutine must be canceled completely not to waste CPU.
simplified:
func A(){
go funcB(){
}()
go funcC(){
}()
data := <-channel //
}
now once the data is received, funcA
or funcB
must be canceled, no matter what they were doing (I don't care for their output anymore, they are just wasting CPU)
what would be the most efficient way to do it? can it be done using channels only?
The context package provides cancelation, timeout and deadline contexts for this purpose. Here you can see a cancelation example, and we wait for the slower goroutine to print the cancelled message:
ctx, cancel := context.WithCancel(context.Background())
// buffer the channel for extra results returned before cancelation
data := make(chan string, 2)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
select {
case <-time.After(100 * time.Millisecond):
data <- "A complete"
case <-ctx.Done():
fmt.Println("A cancelled")
}
}()
wg.Add(1)
go func() {
defer wg.Done()
select {
case <-time.After(200 * time.Millisecond):
data <- "B complete"
case <-ctx.Done():
fmt.Println("B cancelled")
}
}()
resp := <-data
cancel()
fmt.Println(resp)
wg.Wait()
You have some Options depending to your real use case:
1- Using two goroutines:
This needs sync/Lock
:
Try this simulated sample (The Go Playground):
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
func main() {
rand.Seed(time.Now().Unix())
time.AfterFunc(time.Duration(rand.Intn(1000))*time.Millisecond, func() { ES <- 101 })
time.AfterFunc(time.Duration(rand.Intn(1000))*time.Millisecond, func() { REDIS <- 102 })
go B()
go C()
data := <-channel
fmt.Println(data)
}
func B() {
check := true
data := 0
for {
select {
case <-quit:
return
case data = <-ES: // receive data
}
if check {
mx.Lock()
//defer mx.Unlock()
if mx.done {
mx.Unlock()
return
}
check = false
close(quit)
mx.done = true
mx.Unlock()
}
fmt.Println("ES ready")
channel <- data
}
}
func C() {
check := true
data := 0
for {
select {
case <-quit:
return
case data = <-REDIS: // receive data
}
if check {
mx.Lock()
//defer mx.Unlock()
if mx.done {
mx.Unlock()
return
}
check = false
close(quit)
mx.done = true
mx.Unlock()
}
fmt.Println("REDIS ready")
channel <- data
}
}
var (
channel = make(chan int)
ES = make(chan int)
REDIS = make(chan int)
quit = make(chan struct{})
mx lockdown
)
type lockdown struct {
sync.Mutex
done bool
}
2- In this sample you just start one goroutine B
or C
:
see this pseudo code:
func main() {
go A()
data := <-channel
fmt.Println(data)
}
func A() {
for{
if ES ready
go B(data)
return
if REDIS ready
go C(data)
return
}
}
You may start A
goroutine, in A
goroutine it detects which input is ready e.g. ES
or REDIS
, then starts B
or C
goroutine accordingly:
Try this simulated sample (The Go Playground):
AfterFunc is just for simulation, in real code you don't need it, it simulates random timing for one input.
package main
import (
"fmt"
"math/rand"
"time"
)
func main() {
rand.Seed(time.Now().Unix())
time.AfterFunc(time.Duration(rand.Intn(1000))*time.Millisecond, func() { ES <- 101 })
time.AfterFunc(time.Duration(rand.Intn(1000))*time.Millisecond, func() { REDIS <- 102 })
go A()
data := <-channel
fmt.Println(data)
}
func A() {
select {
case data := <-ES:
go B(data)
return
case data := <-REDIS:
go C(data)
return
}
}
func B(data int) {
for {
fmt.Println("ES ready")
channel <- data
data = <-ES
}
}
func C(data int) {
for {
fmt.Println("REDIS ready")
channel <- data
data = <-REDIS
}
}
var (
channel = make(chan int)
ES = make(chan int)
REDIS = make(chan int)
)
output from run 1:
REDIS ready
102
output from run 2:
ES ready
101