I have an API that I am POSTing a small JSON object too.
This runs as an infinite loop, looping over 7 colours (the rainbow) and send these inside the aforementioned JSON object.
The API I am connecting to has a rate limit on it of 40 requests per minute.
I don't want to hit the rate limit and so I have devised a method to avoid this;
It looks a little something like this:
var rateLimit int
func main() {
request := gorequest.New().SetDebug(false)
// Set the initial request bucket to 40
rateLimit = 40
go topUpLimiter()
for {
makeTheLightsBlinkTheRainbow(request)
}
}
func topUpLimiter() {
for range time.Tick(60 * time.Second) {
rateLimit += 40
}
}
func makeTheLightsBlinkTheRainbow(request *gorequest.SuperAgent) {
colours := [7]string{"red", "orange", "yellow", "green", "blue", "purple", "pink"}
for _, colour := range colours {
if rateLimit > 0 {
response, _, _ := request.Post("http://example.com/blink").
Send(fmt.Sprintf(`{"color":"%v"}`, colour)).
End()
rateLimit--
} else {
time.Sleep(1 * time.Second)
}
}
}
This works and I don't ever hit the rate limit, but once I run out of requests, the loop continues to run and will only starts making requests again when the rateLimit variable is topped up.
I am making an IoT light blink the colours of the rainbow and the result is the colours get out of order once the rateLimit variable runs out and is then later topped up due to the fact that the for loop just keeps running.
I'd like to pause/block the for loop while I wait for the rateLimit variable to replenish so that the colours don't get out of order.
How would I implement something like that? From my searches it seems like it may be possible with channels, but I'm not too sure how to do it like that.
This is an example of what ends up happening: https://play.golang.org/p/r6OG4kK9vCP Once it has finished running, you will notice the colours printed out get out of order about halfway through.
My suggestion: Take an entirely different approach to rate limiting.
I would put it in the HTTP transport, since that's logically where such a limit ought to exist, and then you needn't mangle your application code at all. Something like:
import (
"net/http"
"golang.org/x/time/rate"
)
type rateLimitTransport struct {
limiter *rate.Limiter
xport http.RoundTripper
}
var _ http.RoundTripper = &rateLimitTransport{}
func newRateLimitTransport(r float64, xport http.RoundTripper) http.RoundTripper {
return &rateLimitTransport{
limiter: rate.NewLimiter(rate.Limit(r), 1),
xport: xport,
}
}
func (t *rateLimitTransport) RoundTrip(r *http.Request) (*http.Response, error) {
t.limiter.Wait(r.Context())
return t.xport.RoundTrip(r)
}
This uses the golang.org/x/time/rate package to implement the rate limiting. newRateLimitTransport() creates a new rate-limiting transport, where r is the maximum number of requests to allow per second.
To take advantage of this, use an instance of the rate-limited transport in the HTTP client for your backend API:
// set this in `init()` for example
myClient := http.&Client{
// Use a rate-limiting transport which falls back to the default
Transport: newRateLimitTransport(60, http.DefaultTransport)
}
// Then later use `myClient` instead of the default, when making API
// requests:
req, err := http.NewRequest(http.MethodPost, url, body)
if err != nil {
return err
}
myClient.Do(req)
Try changing to
func makeTheLightsBlinkTheRainbow(request *gorequest.SuperAgent) {
colours := [7]string{"red", "orange", "yellow", "green", "blue", "purple", "pink"}
i := 0
for {
if rateLimit > 0 {
response, _, _ := request.Post("http://example.com/blink").
Send(fmt.Sprintf(`{"color":"%v"}`, colours[i])).
End()
i = (i+1) % 7
rateLimit--
} else {
time.Sleep(1 * time.Second)
}
}
}