My application consumes data from Kinesis, process it and forwards it to another microservice. In order to reduce the number of requests to the last, I'm trying to implement a queue of objects with a policy of forwarding its items once a defined queue length is reached or once a timeout is reached (to avoid stuck items for a long time).
So far I've found a good example to implement the timeout here, but I'm struggling with the queue of objects. I've read a lot about buffered channels, however I'm not sure if they are the way to go.
According to this reference, a buffered channel can be used to limit the amount of work that is queued up, preventing your services from falling behind and becoming overwhelmed. However, it seems that what I need is not limit the amount of work, but simply accumulate work (my Kinesis stream data) to be dealt in batches.
So for my queue of objects, would it be enough a slice and a go routine checking its length, so the data would be forwarded once it reached a certain size? Or there are good reasons to use a buffered channel?