We are planning to implement a massively scaling backend system which essentially has to count hundreds of thousands of end user votes within a pretty short amount of time (approx. 5 minutes).
The implementation will likely be done on App Engine, using the Go runtime and Dedicated Memcache service. Perhaps, Datastore will be used for persisting counter values after the voting period.
Our current architectural ideas and questions:
We plan to use instance memory for the immediate per-request-counts. Are we correct in assuming that just using a Go global variable actually translates into "using instance memory"?
We plan to store each instances total counter value (the value of the global variable) into Dedicated Memcache in a to-be-defined interval, e.g. every 10 seconds or on 250 increments. We possibly will shard these memcached counters to avoid peak load on single Key / Items.
Using an App Engine cron job, we might persist the Memcache counters for long term use.
What do you think? Does this make sense? Is this a good approach?
For something like this, you probably want to use the task queue mechanism (in particular, the pull queue mechanism). This would allow you to process (tally + commit) several vote tasks in a single operation. This is likely to be more reliable than using instance memory, since the task queue will outlive task restarts.