I am trying to understand if the following is good or bad practice.
If it turns out that this is good / normal practice, it makes it easier for me to implement multiple database connectors into my project.
Situation: I have created an API server and every time a call is done to the API this piece of code runs:
ctx := context.Background()
client, err := datastore.NewClient(ctx, "foobar")
if err != nil {
log.Fatalf("Failed to create client: %v", err)
}
The service gets many requests per second and I'm not sure if I should run the newClient() function only once on startup, or every time the API is called.
PS:
The same would go if another connector would be MySQL. Every time the API gets a request, the following code will run:
db, err = sql.Open("mysql", "yourusername:yourpassword@/yourdatabase")
if err != nil {
panic(err.Error())
}
I'm not sure about datastore.Client, but database.DB manages a pool of connections, is safe for concurrent use, and you shouldn't really call db.Open every time you want to execute a query. In fact, the last paragraph from Open's documentation states that clearly.
The returned DB is safe for concurrent use by multiple goroutines and maintains its own pool of idle connections. Thus, the Open function should be called just once. It is rarely necessary to close a DB.
Update:
If you take a look at the comment in the example code of the Basic Operations section in the datastore's documentation, you'll find the recommended use.
Create a datastore client. In a typical application, you would create a single client which is reused for every datastore operation.
I don't have any experience with Google Datastore but I expect it to be designed similarly to database/sql package. And in that case it's not terrible to use sql.Open on every request as connections are initialized lazily and it doesn't validate the params. But it's much better to reuse it as it is safe to be used concurrently.
Although it’s idiomatic to Close() the database when you’re finished with it, the sql.DB object is designed to be long-lived. Don’t Open() and Close() databases frequently. Instead, create one sql.DB object for each distinct datastore you need to access, and keep it until the program is done accessing that datastore. Pass it around as needed, or make it available somehow globally, but keep it open. And don’t Open() and Close() from a short-lived function. Instead, pass the sql.DB into that short-lived function as an argument.
If you don’t treat the sql.DB as a long-lived object, you could experience problems such as poor reuse and sharing of connections, running out of available network resources, or sporadic failures due to a lot of TCP connections remaining in TIME_WAIT status. Such problems are signs that you’re not using database/sql as it was designed.
from: http://go-database-sql.org/accessing.html
By default it is backed by a connection pool without a limit for connections. So you may starve it with enough concurrent goroutines (not to mention some database systems have pretty heavy connection initialization). However, you can limit the pool size pretty easily: http://go-database-sql.org/connection-pool.html
This is a preference based answer, but here it is for what it's worth.
Use package github.com/mjibson/goon for all your datastore needs. It wraps up calls like Get and Put really nicely and will Marshal/Unmarshal structs for you. It also runs any key queries through memcache automatically to speed things up.
So back to your actual question. It is appropriate with goon to run one of these two lines upon each request.
g := goon.FromContext(c) // if you have Context
g := goon.NewGoon(req) // if you have http.Request