I am attempting to create a wrapper for test emulating around the Go Flex SDK for Google Cloud Datastore. While I am currently successfully running the localhost emulator using
gcloud beta emulators datastore start --no-store-on-disk
in a separate terminal from my testing window, I would prefer to create a mock database emulator that runs as part of the test process itself (without execing the above) so that I can run multiple tests in parallel, each with its own database emulator.
I have run into a problem with the Google SDK not implementing my interface.
My wrapper contains this code:
package google
import (
"context"
"cloud.google.com/go/datastore"
)
type (
// Datastore is a wrapper for the Google Cloud Datastore Client.
Datastore datastore.Client
// Datastorer represents things that can operate like a datastore.Client.
Datastorer interface {
Delete(context.Context, *datastore.Key) error
Get(context.Context, *datastore.Key, interface{}) error
GetAll(context.Context, *datastore.Query, interface{}) ([]*datastore.Key, error)
Put(context.Context, *datastore.Key, interface{}) (*datastore.Key, error)
PutMulti(context.Context, []*datastore.Key, interface{}) ([]*datastore.Key, error)
RunInTransaction(context.Context, func(Transactioner) error, ...datastore.TransactionOption) (*datastore.Commit, error)
}
// Transactioner represents things that can operate like a datastore.Transaction.
Transactioner interface {
Commit() (*datastore.Commit, error)
Delete(*datastore.Key) error
DeleteMulti([]*datastore.Key) error
Get(*datastore.Key, interface{}) error
GetMulti([]*datastore.Key, interface{}) error
Put(*datastore.Key, interface{}) (*datastore.PendingKey, error)
PutMulti([]*datastore.Key, interface{}) ([]*datastore.PendingKey, error)
Rollback() error
}
)
// Delete deletes the entity for the given key.
func (d *Datastore) Delete(ctx context.Context, key *datastore.Key) error {
return (*datastore.Client)(d).Delete(ctx, key)
}
// Get retrieves the entity for the given key.
func (d *Datastore) Get(ctx context.Context, key *datastore.Key, dst interface{}) error {
return (*datastore.Client)(d).Get(ctx, key, dst)
}
// GetAll retrieves all entities for the given query.
func (d *Datastore) GetAll(ctx context.Context, q *datastore.Query, dst interface{}) ([]*datastore.Key, error) {
return (*datastore.Client)(d).GetAll(ctx, q, dst)
}
// Put stores an entity for the given key.
func (d *Datastore) Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error) {
return (*datastore.Client)(d).Put(ctx, key, src)
}
// PutMulti is a batch version of Put.
func (d *Datastore) PutMulti(ctx context.Context, keys []*datastore.Key, src interface{}) ([]*datastore.Key, error) {
return (*datastore.Client)(d).PutMulti(ctx, keys, src)
}
// RunInTransaction runs the given function in a transaction.
func (d *Datastore) RunInTransaction(ctx context.Context, f func(tx Transactioner) error, opts ...datastore.TransactionOption) (*datastore.Commit, error) {
return (*datastore.Client)(d).RunInTransaction(ctx, func(t *datastore.Transaction) error {
return f(t)
}, opts...)
}
Note that these interfaces do not emulate the complete SDK. I am only including functions that I actually call in my code. I'll add new ones as needed later.
When I try to use an instance of *datastore.Client as a Datastorer, I get the following error:
cannot use client (type *"cloud.google.com/go/datastore".Client) as type Datastorer in field value:
*"cloud.google.com/go/datastore".Client does not implement Datastorer (wrong type for RunInTransaction method)
have RunInTransaction(context.Context, func(*"cloud.google.com/go/datastore".Transaction) error, ..."cloud.google.com/go/datastore".TransactionOption) (*"cloud.google.com/go/datastore".Commit, error)
want RunInTransaction(context.Context, func(Transactioner) error, ..."cloud.google.com/go/datastore".TransactionOption) (*"cloud.google.com/go/datastore".Commit, error)
because *datastore.Client requires a function that takes a func(*datastore.Transaction) error and my interface wants a func(Transactioner) error.
Is there any way to change this so that it compiles?
If I can get it working, I plan to create types that implement my Datastorer and Transactioner interfaces and use maps to mock the real database. As far as tranactions go, for testing I can use sync.Mutex if I need them, but since each test is a single thread and will get its own database object, I may not need to lock them.
I've gotten it to compile by using this code:
package google
import (
"context"
"cloud.google.com/go/datastore"
)
type (
// Datastore is a wrapper for the Google Cloud Datastore Client.
Datastore struct {
*datastore.Client
}
// Datastorer represents things that can operate like a datastore.Client.
Datastorer interface {
Delete(context.Context, *datastore.Key) error
Get(context.Context, *datastore.Key, interface{}) error
GetAll(context.Context, interface{}, interface{}) ([]*datastore.Key, error)
Put(context.Context, *datastore.Key, interface{}) (*datastore.Key, error)
PutMulti(context.Context, []*datastore.Key, interface{}) ([]*datastore.Key, error)
RunInTransaction(context.Context, func(Transactioner) error, ...datastore.TransactionOption) (*datastore.Commit, error)
}
// Querier represents things that can operate like a datastore.Query.
Querier interface {
Filter(string, interface{}) Querier
}
// Transactioner represents things that can operate like a datastore.Transaction.
Transactioner interface {
Commit() (*datastore.Commit, error)
Delete(*datastore.Key) error
DeleteMulti([]*datastore.Key) error
Get(*datastore.Key, interface{}) error
GetMulti([]*datastore.Key, interface{}) error
Put(*datastore.Key, interface{}) (*datastore.PendingKey, error)
PutMulti([]*datastore.Key, interface{}) ([]*datastore.PendingKey, error)
Rollback() error
}
)
// Delete deletes the entity for the given key.
func (d *Datastore) Delete(ctx context.Context, key *datastore.Key) error {
return d.Client.Delete(ctx, key)
}
// Get retrieves the entity for the given key.
func (d *Datastore) Get(ctx context.Context, key *datastore.Key, dst interface{}) error {
return d.Client.Get(ctx, key, dst)
}
// GetAll retrieves all entities for the given query.
func (d *Datastore) GetAll(ctx context.Context, q interface{}, dst interface{}) ([]*datastore.Key, error) {
return d.Client.GetAll(ctx, q.(*datastore.Query), dst)
}
// Put stores an entity for the given key.
func (d *Datastore) Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error) {
return d.Client.Put(ctx, key, src)
}
// PutMulti is a batch version of Put.
func (d *Datastore) PutMulti(ctx context.Context, keys []*datastore.Key, src interface{}) ([]*datastore.Key, error) {
return d.Client.PutMulti(ctx, keys, src)
}
// RunInTransaction runs the given function in a transaction.
func (d *Datastore) RunInTransaction(ctx context.Context, f func(tx Transactioner) error, opts ...datastore.TransactionOption) (*datastore.Commit, error) {
return d.Client.RunInTransaction(ctx, func(t *datastore.Transaction) error {
return f(t)
}, opts...)
}
I changed DataStore to a struct containing the datastore.Client and added a new interface Querier that contains the functions that I am using from datastore.Query. I also updated GetAll to accept an interface{} instead of a *datastore.Query and then type-assert it to be a *datastore.Query. I cannot have it accept a Querier because then I cannot pass variables of type *datastore.Query because they do not satisfy the Querier interface (Filter returns a Querier instead of a *datastore.Query).
All existing tests using the emulator running in a separate process are passing.
UPDATE:
I changed Datastore to
Datastore datastore.Client
and added a wrapper Query around datastore.Query:
Query datastore.Query
Now, the Datastorer interface contains
GetAll(context.Context, Querier, interface{}) ([]*datastore.Key, error)
and the GetAll function is defined as
func (d *Datastore) GetAll(ctx context.Context, q Querier, dst interface{}) ([]*datastore.Key, error) {
return (*datastore.Client)(d).GetAll(ctx, (*datastore.Query)(q.(*Query)), dst)
}
and Query.Filter is defined as
func (q *Query) Filter(filterStr string, value interface{}) Querier {
return (*Query)((*datastore.Query)(q).Filter(filterStr, value))
}
In calling code, I use
q := datastore.NewQuery(entity).Filter("Deleted =", false)
_, err := r.client.GetAll(ctx, (*Query)(q), data)
This compiles and all tests are passing.
I know that question has been asked a long time ago but in case one still may wondering how to Mock Google Datastore Client and Transaction here is a snapshot of how I got it to work.
type Client interface {
Get(ctx context.Context, key *datastore.Key, dst interface{}) (err error)
NewTransaction(ctx context.Context, opts ...datastore.TransactionOption) (t Transaction, err error)
Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error)
}
type Transaction interface {
Commit() (c *datastore.Commit, err error)
Rollback() (err error)
Get(key *datastore.Key, dst interface{}) (err error)
Put(key *datastore.Key, src interface{}) (*datastore.PendingKey, error)
Delete(key *datastore.Key) error
}
type Datastore struct {
*datastore.Client
}
func (d *Datastore) NewTransaction(ctx context.Context, opts ...datastore.TransactionOption) (t Transaction, err error) {
return d.Client.NewTransaction(ctx, opts...)
}
Of course, If you are using different method against Datastore it's up to you to implement them.
In my tests, I can now Mock Datastore Client and Transaction like :
mockedClient := mock_gcloud.NewMockClient(ctrl)
mockedTransaction := mock_gcloud.NewMockTransaction(ctrl)
...
mockedClient.EXPECT().NewTransaction(context.Background()).Return(mockedTransaction, nil).Times(1)
Et voilà.