Is there a Golang API to convert data to a serialized tf.Example, also known as tensorflow tfrecords.
I can find a Python api tf.python_io.TFRecordWriter()
to achieve this.
But examples with golang can not be found.
I want do this coz i use a golang client to call the tensorflow serving and my input feature for nn is a SparseTensor.
tf.Example
refers to a protocol buffer, while tfrecords
typically refers to a file format which stores "records" in the form of strings (which is why tf.python_io.TFRecordWriter.write()
takes a string). So the two are separate things, not technically related. Though often one serializes tf.Example
protocol buffers into a strings, one each per records in a TFRecordWriter
.
That said, it is common for models to take as input a serialized tf.Example
protocol buffer as input in the form of a STRING
tensor. If that is the case, then you'd want to construct the tf.Example
protocol buffer in Go and then use proto.Marshal
to construct the tf.Tensor
object to feed.
Unfortunately, as of January 2018, you have to generate the Go files for the tf.Example
protos yourself. That can be done with something like:
# Fetch the tools
GOPATH=$(go env GOPATH)
go get github.com/golang/protobuf/proto
go get github.com/golang/protobuf/protoc-gen-go
# Create a directory to generate the files
mkdir -p /tmp/protos/out
cd /tmp/protos
# Clone the TensorFlow sources to get the source .proto files
git clone https://github.com/tensorflow/tensorflow ./src
PATH=$PATH:${GOPATH}/bin
# Generate Go source files from the .proto files.
# Assuming protoc is installed.
# See https://github.com/google/protobuf/releases
protoc \
-I ./src \
--go_out ./out \
./src/tensorflow/core/framework/*.proto ./src/tensorflow/core/example/*.proto
rm -rf ./src
This will generate a bunch of .pb.go
files in /tmp/protos/out
which can be used to build the tf.Example
protocol buffer structures to marshal.
Hope that gives you enough information to get going.