I am implementing client server communication by self defined packets. I am using Go net.conn
. It can dial tcp/unix schemes which is very convenient. I use protocol buffer
to define my messages.
I defined a Packet which contains length
and buffer
type Packet struct {
length uint32
buffer []byte
}
The API function is like this:func(api *API) Send(m *proto.Message) error
func(api *API) Receive(p *Packet) error
Take send
function as an example, it takes in a protobuf message, marshal it into Packet
. And write it to the net.conn
.
Here is a simplified version of the Send function:
func(api *API) Send(m *proto.Message) error {
bytes, err := proto.Marshal(m)
if err != nil {
return err
}
buffer := api.packet[:length]
copy(buffer, bytes)
_, err := api.conn.Write(buffer)
if err != nil {
return err
}
return nil
}
I was copying bytes
into buffer
. Because Go protocol buffer API only providesfunc Marshal(pb Message) ([]byte, error)
In protocol buffer C++, it provides bool SerializeToArray(void * data, int size) const
, which is serializing the message and storing it in the given byte array. But I can not find the same thing in Go protocol buffer API.
Is there any way to avoid the copy if I want to directly storing serialized result in the given byte array?
It is not clear what you are asking. Notice that the proto Marshal() function does exactly what you are looking for: It serializes the message into a byte slice (what you probably mean by byte array)
See if either of these help:
func(api *API) Send(m *proto.Message) error {
p := Packet{}
p.buffer, err := proto.Marshal(m)
if err != nil {
return err
}
_, err := api.conn.Write(p.buffer)
if err != nil {
return err
}
return nil
}
Or
func(api *API) Send(m *proto.Message) error {
buffer := api.packet[:length]
buffer, err := proto.Marshal(m)
if err != nil {
return err
}
_, err := api.conn.Write(buffer)
if err != nil {
return err
}
return nil
}
Seems you can make Packet.buffer
to be a proto.Buffer
type Packet struct {
length uint32
buffer proto.Buffer
}
...
var packet Packet
packet.length = YouLength
packet.buffer = proto.NewBuffer(make([]byte, YouLength))
//Then you can Marshall in Packet directly and it may be reused.
err := packet.Marshal(message)
You're looking for MarshalTo method from gogo/protobuf
, another implementation of protobuf, compatible with the original.
You can re-use the same buffer through multiple marshal calls as you pass it the buffer to be filled. Obviously the buffer should be big enough.
func MarshalTo([]byte, m) error