慢的JSON封送和解封

My project has a requirement API to return an array(about 500 element struct)

I have tried to use lib like fasthttp, easyjson, rapidjson(call with cgo),but result is not good enough.

Do you guys have any better recommendation?

Here is my code:

type Line struct {
    Time   string  `json:"time" bson:"time"`
    Open   float64 `json:"open" bson:"open"`
    Close  float64 `json:"close" bson:"close"`
    High   float64 `json:"high" bson:"high"`
    Low    float64 `json:"low" bson:"low"`
    Volume float64 `json:"volume" bson:"volume"`
    Amount float64 `json:"amount" bson:"amount"`
}

type MultiLines struct {
    AllLines []Line `json:"lines"`
}

Test Code:

func BenchmarkJson500(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _, err := json.Marshal(&sliceData)
        if err != nil {
            panic(err)
        }
    }
}

func BenchmarkUnmarshalJson500(b *testing.B) {
    lines := make([]Line, 500)
    for i := 0; i < b.N; i++ {
        err := json.Unmarshal(sliceJson, &MultiLines{lines})
        if err != nil {
            panic(err)
        }
    }
}

func BenchmarkEasyJson500(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _, err := sliceData.MarshalJSON()
        if err != nil {
            panic(err)
        }
    }
}

func BenchmarkEasyUnmarshalJson500(b *testing.B) {
    for i := 0; i < b.N; i++ {
        slice := MultiLines{}
        err := slice.UnmarshalJSON(sliceJson)
        if err != nil {
            panic(err)
        }
    }
}

And benchmark tests result:

BenchmarkUnmarshalJson500-4          500           2821450 ns/op
BenchmarkJson500-4                   500           2151984 ns/op

Because EasyJson Rewrite the UnmarshalJSON/MarshalJSON,so i test with the generated code at different time.

BenchmarkEasyJson500-4              1000           1434724 ns/op
BenchmarkEasyUnmarshalJson500-4     1000           1276298 ns/op

Anyway,ffjson is very similar with easyjson.

What do you mean by "the results are not good enough"?

I'd use the the json package of the standard library:

https://golang.org/pkg/encoding/json/

As far as I can tell, it's used everywhere and pretty fast and resource-friendly. If that does not cut it either, you might want to consider ffjson:

https://github.com/pquerna/ffjson

Let me know if this was helpfull!

While using easyjson you should use easyjson.Unmarshal() function instead of json.Unmarshal(). Generated custom MarshalJSON()/UnmarshalJSON() are only a small part of performance optimization (they don't use reflect package). The main profit from easyjson you get from optimized easyjson.Unmarshal() and easyjson.Marshal(). First one doesn't make JSON pre-validation which is very slow. The second one uses concurrency to boost encoding.

From my experience easyjson boosts unmarshaling up to 2 times for small JSON or JSON with simple structure (e.g. array of simple objects), and up to 4 times for big or/and deep-nested JSON structures. Unfortunately, I don't have any statistics for marshaling.

Try fastjson. It should parse the input string much faster than encoding/json and faster than easyjson. Additionally, it doesn't require pre-defining Go structs for the parsed JSON and doesn't need code generation. See the example below:

var p fastjson.Parser
v, err := p.Parse(input)
if err != nil {
    log.Fatal(err)
}

for _, vv := range v.GetArray() {
    fmt.Printf("time=%s
", vv.GetStringBytes("time"))
    fmt.Printf("open=%f
", vv.GetFloat64("open"))
    // etc...
    // There is no need in extracting all the fields -
    // extract only required fields in order to get better performance.
}