Problem

Found many similar questions (title) but none solved my problem, so here is it.

I have a JSON string that contains some known fields (should always be present) plus any number of unknown/arbitrary fields.

Example

{"known1": "foo", "known2": "bar", "unknown1": "car", "unknown2": 1}

In this example known1 and known2 are known fields. unknown1 and unknown2 are arbitrary/unknown fields.

The unknown fields can have any name (key) and any value. The value type can be either a string, bool, float64 or int.

What I want is to find the simplest and most elegant (idiomatic) way to parse a message like this.

My solution

I've used the following struct:

type Message struct {
    Known1   string `json:"known1"`
    Known2   string `json:"known2"`
    Unknowns []map[string]interface{}
}

Expected result

With this struct and the above sample JSON message I want to achieve a Message like the following (output from fmt.Printf("%+v", msg)):

{Known1:foo Known2:bar Unknowns:[map[unknown1:car] map[unknown2:1]]}

Attempts

1. Simple unmarshal

https://play.golang.org/p/WO6XlpK_vJg

This doesn't work, Unknowns is not filled with the remaining unknown key/value pairs as expected.

2. Double unmarshal

https://play.golang.org/p/Mw6fOYr-Km8

This works but I needed two unmarshals, one to fill the known fields (using an alias type to avoid an infinite loop) and a second one to get all fields as a map[string]interface{} and process the unknowns.

3. Unmarshal and type conversion

https://play.golang.org/p/a7itXObavrX

This works and seems the best option among my solutions.

Any other option?

Option 2 and 3 work but I'm curious if anyone has a simpler/more elegant solution.

TMTOWTDI, and I think you found a reasonable solution. Another option you could consider -- which I guess is similar to your option 2 -- is to unmarshal it onto a map[string]interface{}.

Then range over the keys and values and do whatever you need to with the data, running type assertions on the values as necessary. Depending on what you need to do, you may skip the struct entirely, or still populate it.

package main

import (
    "encoding/json"
    "fmt"
)

func main() {
    jsonMsg := `{"known1": "foo", "known2": "bar", "unknown1": "car", "unknown2": 1}`
    var msg map[string]interface{}
    fmt.Println(json.Unmarshal([]byte(jsonMsg), &msg))
    fmt.Printf("%+v", msg)
}

prints

<nil>
map[known1:foo known2:bar unknown1:car unknown2:1]

https://play.golang.org/p/Bl99Cq5tFWW

Probably there is no existing solution just for your situation. As an addition to your solutions you may try to use raw parsing libraries like:

• https://github.com/buger/jsonparser
• https://github.com/json-iterator/go
• https://github.com/nikandfor/json

Last one is mine and it has some unfinished features, but raw parsing is done.

You may also use reflection with libraries above to set known fields in the struct.