I try to improve the user experience of a cli I maintain. A main goal is to provide reasonable defaults. It uses yaml
extensively for configuration.
A basic demo implementation of the configuration can be found here: https://github.com/unprofession-al/configuration/tree/bf5a89b3eee7338899b28c047f3795546ce3d2e6
General
The main configuration looks like this:
type Config map[string]ConfigSection
type ConfigSection struct {
Input InputConfig `yaml:"input"`
Output OutputConfig `yaml:"output"`
}
Config
holds a bunch of ConfigSections
. This allow the user to define variations of the configuration (lets say prod
, dev
and testing
for example) and use YAML Achors to do so.
The parts of the ConfigSection
(Input
and Output
) would be defined in the package that consumes the configuration. Each of this part provides a Defaults()
and a custom UnmarshalYAML()
func. Also ConfigSection
itself provides an UnmarshalYAML()
func. This idea is stolen from https://github.com/go-yaml/yaml/issues/165#issuecomment-255223956.
Question
In data.go
in the repo some test input as well as the expected output is defined. Running the tests (go test -v
) shows:
empty
example) no defaults are applied.ConfigSection
) with no data field is defined, this part will have no defaults. The "undefined" part has defaults (see input
, output
).both
) but have no data fields, any defaults are set.I see no pattern at all and ran out of ideas why this works like this and how to get the expected results (eg. get the test to pass).
Ok, so the pattern I did not see was quite obviouse: The "deepest leaf" of a configuration overwrites everything below, either with the given data or with the go defaults for empty values:
That means that a struct like this...
[key_string]:
input:
listener: [string]
static: [string]
output:
listener: [string]
details:
filter: [string]
retention: [string]
... defaulted with the data...
defaults:
input:
listener: 127.0.0.1:8910
static: default
output:
listener: 127.0.0.1:8989
details:
filter: '*foo*'
retention: 3h
... feeded with a yaml of this form..
empty:
both:
input:
output:
input: &input
input:
input-modified-with-anchor:
<<: *input
input:
static: NOTDEFAULT
input-modified-without-anchor:
input:
static: NOTDEFAULT
output: &output
output:
output-modified-with-anchor:
<<: *output
output:
details:
filter: NOTDEFAULT
output-modified-without-anchor:
output:
details:
filter: NOTDEFAULT
... turn out as...
both:
input:
listener: ""
static: ""
output:
listener: ""
details:
filter: ""
retention: ""
empty:
input:
listener: ""
static: ""
output:
listener: ""
details:
filter: ""
retention: ""
input:
input:
listener: ""
static: ""
output:
listener: 127.0.0.1:8989
details:
filter: '*foo*'
retention: 3h
input-modified-with-anchor:
input:
listener: 127.0.0.1:8910
static: NOTDEFAULT
output:
listener: 127.0.0.1:8989
details:
filter: '*foo*'
retention: 3h
input-modified-without-anchor:
input:
listener: 127.0.0.1:8910
static: NOTDEFAULT
output:
listener: 127.0.0.1:8989
details:
filter: '*foo*'
retention: 3h
For my use case this is a too complicated behavior, I therefore try a different approach: I if demanded I inject a default config into the yaml and put a reference to its anchor in each section. I feel like this is more transparent and reproducible to the end user. Heres an ugly draft of the func:
func injectYAML(data []byte) ([]byte, error) {
// render a default section an add an anchor
key := "injected_defaults"
defaultData := Config{key: Defaults()}
var defaultSection []byte
defaultSection, _ = yaml.Marshal(defaultData)
defaultSection = bytes.Replace(defaultSection, []byte(key+":"), []byte(key+": &"+key), 1)
// get list of sections in input data
c := Config{}
err := yaml.Unmarshal(data, &c)
if err != nil {
return data, fmt.Errorf("Error while reading sections from yaml: %s", err.Error())
}
// remove "---" at beginning when present
data = bytes.TrimLeft(data, "---")
// add reference to default section to each section
lines := bytes.Split(data, []byte("
"))
var updatedLines [][]byte
for _, line := range lines {
updatedLines = append(updatedLines, line)
for section := range c {
if bytes.HasPrefix(line, []byte(section+":")) {
updatedLines = append(updatedLines, []byte(" <<: *"+key))
}
}
}
updatedData := bytes.Join(updatedLines, []byte("
"))
// compose injected yaml
out := []byte("---
")
out = append(out, defaultSection...)
out = append(out, updatedData...)
return out, nil
}
Full example at: https://github.com/unprofession-al/configuration/tree/7c2eb7da58b51f52b50f2a0fbac193c799c9eb08