To give some background, I am reading device logs from android and ios devices using adb logcat and idevicesyslog. The specific logs I am after are swift/c#/java/etc dictionaries converted to strings via adb logcat / idevicesyslog. I hope to take these logs that contain JSON like strings and convert those to valid JSON. This works for the most part no problem.
However, sometimes these logs/string outputs contain characters like (\134, \M, \t, etc etc) that cause issues when unmarshalling into JSON. I unmarshal them to JSON to send them elsewhere.
For example a raw device log may have something like the following: {"foo":"bar","foo":"bar\134/\134/bar\134/bar\134/bar"} {"foo":"bar","foo":"bar\M/\134/bar\134/bar\M/bar"}
These result in errors like "panic: invalid character 'M' in string escape code" when attempting to unmarshal
The majority of logs do not contain these characters and so it's not a problem. However, a few edge cases contain these and it creates problems.
Is there a proper name for these types of characters? (c# escape characters?) Is there a golang package that can remove them from a string? Currently, I am just deleting the specific ones i come across if they appear in a string but I feel like there's a better way to do it. Adding the ones I come across to a list of deletable characters is not a good practice.
To summarize,
idevicesyslog log gives me a string like so: {"foo":"bar","foo":"bar\134/\134/bar\134/bar\134/bar"}
This can not be unmarshalled.
idevicesyslog log gives me a string like so: {"foo":"bar","foo":"bar bar bar bar"}
This can be unmarshalled.
Current solution: add new ones I come across to a list and remove them before unmarshaling
Hopeful solution: detect automatically and remove
Use a regexp to replace the invalid octal escape sequences with a space:
var octalEscapePat = regexp.MustCompile(`\\[0-7]{3}`)
func fix(src string) string {
return octalEscapePat.ReplaceAllString(src, " ")
}
You can also parse the octal value and convert to a valid JSON escape sequence:
func fix(src string) string {
return octalEscapePat.ReplaceAllStringFunc(src, func(s string) string {
// Parse octal value
n, _ := strconv.ParseInt(s[1:], 8, 0)
// Convert to string and marshal to JSON to handle any escaping
b, _ := json.Marshal(string(n))
// return string with surrounding quotes removed
return string(b[1 : len(b)-1])
})
}
The \M can be handled in a similar way.