I have a file with JSON in this format:
[{
"id": "1055972353245622272",
"lang": "und",
"date": "Sat Oct 27 00:00:02 +0000 2018",
"text": "#BTC 6474 346 0 08 #ETH 203 317 0 13 #XRP 0 459 0 04 #BCH 438 922 0 0 #EOS 5 388 0 12 #XLM 0 235 0 41 #LTC 52 106 0 03 #ADA 0 074 0 17 #USDT 0 99 0 07 #XMR 105 022 0 13 #TRX 0 024 0 21 "
},
{
"id": "1055972355506401280",
"lang": "en",
"date": "Sat Oct 27 00:00:03 +0000 2018",
"text": "Don t want to miss any of our public #crypto trading #signals Want instant updates of our premium channel #performance Searching for #crypto news Get instantly notified on our public telegram channel Join now at https t co akfmLiArya #DGB #SC #MFT #EOS #XVG #BTC #TRX https t co HT2RAOIjfh"
},
This file is being worked on by program1 in random intervals (when a tweet that match filter is found). I want to read this file by program2 - in 5 minutes timesteps. But I am unable to.
The unmarshaling (json.Unmarshal(file, &data)
) is not allowing me to read it - as it throws an error as JSON is not correct.
I do not want to redesign the architecture with the usage of DB, I want to be able to operate on files as intended.
How can I access the files and have them parsed as JSON?
EDIT1: workaround with reading the file and closing the JSON
file, _ := ioutil.ReadFile(fileName)
closingJson := "{}]"
file = append(file, closingJson...)
json.Unmarshal(file, &data)
You just need to treat it as a JSON stream:
https://play.golang.org/p/6drcizYKrrJ
type Message struct {
Id string `json:"id"`
Lang string `json:"lang"`
Date string `json:"date"`
Text string `json:"text"`
}
jsonStream, err := os.Open(`/tmp/json`)
if err != nil {
panic(err)
}
dec := json.NewDecoder(jsonStream)
// read open bracket
_, err := dec.Token()
if err != nil {
log.Fatal(err)
}
// while the array contains values
for dec.More() {
var m Message
// decode an array value (Message)
err := dec.Decode(&m)
if err == nil {
fmt.Printf("%v : %v : %v : %v
", m.Id, m.Lang, m.Date, m.Text)
} else {
// wait for more contents - sleep? use a channel and wait to be notified?
}
}