If I have a csv read into a struct how can I manipulate the input to build the struct how I want? I am getting stuck in circles following various tutorials. This is the closest I have come.
I essentially want to open a csv, read selected columns, ensure the value is recorded from the same row when referencing the column. Then the resulting data in a format which can be put into a database.
Example CSV:
Ignore,Customer,Fruit,Number
123,A,Apple,1
123,A,Apple,3
123,B,Orange,4
123,C,Melon,5
Example Code:
package main
import (
"bufio"
"encoding/csv"
"encoding/json"
"fmt"
"io"
"log"
"os"
)
type Account struct {
Customer string `json:"Customer"`
LineItem *LineItem `json:"LineItem"`
}
type LineItem struct {
ProductName string `json:"ProductName"`
Count string `json:"Count"`
}
func main() {
csvFile, _ := os.Open("/home/frank/gocode/src/local/billing/fruit.csv")
reader := csv.NewReader(bufio.NewReader(csvFile))
var billData []Account
for {
line, error := reader.Read()
if error == io.EOF {
break
} else if error != nil {
log.Fatal(error)
}
billData = append(billData, Account{
Customer: line[1],
LineItem: &LineItem{
ProductName: line[2],
Count: line[3],
},
})
}
billingJson, _ := json.Marshal(billData)
fmt.Println(string(billingJson))
}
The current output is:
[{"Customer":"Customer","LineItem":{"ProductName":"Fruit","Count":"Number"}},{"Customer":"A","LineItem":{"ProductName":"Apple","Count":"1"}},{"Customer":"A","LineItem":{"ProductName":"Apple","Count":"3"}},{"Customer":"B","LineItem":{"ProductName":"Orange","Count":"4"}},{"Customer":"C","LineItem":{"ProductName":"Melon","Count":"5"}}]
I would like to get rid of first record so the headers are not kept. e.g.
[{"Customer":"A","LineItem":{"ProductName":"Apple","Count":"1"}},{"Customer":"A","LineItem":{"ProductName":"Apple","Count":"3"}},{"Customer":"B","LineItem":{"ProductName":"Orange","Count":"4"}},{"Customer":"C","LineItem":{"ProductName":"Melon","Count":"5"}}]
Consolidate so Customer A is one record with both LineItems e.g.
[{"Customer":"A","LineItem":{"ProductName":"Apple","Count":"1"},"LineItem":{"ProductName":"Apple","Count":"3"}},{"Customer":"B","LineItem":{"ProductName":"Orange","Count":"4"}},{"Customer":"C","LineItem":{"ProductName":"Melon","Count":"5"}}]
Any best practices - alternate methods welcomed (not sure if a map is better here). Hopefully enough info to give me a hand.
Getting rid of the first entry is as easy as billData = billData[1:]
. That, or do an initial read to pull the column names.
On the second part, your current data structure does not tolerate a one-to-many relationship (each Account has one and only one LineItem). You'll need to do some processing on the list afterwards. CSV files are necessarily 1:1, as each line is considered a single independent record. The easiest way is to make it one-to-many is by using a map, but you can also simply loop over a slice (which retains closer to your existing code):
https://play.golang.org/p/3uevo0taKR5
package main
import (
"bytes"
"encoding/csv"
"encoding/json"
"fmt"
"io"
"log"
)
var data = `Ignore,Customer,Fruit,Number
123,A,Apple,1
123,A,Apple,3
123,B,Orange,4
123,C,Melon,5`
type Account struct {
Customer string `json:"Customer"`
LineItems []LineItem `json:"LineItems"`
}
type LineItem struct {
ProductName string `json:"ProductName"`
Count string `json:"Count"`
}
func main() {
reader := csv.NewReader(bytes.NewBufferString(data))
// Read column label data and discard
if _, err := reader.Read(); err != nil {
log.Fatal(err)
}
var billData []Account
for {
line, err := reader.Read()
if err == io.EOF {
break
}
if err != nil {
log.Fatal(err)
}
found := false
for i := range billData {
if billData[i].Customer == line[1] {
found = true
billData[i].LineItems = append(billData[i].LineItems, LineItem{
ProductName: line[2],
Count: line[3],
})
break
}
}
if !found {
billData = append(billData, Account{
Customer: line[1],
LineItems: []LineItem{
{
ProductName: line[2],
Count: line[3],
},
},
})
}
}
billingJson, err := json.MarshalIndent(billData, "", " ")
if err != nil {
log.Fatal(err)
}
fmt.Println(string(billingJson))
}
Output:
[
{
"Customer": "A",
"LineItems": [
{
"ProductName": "Apple",
"Count": "1"
},
{
"ProductName": "Apple",
"Count": "3"
}
]
},
{
"Customer": "B",
"LineItems": [
{
"ProductName": "Orange",
"Count": "4"
}
]
},
{
"Customer": "C",
"LineItems": [
{
"ProductName": "Melon",
"Count": "5"
}
]
}
]
Lastly, I recommend using err
or similar for your error variable. error
is the name of the built in error type, so by naming your variable that, you're shadowing the type and making it impossible to declare a variable of that type within the same scope. While this doesn't affect your current code, it's still quite bad practice and liable to get you into trouble eventually.