I have written an example of Golang code which is sends query to postgres and send result to the pager:
package main
import (
"fmt"
"database/sql"
_ "github.com/lib/pq"
"log"
"os/exec"
"strings"
"os"
)
func main() {
connstr := "user=postgres dbname=postgres sslmode=disable"
db, err := sql.Open("postgres", connstr)
if err != nil { log.Fatal(err) }
rows, err := db.Query("SELECT schemaname, relname, seq_scan FROM pg_stat_all_tables ORDER BY 1 LIMIT 10")
if err != nil { log.Fatal(err) }
defer rows.Close()
var buf string
for rows.Next() {
var s, r string
var ss int
if err := rows.Scan(&s, &r, &ss); err != nil { log.Fatal(err) }
buf = fmt.Sprintf("%s %s %d
", buf + s, r, ss)
}
cmd := exec.Command("less")
cmd.Stdin = strings.NewReader(buf)
cmd.Stdout = os.Stdout
err = cmd.Run()
if err != nil { log.Fatal(err) }
}
But the following line:
buf = fmt.Sprintf("%s %s %d
", buf + s, r, ss)
looks rude for me and I'm not sure this is a right way. Is there way to achieve result in more elegant way? May be it's possible with some kind of buffers and io.Readers?
Strings in Go are immutable and every time when you assign a new value to a variable it must create a new string and copy contents of existing one to it.
You can use bytes.Buffer
instead of string
to avoid recreation on each iteration.
package main
import (
"fmt"
"database/sql"
_ "github.com/lib/pq"
"log"
"os/exec"
"strings"
"os"
"bytes"
)
func main() {
connstr := "user=postgres dbname=postgres sslmode=disable"
db, err := sql.Open("postgres", connstr)
if err != nil { log.Fatal(err) }
rows, err := db.Query("SELECT schemaname, relname, seq_scan FROM pg_stat_all_tables ORDER BY 1 LIMIT 10")
if err != nil { log.Fatal(err) }
defer rows.Close()
var buf = new(bytes.Buffer)
for rows.Next() {
var s, r string
var ss int
if err := rows.Scan(&s, &r, &ss); err != nil { log.Fatal(err) }
buf.WriteString(fmt.Sprintf("%s %s %d
", s, r, ss))
}
cmd := exec.Command("less")
cmd.Stdin = buf
cmd.Stdout = os.Stdout
err = cmd.Run()
if err != nil { log.Fatal(err) }
}
Btw, the string builder was added in Go 1.10 https://godoc.org/strings#Builder
Read more about string concatenation benchmarks: http://herman.asia/efficient-string-concatenation-in-go
The only problem with what you're doing now is that the concatenation operator +
is an extremely inefficient way to combine a lot of strings into one bigger string.
How inefficient? Well here's a benchmark I did testing three approaches:
BenchmarkMultiplyBasic 300000 4240 ns/op
BenchmarkMultiplyJoinBasic 200000 9942 ns/op
BenchmarkMultiplyConcatenationsBasic 10000 170523 ns/op
The last one is the concatenate operator +
and shows truly miserable performance compared to a couple alternatives.
Here's one approach that will be more efficient, in a simplified runnable example:
package main
import(
"fmt"
"strconv"
"strings"
)
type result struct {
s, r string
ss int
}
func main() {
a := []result{
{"twas", "brillig", 1},
{"and", "the", 2},
{"slithy", "toves", 3},
}
outstrings := make([]string, 0)
for _, part := range a {
outstrings = append(outstrings, part.s, part.r, strconv.Itoa(part.ss))
}
out := strings.Join(outstrings, ` `)
fmt.Printf("%s
", out)
}
prints
twas brillig 1 and the 2 slithy toves 3
How to most efficiently combine strings is a common question on StackOverflow and has been answered many times. See this top-voted question/answer for Go: How to efficiently concatenate strings in Go?.