I'm trying file download with golang.
I'm downloading file it's okay. After I'm using cheggaaa's progressbar library. But I can't dynamic.
How can I do dynamic progressbar?
My code below:
package main
import (
"flag"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"github.com/cheggaaa/pb"
"time"
)
/*
usage = usage text
version = current number
help use Sprintf
*cliUrl from cmd
*cliVersion from cmd
*cliHelp * from cmd
*/
var (
usage = "Usage: ./gofret -url=http://some/do.zip"
version = "Version: 0.1"
help = fmt.Sprintf("
%s
%s", usage, version)
cliUrl *string
cliVersion *bool
cliHelp *bool
)
func init() {
/*
if *cliUrl != "" {
fmt.Println(*cliUrl)
}
./gofret -url=http://somesite.com/somefile.zip
./gofret -url=https://github.com/aligoren/syspy/archive/master.zip
*/
cliUrl = flag.String("url", "", usage)
/*
else if *cliVersion{
fmt.Println(flag.Lookup("version").Usage)
}
./gofret -version
*/
cliVersion = flag.Bool("version", false, version)
/*
if *cliHelp {
fmt.Println(flag.Lookup("help").Usage)
}
./gofret -help
*/
cliHelp = flag.Bool("help", false, help)
}
func main() {
/*
Parse all flags
*/
flag.Parse()
if *cliUrl != "" {
fmt.Println("Downloading file")
/* parse url from *cliUrl */
fileUrl, err := url.Parse(*cliUrl)
if err != nil {
panic(err)
}
/* get path from *cliUrl */
filePath := fileUrl.Path
/*
seperate file.
http://+site.com/+(file.zip)
*/
segments := strings.Split(filePath, "/")
/*
file.zip filename lenth -1
*/
fileName := segments[len(segments)-1]
/*
Create new file.
Filename from fileName variable
*/
file, err := os.Create(fileName)
if err != nil {
fmt.Println(err)
panic(err)
}
defer file.Close()
/*
check status and CheckRedirect
*/
checkStatus := http.Client{
CheckRedirect: func(r *http.Request, via []*http.Request) error {
r.URL.Opaque = r.URL.Path
return nil
},
}
/*
Get Response: 200 OK?
*/
response, err := checkStatus.Get(*cliUrl)
if err != nil {
fmt.Println(err)
panic(err)
}
defer response.Body.Close()
fmt.Println(response.Status) // Example: 200 OK
/*
fileSize example: 12572 bytes
*/
fileSize, err := io.Copy(file, response.Body)
/*
progressbar worked after download :(
*/
var countSize int = int(fileSize/1000)
count := countSize
bar := pb.StartNew(count)
for i := 0; i < count; i++ {
bar.Increment()
time.Sleep(time.Millisecond)
}
bar.FinishPrint("The End!")
if err != nil {
panic(err)
}
fmt.Printf("%s with %v bytes downloaded", fileName, count)
} else if *cliVersion {
/*
lookup version flag's usage text
*/
fmt.Println(flag.Lookup("version").Usage)
} else if *cliHelp {
/*
lookup help flag's usage text
*/
fmt.Println(flag.Lookup("help").Usage)
} else {
/*
using help's usage text for handling other status
*/
fmt.Println(flag.Lookup("help").Usage)
}
}
While the my program is running:
Downloading file
200 OK
After download working progressbar:
6612 / 6612 [=====================================================] 100.00 % 7s
The End!
master.zip with 6612 bytes downloaded
My progressbar code below:
/*
progressbar worked after download :(
*/
var countSize int = int(fileSize/1000)
count := countSize
bar := pb.StartNew(count)
for i := 0; i < count; i++ {
bar.Increment()
time.Sleep(time.Millisecond)
}
bar.FinishPrint("The End!")
How can I solve progressbar problem?
More goroutines is not needed. Just read from bar
// start new bar
bar := pb.New(fileSize).SetUnits(pb.U_BYTES)
bar.Start()
// create proxy reader
rd := bar.NewProxyReader(response.Body)
// and copy from reader
io.Copy(file, rd)
I wrote the below stuff, and it's right in the general case, unrelated to progressbar, but this library is exactly designed to handle this problem, has specialized support for it, and gives an explicit example of downloading a file.
You need to run the download in parallel with updating the progressbar, currently you're downloading the whole file and then updating the progressbar.
This is a little sloppy, but should get you going in the right direction:
First, grab the expected filesize:
filesize := response.ContentLength
Then start your download in a goroutine:
go func() {
n, err := io.Copy(file, response.Body)
if n != filesize {
log.Fatal("Truncated")
}
if err != nil {
log.Fatalf("Error: %v", err)
}
}()
Then update your progressbar as it goes along:
countSize := int(filesize / 1000)
bar := pb.StartNew(countSize)
var fi os.FileInfo
for fi == nil || fi.Size() < filesize {
fi, _ = file.Stat()
bar.Set(int(fi.Size() / 1000))
time.Sleep(time.Millisecond)
}
bar.FinishPrint("The End!")
Like I say, this is a little sloppy; you probably want to scale the bar better depending on the size of the file, and the log.Fatal calls are ugly. But it handles the core of the issue.
Alternately, you can do this without a goroutine just by writing your own version of io.Copy. Read a block from response.Body, update the progress bar, and then write a block to file. That's arguably better because you can avoid the sleep call.