As far as it is my understanding, the only way I can set a proxy for a request is to assign it to a client and then use that for a request. However, I also understand I should reuse the clients for my requests. Is there any way to individually assign proxies for each request whilst using the same client? Or any way to efficiently do a collection of requests with a different proxy for each request?
Creating of http.Client is not so expensive - its just a struct allocation that can be done for each request.
You should create http.Client and transport objects yourself and set proxy directly there.
proxyUrl, err := url.Parse("http://127.0.0.1:123")
if err != nil {
panic(err)
}
myClient := &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(proxyUrl)}}
myClient.Get("http://www.google.com/")
Tere is the other posibility to use always the same client with different proxies:
http.ProxyURL call from the first example is function:
// ProxyURL returns a proxy function (for use in a Transport)
// that always returns the same URL.
func ProxyURL(fixedURL *url.URL) func(*Request) (*url.URL, error) {
return func(*Request) (*url.URL, error) {
return fixedURL, nil
}
}
You can extend it and chose proxy based on the information from request or implement some other logic.
UPD: here is implementation how to use ProxyUrl with only one Client. But overehead of changing context is comparable with overhead on creating a client. I would use the first scenario.
// We must write our own Get (or some other) request and add somewere the info that this is a proxy request. You can do it via context of request.
func GetWithProxy(c *http.Client,proxyUrl *url.URL, url string) (resp *http.Response, err error) {
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
if proxyUrl != nil {
req = req.WithContext(context.WithValue(context.Background(),"proxy",proxyUrl))
}
return c.Do(req)
}
// Organize client that can use this information:
proxyUrl, err := url.Parse("http://127.0.0.1:123")
if err != nil {
panic(err)
}
myClient := &http.Client{Transport: &http.Transport{Proxy: func(req *http.Request) (*url.URL, error) {
currentProxy,_ := req.Context().Value("proxy").(*url.URL)
return currentProxy, nil
}}}
Use it:
GetWithProxy(myClient,nil,"http://www.google.com/")
GetWithProxy(myClient,proxyUrl,"http://www.google.com/")