I'm working on a small web scraper to just get a feel of golang. It currently is grabbing info off of a wiki from a table and then grabbing info specifically from cells. I don't currently have the code on me (not currently at home) but it looks fairly similar to this:
func main() {
doc, err := goquery.NewDocument("http://monsterhunter.wikia.com/wiki/MH4:_Item_List")
if err != nil {
log.Fatal(err)
}
doc.Find("tbody").Each(func(i int, s *goquery.Selection) {
title := s.Find("td").Text()
fmt.Printf(title)
})
}
The issue is that on this website the first cell is an image, so it prints the image source which I don't want. How can I ignore the first cell in each row of the large table?
Let's clear some things. A Selection
is a collection of nodes matching some criteria.
doc.Find()
is Selection.Find()
which returns a new Selection
containing the elements matching the criteria. And Selection.Each()
iterations over each of the elements of the collection and calls the function value passed to it.
So in your case Find("tbody")
will find all tbody
elements, Each()
will iterate over all tbody
elements and call your anonymous function.
Inside your anonymous function s
is a Selection
of one tbody
element. You call s.Find("td")
which will return a new Selection
which will contain all the td
elements of the current table. So when you call Text()
on this, it will be the combined text contents of each td
elements including their descendants. This is not what you want.
What you should do is call another Each()
on the Selection
returned by s.Find("td")
. And check if the Selection
passed to the 2nd anonymous function has an img
child.
Example code:
doc.Find("tbody").Each(func(i int, s *goquery.Selection) {
// s here is a tbody element
s.Find("td").Each(func(j int, s2 *goquery.Selection) {
// s2 here is a td element
if s3 := s2.Find("img"); s3 != nil && s3.Length() > 0 {
return // This TD has at least one img child, skip it
}
fmt.Printf(s2.Text())
})
})
Alternatively you could search tr
elements and skip the first td
child of each row by checking if the index passed to the 3rd anonymous function is 0
(first child), something like this:
doc.Find("tbody").Each(func(i int, s *goquery.Selection) {
// s here is a tbody element
s.Find("tr").Each(func(j int, s2 *goquery.Selection) {
// s2 here is a tr element
s2.Find("td").Each(func(k int, s3 *goquery.Selection) {
// s3 here is a td element
if k == 0 {
return // This is the first TD in the row
}
fmt.Printf(s3.Text())
})
})
})