I have this program to web scrape one of the website and works fine for the current year data. But for the previous year data, they have this year drop down, which allows you to select the year for which data needs to be seen but the problem is that it doesn't change the year. I noticed that they have this onclick="javascript:return post_chg_year function at the start of the code, which i believe doing the trick.
Now question is, how can I invoke this function in my code, to reflect the same year change.
Website Code:
<select name="keystat_yrc" id="keystat_yrc">
<option value=''>Select</option><option value='2018' >2018</option><option value='2017' >2017</option><option value='2016' >2016</option> …</select>
<img src="goBut3.gif" border="0" onclick="javascript:return post_chg_year('keystats');" align="absmiddle">
<a href="#" class="btnNext" rel="1"></a></h2>
</th>
</tr>
<tr>
<th>Year<br>Ended</th>
<th><a href='/stocks/data-bank/standalone/auto-2-3-wheelers/1/keystats/equity' target='data_bank'>Equity<br><font size='1'>(Rs Cr)</font></a> <span class='ar'></span></th>
<th><a href='/stocks/data-bank/standalone/auto-2-3-wheelers/1/keystats/bv' target='data_bank'>BV</a> <span class='ar'></span></th>
<th>Face<br>Value</th>
<th><a href='/stocks/data-bank/standalone/auto-2-3-wheelers/1/keystats/dividend' target='data_bank'>Dividend</a> <span class='ar'></span></th>
<th><a href='/stocks/data-bank/standalone/auto-2-3-wheelers/1/keystats/prom-holding' target='data_bank'>Promoters<br>Holding</a> <span class='ar'></span></th>
<th><a href='/stocks/data-bank/standalone/auto-2-3-wheelers/1/keystats/ronw' target='data_bank'>RONW</a> <span class='ar'></span></th>
<th><a href='/stocks/data-bank/standalone/auto-2-3-wheelers/1/keystats/roce' target='data_bank'>ROCE</a> <span class='ar'></span></th>
<th><a href='/stocks/data-bank/standalone/auto-2-3-wheelers/1/keystats/dte' target='data_bank'>Debt to Equity</a> <span class='ar'></span></th>
<th><a href='/stocks/data-bank/standalone/auto-2-3-wheelers/1/keystats/mktcap-sto' target='data_bank'>Market Cap/Sales</a> <span class='ar'></span></th>
<th> </th>
My R Code:
SectorList <- read.csv("/SectorList.csv",header = TRUE, sep = ",")
SectorList <- as.data.frame(SectorList)
n <- 112
for (n in 1:n) {
Sector_Name <- as.character(SectorList[n,1])
SectorID <- as.character(SectorList[n,2])
aUrl <- paste0("https://www.moneycontrol.com/stocks/data-bank/standalone/",SectorID,"/1/keystats/equity")
bUrl <- paste0("https://www.moneycontrol.com/stocks/data-bank/standalone/",SectorID,"/1/annual/equity")
cUrl <- paste0("https://www.moneycontrol.com/stocks/data-bank/standalone/",SectorID,"/1/quarterly/equity”)
dUrl <- paste0("https://www.moneycontrol.com/stocks/data-bank/standalone/",SectorID,"/1/mkt-data/equity")
#Key Stats Gathering
acon <- getURL(aUrl, ssl.verifyhost=FALSE, ssl.verifypeer = FALSE)
awebpage <- read_html(acon)
a <- awebpage %>%
html_nodes("table.cntTbls_lrg") %>%
html_nodes("td") %>%
html_text() %>%
gsub("^\\s+|\\s+$","",.)
if(length(a)==0){
next
}
. . . .
Appreciate your help on this.