So I'm using php loadHTMLFile() to find links on my webpage. but it always collects the links as if I am logged out, which misses a lot of log in only links I need to find. the pages already have the database session info able to be called. Is there a way to send the needed session info to the pages being crawled to find the dynamic links? I can make a false log in data if needed, I just need to be able to find the logged in only links.
Below is the code I use to locate the links:
set_error_handler (function($errno, $errstr, $errfile, $errline) {}); //Swallow unadvoidable errors caused by loadHTMLFile
$valid = $this->doc->loadHTMLFile($path); //Check to insure url is valid link.
restore_error_handler(); //Restore normal error handling
if($valid !== false)
{
$xpath = new DOMXpath($this->doc); //Create instance of DOMXpath() class. --php core--
$elements = $xpath->query("//a[not(@rel='nofollow')]/@href"); //Use $xpath to pull links from page. nofollow links are ignored.
$this_page= array(); //Insure $this_page is set, even if 0 links are found causing a false null on is_null
if (!is_null($elements)) foreach ($elements as $element) $this_page[]= $element->nodeValue; //Create a array of located links from DOMXpath object.
Anyone have any ideas? I have complete control of the site and database and this is a admin only script so user restrictions can be minimal. Thank you!