I have a custom designed script for my website, and /admin/ shows up in google search. How can i make /admin/ not visible and knowable to others. At least on google or searchable?
What I do is return a 404 not found unless the user is already logged in. Put the login url above /admin.
That combined with robots.txt should take care of it.
However, using robots.txt will alert those with ill-intent (hackers, etc) that you have a url to be attacked. I never put admin dashboards in robots.txt - I simply deny they exist.
<?php
$loggedIn = false; // replace with how you track logged in status
if ( ! $loggedIn )
{
header( 'HTTP/1.0 404 Not Found' );
echo '<h1>404 - Not Found</h1>';
exit();
}
As already stated by andrewsi, you create a robots.txt file and put the following:
User-agent: *
Disallow: /admin
Place that in the root of your site.
create a file called robots.txt and place it in the root of your website. Add the following to it:
User-agent: *
Disallow: /admin/
this is all explained in more detail on robotstxt.org
Use a robots.txt file with the following:
User-agent: *
Disallow: /admin/
Not all crawlers obey this file, however.
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=156449