I have a directory with 3 kind of files
1. thisdir/some/phpfile.php
2. thisdir/some/after-url-rewrite/1 (all numbers)
3. thisdir/something.htm (block any file end with .htm in thisdir/
I just want to block the 3rd type of files, since there are not existing any more, but search engine spider keep asking for them
Thanks for your help.
You could write in your robots.txt
disallow: thisdir/*.htm$
https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt
If that file doesn't exist then you can just redirect all requests of that 3rd file to a valid link with 301 code.
RewriteEngine On
RewriteRule ^thisdir/something\.htm$ /thisdir/some/phpfile.php? [L,NC,R=301]