清理网址错误404

Just wanted to customize Error 404 in my htaccess. The problem is when the user goes to Error 404 page. The URL is not clean compares to others. URL looks like these index.php?page=page-not-found .. How can I make it http://example.com/page-not-found.htm or http://example.com/page-not-found/ ??? By the way, have a look on my codes.

Options All -Indexes
RewriteEngine On

ErrorDocument 404 http://example.com/index.php?page=page-not-found

RewriteRule ^([^/]*)\.htm$ /website/index.php?page=$1 [L,QSA]

RewriteRule ^([^/]*)/lesson/([^/]*)\.htm$ /thesis/index.php?page=$1&lesson=$2

Any idea guys? By the way, thanks..

Remove http:// from your 404 directive to avoid server making a full redirect for 404:

Options All -Indexes
RewriteEngine On

ErrorDocument 404 /index.php?page=page-not-found

RewriteRule ^([^/.]+)\.htm$ /website/index.php?page=$1 [L,QSA]

RewriteRule ^([^/]+)/lesson/([^/.]+)\.htm$ /thesis/index.php?page=$1&lesson=$2 [L,QSA]
Please try this one.First remove your extensions from all files and try this one

    ErrorDocument 404 /404.php

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^p/([0-9]+)$ /p.php?photo=$1 [QSA,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(Joe|Mao|Napoleon|Gandi)$ /profile.php?username=$1 [QSA,L,NC]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^.]+)$ /$1.php [QSA,L]