Pretty much all the methods I found on the internet for this:
www.website.com/page?id=43432
to be like this
www.website.com/page/43432
are using htaccess, what I'm asking is, whether there is any other way to achieve this?
With Apache and PHP, as far as I know, there isn't.
Without .htaccess the best you can do is to have page like "page.php" which you will use to form all urls, like www.website.com/page.php/43432. That will add the .php to all your urls.