Currently my links look like
index.php/2011-05-04-06-52-18
How do I turn this off, and use the linkage like in old Joomla 1.5 with /about-us
You have to enable SEF Urls and mod-rewrite in the server configuration.
Here is a Video : http://www.webmaster-tips.net/Joomla-1.6/Video-Tutorials/How-to-enable-SEF-Search-Engine-Friendly-URLs-in-joomla-1.6.html
And a description of the options: http://www.finishjoomla.com/blog/28/joomla-16-seo-recommended-settings/