Sean,
Here are some resources that I have for you
http://www.webforgers.net/mod-rewrite/mod-rewrite-syntax.php
http://roshanbh.com.np/2008/03/url-rewriting-examples-htaccess.html
Hope they help you in understanding how to go about .htaccess.
As far as I understand, you are facing the issue since you pointed yourinterlinking URLs to your .index.html page rather than your absolute URLs.