3. Is there a reason Why can't it just look like this:
Yes, It would generate a lot of duplicates issues, for example your robots.txt you have the follow line:
Disallow: /catalog/category/view/ -> That's the "real" category URL, you can access any category on magento by /catalog/category/view/id or by the "pretty" URL.
Because you disallow the "real: URL only the pretty URL will be viable for search engines.
This same rule apply for many other parts of the robots.txt.
