Welcome to the Q&A Forum

I have actually had hands on experience with this.  You will likely read almost everywhere that the shorter, the better but it wasn't the case for one of our clients.  We had a client who was ranking pretty high for a very competitive word.  We began going through an on-site SEO audit and clean up issues such a tags, meta, alt, etc.  We also decided to shorten all of the URL's to http://example.com/_page and set up 301's to point to the new url.  It immediately lost all rankings even after a very lengthy wait of a de/re index.  Their rankings came back  quickly after we moved it back to long tail._ I doubt this will be the case for all scenarios, but it was better to keep the category in this case in the URL.

Unfortunately, that would be specific to your own platform and server-side code. When you look at the SEOmoz source code, you're either going to see a nofollow or you're not. The code that drives that is on our servers and is unique to our build (PHP/Cake, I think). You'd have to dig into the source code generating the Robots.txt file. I don't think you can have a fully dynamic Robots.txt (it has to have a .txt extension), so there must be a piece of code that generates a new Robots.txt file, probably on a timer. It could be called something similar, like Robots.php, Robots.aspx, etc. Just a guess. FYI, dynamic Robots.txt could be a little dicey - it might be better to do this with a META NOINDEX in the header of the user profile pages. That would also avoid the timer approach. The pages would dynamically NOINDEX themselves as they're created.

Keri, You never cease to be able to recall the good stuff. Funny, its been a busy week so am just getting to some of this. Great post here, thanks.

I am having a similar issue. I am getting hit with 404 errors for pages that do not exist anymore of have been fixed. How do I get these to stop showing up?