Questions
-
Robots.txt - What is the correct syntax?
I added your suggestion to robots.txt and requested a crawl again. I only have 3 pages with dublicate page content now So your suggestion seemes to have worked. Thanks for your reply.. it worked!
Technical SEO Issues | | teleman0 -
Best use of robots.txt for "garbage" links from Joomla!
Hi Henrik, It can take up to a week for SEOmoz crawlers to process your site, which may be an issue if you recently added the tag. Did you remember to include all user agents in your first line? User-agent: * Be sure to test your robots.txt file in Google Webmaster Tools to ensure everything is correct. Couple of other things you can do: 1. Add a rel="nofollow" on your send to friend links. 2. Add a meta robots "noindex" to the head of the popup html. 3. And/or add a canonical tag to the popup. Since I don't have a working example, I don't know what to canonical it too (whatever content it is duplicating) but this is also an option.
Technical SEO Issues | | Cyrus-Shepard0