Questions
-
RegEx help needed for robots.txt potential conflict
Hey James It looks to me like you are just disallowing access to any URLs that have more than the initial p= variable. So, you are reducing the impact of potential duplication through searches and the like. Good ?p=1 Bad ?p=1&q=search string I am no magento expert but this seems to be a simple attempt to reduce the myriad duplication that can happen with search pages and the like inside a complex CMS like Magento. The SEOMoz crawler tool should give you some good insight and to be sure, try removing the 'Disallow: /?p=&' and see if you get a buckletload of duplicate content warnings. Ultimately, the thing to remember here is that the & is part of the URL and not part of the regex. Hope that helps! Marcus
Technical SEO Issues | | Marcus_Miller0