I love this question, Adriaan. It's one that a lot of people have asked over the years and that a lot of people have had to deal with over time especially with ecommerce sites like those you work on.
As you well know, there are multiple ways to handle duplicate content:
- The way you are proposing, which is moving to a static URL structure that always keeps the same order
- A web of canonicals like you seem to have set up (and it sounds like you have it set up correctly)
- The whack-a-mole approach of periodically looking for duplicate content and implementing redirects, which can lead to further issues with internal redirects. This is not a good scalable option.
SEO is all about processes. If you have a canonical process that is working for you and has been scalable (eg you are not manually specifying the URL for each new category created, which is probably done when the merchandising team or feeds update the site), that works to a certain extent.
However, this is like treating a bunch of cuts on your hands with bandaids but not dealing with the fact that a) you only have so much space on your hands and can only apply so many bandaids, and b) that you're still getting cuts on your hands.
I prefer to deal with the root of the issue, which in your case is that you can have multiple URLs targeting the same terms based on the user's (or Googlebot's!) crawl path on your site. I am assuming that you are only putting the canonicals in your XML and HTML sitemaps, by the way?
If I were you, this is how I would tackle your problem:
-
Make sure you are only putting in the canonical URLs to your XML sitemaps. Start here.
-
Do a full crawl of your site and pull all the URLs that are canonicaling elsewhere. Then get your log files and see how much time the search engines are spending on these canonical'd URLs.
-
Also check to see that Google is indeed respecting all of your canonicals! At this scale of canonicals, I'd expect that they are semi-often not respecting them and you are still dealing with duplicate content issues. But again, that's just a hunch I have.
-
Make a decision from there, off of discussions with your engineers/designers/etc about how much work is involved, about if you think it's worthwhile to make the change.
I am **always **a fan of eliminating pages that are canonical'd and not serving a purpose (example: a PPC landing page might be canonical'd and noindexed, and you don't want to remove that page). My suspicion in your case, as well, is that having /brand/mens won't convert any differently from /mens/brand.
At the end of the day, you need to decide how you want your site organized and if your customers (the people buying things on the site) prefer to shop by brand or by gender/sport/whatever. This will help you decide what way to architect your URLs and your site's flow.
Hope that helps!
John