Yeah - there is various speculation about how signals or authority traverse folder structures (see for example this whiteboard Friday ) but I haven't seen anything suggesting it's permanent - all of this may be an argument for adding /famous-dogs/ at some point, but I wouldn't personally stress about it not being there at launch.

Hi Lori,

On closer inspection, I think that only the rewriterule should have the [L] flag and that placing the specific Redirect at the top of the file should work fine without chained redirects as the other commenters suggested. I tested that here: http://htaccess.mwl.be/ and it appears to work fine using the following .htaccess - can you confirm with your developer that this is what they were trying?:

Redirect 301 /old.php https://www.clientdomain.com/new.php

RewriteEngine on

if non-SSL and one of these, redirect to SSL

RewriteCond %{HTTPS} !on
RewriteRule ^(.*)$ https://www.clientdomain.com/$1 [R=301,L]

Yes - of course. Happy to take a look.

Hi Phil,

Sorry for the slow response to the question - we had a hiccup with the email alerting system and so I wasn't notified as quickly as I should have been.

A couple of resources that might help give you context on the general question about sub-domains vs. sub-folders:

1. Video explainer
2. Overview of URLs generally
3. Recent follow-up showing that moving from sub-domain to sub-folder can result in positive movements

Having said all of that, given that you reference the fact that technical limitations have caused this situation, it's worth noting that creating the content on a separate sub-domain is clearly better than not creating it at all if those are the pragmatic real-world options on the table.

Good luck!

Hmmm. This is tricky. Some ideas - hope something here is helpful:

1. Have you tried "inspect URL" in search console? That has information about canonical selections these days and may be helpful
2. Are the canonical URLs (and no parameter URLs) included in the XML sitemap? Might be worth trying cleaning that up if there is any confusion
3. Cookies could work - but it sounds to me as though that would go against your client preferences as the non-cookie version would have to remove / work without parameters I think - which you indicated they weren't prepared to do
4. Failing all of that, what about picking one category to be the primary category for each product and canonicalising to that (which will have internal links) instead of to the version with no parameters? Could that work? Might nudge towards the canonical being respected

I think you could legitimately take either approach to be honest. There isn't a perfect solution that avoids all possible problems so I guess it's a combination of picking which risk you are more worried about (pages getting indexed when you don't want them to, or crawl budget -- probably depends on the size of your site) and possibly considering difficulty of implementation etc.

In light of the fact that we heard about noindex,follow becoming equivalent to noindex,nofollow eventually, that does dampen the benefits of that approach, but doesn't entirely negate it.

I'm not totally sold on the phrasing in the yoast article - I wouldn't call it google "ignoring" robots.txt - it just serves a different purpose. Google is respecting the "do not crawl" directive, but that has never guaranteed that they wouldn't index a page if it got external links.

I personally might lean towards the robots.txt solution on larger sites if crawl budget were the primary concern - just because it wouldn't be the end of the world if (some of) these pages got indexed if they had external links. The only reason we were trying to keep them out was for google's benefit, so if they want to index despite the robots block, it wouldn't keep me awake at night.

Whatever route you go down, good luck!

Hi Frankie,

Sorry for the slow reply to this one. I hope it's still relevant to offer some thoughts.

First, at the top level, I would say that the stated reasons don't necessarily mean that you should not have the kinds of pages you describe. My first preference would be to modify the functionality so that the filters you describe users actually using are those sub-category pages. Even if this meant changing URLs (and hence 301 redirecting the pages you currently have), it is possible to have filter / facet pages be indexable and have unique URLs and meta information.

If that's not possible for whatever reason, I would separate my efforts into the micro and the macro:

1. Micro: apply a 80:20 or 90:10 rule to the pages that you are losing - find the small number of most important and highest traffic / conversion pages and find a way to keep versions of those pages (again - even if you have to 301 redirect them, you could create them as static content pages targeting those keywords or something if you had to)
2. Macro: where you simply have no choice but to lose these pages, I think your best bet will be to redirect them to the absolutely best (/ next best!) page on the site for those queries - these might be other (sub-)category pages or they might be individual products or content pages, but at least for the highest traffic end, it'd be worth specific research effort to identify the best redirect targets

One final thought: it's not always the case that the URL has to represent every level in the hierarchy. I don't know your underlying technology, but it might be possible to recreate some of these sub-categories as top-level categories if products are allowed by your CMS to be in more than one category at once. I wrote this article about the difference between URL structures and site architecture that might give more clarity on what I mean here.

Very hard to prove these things before they're done - good luck with getting buy-in for what you need to do and in undoing the worst of the damage.