Hi there. Sorry for the slow follow-up on this - there was an issue that meant I didn't get the email alert when it was assigned to me.

There is increasing evidence that culling old / poor performing content from your site can have a positive effect, though I wouldn't be particularly confident that this would transfer across sub-domains to benefit the main site.

In general, I suspect that most effort expended here will be better-placed elsewhere, and so I would angle towards the least effort option.

I think that the "rightest" long-term answer though would be to move the best content to the main domain (with accompanying 301 redirects) and remove the remainder with 410 status codes. This should enable you to focus on the most valuable content and get the most benefit from the stuff that is valuable, while avoiding having to continue expending effort on the stuff that is no longer useful. The harder this is, though, the less I'd be inclined to do it - and would be more likely to consider just deindexing the lowest quality stuff and getting whatever benefit remains from the better content for as long as it is a net positive, with an eye to eventually removing it all.

Hope that helps - I don't think it's a super clear-cut situation unfortunately.

Hi Phil,

Sorry for the slow response to the question - we had a hiccup with the email alerting system and so I wasn't notified as quickly as I should have been.

A couple of resources that might help give you context on the general question about sub-domains vs. sub-folders:

1. Video explainer
2. Overview of URLs generally
3. Recent follow-up showing that moving from sub-domain to sub-folder can result in positive movements

Having said all of that, given that you reference the fact that technical limitations have caused this situation, it's worth noting that creating the content on a separate sub-domain is clearly better than not creating it at all if those are the pragmatic real-world options on the table.

Good luck!

This doesn't ring true to me because:

1. There are all kinds of ways you can find links to your site (referring traffic in GA, link analysis sources like moz's own tools, ahrefs, majestic etc, and tools like Google Alerts)
2. You've already got the link in these cases, so you would have no incentive to "bother" the webmasters
3. In general, my experience has been that most providers want to tell you about as many placements as they can!

Add to this some of the phrasing (e.g. "webmasters we have" -- emphasis mine) suggests that there may be payments taking place or other schemes afoot that place these well outside the Google guidelines.

Per the conversation in the other comments - I would recommend that you get at least a statement of work in place to govern what you are expecting for the money you are paying even if you are not committing to a long-term contract. This could include agreements on any of the above (what information they will provide you with, what you will do with that information etc). At the very least, you should want to understand the techniques in use in order to make your own informed decisions about the risks and rewards.

Hope that helps.

After a bit more digging (and a venture down a character encoding rabbit hole) I haven't come up with any smoking guns (mixed metaphor alert).

I don't suppose there's any chance those pages could have had a noarchive meta tag on them (or nosnippet) at any point in the past is there? It's possible that would remove the link but not remove the cached copy itself.

I'm honestly not sure there is going to be a ton of ROI to going further down the rabbit hole to be honest. I think it's most likely to either remain unsolved or be something you can't do anything about (and we don't even know if it's really causing actual issues).

If you want to continue debugging, I would run through these steps:

1. Take a page like the one you mentioned and duplicate it on a different URL outside of that folder structure and see if that gets a cache link
2. If it does, try 301 redirecting the page without the cache link to the one with the cache link and see if it disappears
3. Try the above but with the redirect the other way around

This will help narrow down what's going on - but doesn't guarantee that it'll be fixable, nor that there is actually any value to getting the cache link back.

If you have any more evidence on the other issues you referred to (featured snippets etc) let me know and I can look at that separately.

Let me know if you dig anything out. Good luck.

Hi Fred. Just having a dig into this. A couple of things I have noticed so far that won't be causing this issue but that you might want to know about:

1. The page you mentioned is in the cache - you can see the cached version by going to cache:https://www.jet2holidays.com/destinations/balearics/majorca -- the cached html appears to match the live html so there doesn't seem to be a crawling or indexing problem
2. Your sitemap.xml link in your robots.txt goes to the http version which 301 redirects to https
3. There are warnings firing in the console in Chrome about distrusting your SSL certificate provider in an upcoming release: https://security.googleblog.com/2017/09/chromes-plan-to-distrust-symantec.html

Hi there. There are a number of ways in which this is a bad idea - as others have pointed out. But in particular:

1. What you are doing may manipulate the "bounce rate" you see in analytics, but won't affect the % of users who return immediately to the search results after landing on your page - which we know is one of the ways Google evaluates the quality of their own search results
2. Going via an interstitial page is on its own not great UX, but having a delay on that page just makes it more likely that a user will give up and leave immediately - and also have a greater chance of remembering your site as being a poor UX and be less likely to return

At an overarching level, there's nothing wrong with having external links off a page. This is normal and expected on any site.

I would say you should put all your efforts that are currently going into manipulating this behaviour and these metrics into improving the user experience of your site to make it better for the people who do find it.

I hope that helps.

Yes - of course. Happy to take a look.

Hi Lori,

On closer inspection, I think that only the rewriterule should have the [L] flag and that placing the specific Redirect at the top of the file should work fine without chained redirects as the other commenters suggested. I tested that here: http://htaccess.mwl.be/ and it appears to work fine using the following .htaccess - can you confirm with your developer that this is what they were trying?:

Redirect 301 /old.php https://www.clientdomain.com/new.php

RewriteEngine on

if non-SSL and one of these, redirect to SSL

RewriteCond %{HTTPS} !on
RewriteRule ^(.*)$ https://www.clientdomain.com/$1 [R=301,L]

Hi Lori. The solution the other commenters have suggested is definitely the way to do this - so it sounds like it needs more debugging. I suspect it's to do with the [L] option being needed on the specific redirect once it's moved above the general http-->https redirect. This stops other redirects below it firing, if I remember correctly.

If that doesn't work, do you want to share back here the specific different htaccess files the developer has tried?

Yeah - there is various speculation about how signals or authority traverse folder structures (see for example this whiteboard Friday ) but I haven't seen anything suggesting it's permanent - all of this may be an argument for adding /famous-dogs/ at some point, but I wouldn't personally stress about it not being there at launch.

Yeah. I'd just leave it as a 404 in that case

PS - if you're worried about the crawling, you could always block it in robots.txt if you really wanted (but unless it's a huge site I wouldn't bother). Note - if you do go this route, do it carefully so as not to block all contents of the folder at the same time!

The short answer is that there should be no harm going with your proposed approach.

Longer version: I believe there are cases where Google has tried to crawl a directory like "/famous-dogs/" in your example purely because it appears as a sub-folder in the paths of other pages even though there are not any direct links to it. But even if it does crawl it, if you don't have or intend to have a page there, a 404 is a perfectly valid response.

In general, while there could be a case that it's worth creating a "/famous-dogs/" page if there is search demand you can fulfil, until or unless you do, there is no harm in it returning a 404 response.

In theory, there should be no difference - the canonical header should mean that Google treats the inclusion of /images/123456 as exactly the same as including /images/golden-retriever.

It is slightly messier so I think that if it was easy, I'd go down the route of only ever using the /golden-retriever version - but if that's difficult, this is theoretically the same so should be fine.

Hi James. I've responded with what I believe is a correct answer to MarathonRunner's question. There are a few inaccuracies in your responses to this thread - as pointed out by others below - please can you target your future responses to areas where you are confident that you are correct and helpful? Many thanks.

@MarathonRunner - you are correct in your inline responses - it's totally valid to serve an image (or other filetype) without an extension, with its type identified by the Content-Type. Sorry that you've had a less-than-helpful experience here so far.

To answer your original questions:

1. From an SEO perspective, there is no need that I know of for your images to have a file extension - the content type should be fine
2. However - I have no reason to think that a filename in the Content-Disposition header will be recognised as a ranking signal - what you are describing is a rare use-case and I haven't seen any evidence that it would be recognised by the search engines as being the "real" filename

If you can't always refer to the image by its keyword-rich filename, then could you:

• Serve it as you propose (though without the Content-Disposition filename)
• Serve a rel="canonical" link to a keyword-rich filename (https://example.com/images/golden-retriever in your example)
• Also serve the image on that URL

This only helps if you are able to serve the image on the /images/golden-retriever path, but need to have it available at /images/123456 for inclusion in your own HTML templates.

I hope that helps.

Hi Jacob,

Experts have been writing about the declining importance of exact match domains for many years (those two articles are both from 2012).

In general, if you already had a setup like the one you are proposing, there's a good chance we would be recommending consolidating the domains.

You might also be interested in this whiteboard Friday.

Good luck - hope something there helps.

Google is incredibly good at entity detection (i.e. figuring out when someone is searching for a "thing"). When there is a canonical right answer for that thing (e.g. the company website in the case of a company), it will often rank very well despite not necessarily having any of the "traditional" ranking factors in place.

Typically, when the search is unambiguously for the thing in question, the company will rank #1 even if there are much stronger websites and pages about that company. You will rarely find that a company is outranked on their own company name when it's sufficiently distinguished from other entities even if it's a really small company and there are (for example) major media stories about it.

There are a variety of other factors - like query sequences (e.g. a user searching [seo] followed by [seo trucks] in your example) that Google can use to associate more specific sites with more general queries as well.

Most importantly - I'm not sure there's a great deal you can do about it, not any general lessons you can apply to your own attempts in this market / keyword space - so I wouldn't spend too long worrying about it!

You appear to have the MX sub-domain also set up as an A record.

If you have a mac / linux you can run the command: host aspmx3.googlemail.com.sullivansolarpower.com

You get the result aspmx3.googlemail.com.sullivansolarpower.com has address 72.10.48.198

Where you should get the result "not found".

I think you want to delete the A record (though check the documentation of your email provider first). You should only need them set up as MX records and shouldn't need the A record.

You've done the right thing by setting up the redirect - which should mean that the pages drop out of the index and those links disappear. (Note that there is also an https error on the aspmx3 sub-domain - but given that you don't actually want it, I don't suppose that matters that much).

Hope that helps.

Hi hoosteeno,

It's obviously tricky to diagnose specific technical issues in the abstract, without looking at the specific site, but here are some resources and ideas that might help:

1. Rand recently talked about ranking fluctuations on whiteboard Friday - that'll be worth checking out
2. Major site redesigns can cause quite a bit of fluctuation, but you don't want to wait around to find out weeks or months later that there is a technical issue, so I would suggest working through this checklist
3. It's worth checking whether you have accidentally caused a load of new on-site duplicate content with the recent work - that's one other thing that could cause weird in / out fluctuations

Hope something there helps. Good luck!