Welcome to the Q&A Forum

Hi Stu,

How are you setting up these 301 redirects? In the code itself? Or, nginx/apache config? Could you please share the redirection rules you've mentioned for this? It seems an issue with redirection rules itself.

Alrighty! Thanks once again Cyrus

Thanks for sharing your experience and this great article Cyrus, really useful stuff.

What's your observation on post migration traffic trends? Does it makes sense to migrate to https for SEO perspective only?

Hi Dirk,

Good observation, I missed the canonical part somehow. So, google is indexing the canonical URLs here which doesn't have /blog/ in it and that's the problem. Have a look at the indexed page for this particular instance here. Non /blog/ instance is indexed, which will take you to its /blog/ version with wrong canonical URL.

Solution: Either remove the canonical URLs on these pages to point them to the current page itself. And yeah! As rightly mentioned by Dirk, do a proper /blog/ page linking from the blog page and other pages from where you're linking these articles.

Hi Rob Morgan,

You've talked about 2 issues here, please find my views on them:

1. "Since launching my new website design - www.advanced-driving.co.uk I am not convinced Google is seeing all the content on the page. I took a long extract of text and did a search on Google and nothing was found. Also although in the search results for "advanced driving course" I can see the new title tag, the snippet isn't showing"

   => Here, you can check the cached version of any given page by googling "cache:<url-you-want-to-check>" and then clicking on "Text Version".</url-you-want-to-check>

2. As a scroll down I can see the URL changes i.e:
   www.advanced-driving.co.uk
   then: http://www.advanced-driving.co.uk/#da-page_in_widget-3
   then: http://www.advanced-driving.co.uk/#da-page_in_widget-4
   then: http://www.advanced-driving.co.uk/#da-page_in_widget-5

   => This is absolutely fine as far as canonical URL is there, and its there in your case. So, good to go

Hope this helps!

Hi Jason,

I had a word with a friend who handled the similar project for a large organisation a month back and what they observed is, the traffic went down initially for sometime and picked up the flow later on.

Hi Happy SEO,

Well, the robots.txt looks find here. Could you try to fetch any of the blog page/post as google in the search console and share the screenshot here?

Also, to cross check the robots.txt (which looks fine though), you have robots.txt tester in search console where you can put any blog page/post to check if bots can crawl it. Please share a screenshot of that as well.

On a separate note, the sitemap.xml link mentioned in the robots.txt (http://www.keukensduitsland.nl/sitemap.xml) is broken. Fix that as well.

Hi Jamie Booker,

Interesting point. I would like to share my points to handle it from UX and SEO perspective neatly. So, here's my understanding about your product: you have multiple product pages and their variations, say you have a product "My Product" which has 2 variations "Variation A" and "Variation B" (variation can be based on color, type etc. attributes).

Here, you can itemise these product variations and consider the following solution to handle it:

• Variation pages would be available at www.example.com/my-product/variation-a and www.example.com/my-product/variation-b respectively for "Variation A" and "Variation B" of the product "My Product".

• Make sure you have the canonical URL without the variation id/title set for your variation pages. For instance,

  for these variation pages.

• Mark one of these variations as your default product which will be available at www.example.com/my-product. For example, www.example.com/my-product will display the same content that www.example.com/my-product/variation-a will display.

This way you'll have one single URL for a product page for bots which is www.example.com/my-product and all other variation pages exist with variation identifier which will resolve customer's experience point. Also, as we're exposing only single page to bots in this case, we're neatly able to handle duplicate content penalty issue as well.

You can refer the following 2 variation URLs for the same:

• Apple iPhone 6 - Gold: http://www.flipkart.com/apple-iphone-6/p/itme8ra6fzzme5sz?pid=MOBEYHZ2VSVKHAZH
• Apple iPhone 6 - Silver: http://www.flipkart.com/apple-iphone-6/p/itme8ra6fzzme5sz?pid=MOBEYHZ2VRNZZ2J5

Hope this helps!

Hi Happy SEO,

Could you please share the blog URL here? Sounds like an interesting issue and would love to give a try to help you with this

Hi,

IMO, don't remove/de-index/404 the pages which are not your traffic drivers today, they might bring some traffic for you tomorrow. I assume, these pages aren't hurting (going-to-hurt) you i.e they are not duplicate content candidates or something.

Of course you can promote whatever pages you want, like you want to promote those 2k traffic drivers in your case.

P.S Taking down a page and showing 404 is not a good practice in general. If you're permanently closing a page and don't want to 301/302 redirect it, then handle it using 410 instead of 404 here. You can check more about http status codes here: https://moz.com/learn/seo/http-status-codes

Well, these aren't "useless" links. After all, they're linking your categories/sub-categories etc. and should be followed by bots even if a HTML snapshot of any page captures 2-3 follow links (from flyout/menu navigation, breadcrumbs etc.) of another page.

Hope this helps!

No! Don't "nofollow" them. Why do you want to nofollow them now?

You're not spamming here, inter-linking from flyout-navigation/header/footer/on-page-navigation/breadcrumbs are the natural ways people use for internal-linking, that won't hurt you for sure.

Are you trying to say that you're planning to have multiple URLs for a single product page here? For instance, if you have a product which can be reached from multiple navigation paths, so you want to have those multiple URLs for it?

Like if a product is tagged in category "x" and "x" is a sub-category of category "y", then the number of possible URLs for product page "p" would be

• www.example.com/y/p
• www.example.com/y/x/p

So, here these 2 URLs are candidates of duplicate content penalty and hence, you want to noindex them? Is this what you're trying to explain?

Hi,

Well, following "pyramid" scheme and noindexing pages are two different things altogether. Let's not mix them, its creating confusion actually. So, tell me why do you want to noindex your pages?

Using pyramid scheme and optimizing your site's architecture the best possible way can be done independently.

Hi,

Could you please help me understand your concern here? What do you mean by "noindex, follow all links except 1 from breadcrumbs"?

May be you need to elaborate your concern or share some screenshots to help me understand it.

P.S noindexing a subset of pages is not harmful for sure.

Hi  Panagiotis Triantopoulos,

There isn't any hard and fast rule for it. So, go ahead with a different anchor text without a doubt. As far as it's making sense w.r.t the linked article (descriptive enough to talk about the article theme), its good to go.

Hope that answers!

No problem. Do share screenshots of product pages and the URLs (once available) here. Will be able to help you out with this. Fixing is using canonical or meta robots is not a time taking solution to implement in general and hence, can be fixed at the last moment (before going live) as well. So, this can be parked for now.

Hi David,

Thanks for sharing a couple of instances to help me understand the point here. Well, I don't think there is any need of blocking these pages from indexing. You're confused about it just because you don't have much content to show on these pages and the templates is similar and hence, google might consider them as duplicate pages, right?

To resolve this issue and also, to make these pages stronger from organic visibility perspective, you would need to add on-page content and other "cool" features to make them powerful anyways. But, blocking them for bots won't be a good solution I believe.

Btw, if sharing the URLs of the pages is not possible as its in development phase, could you please share the screenshots of the pages here? Would be able to comment on how this should be handled once after having a look at it.

Hi David,

If header and other details are different on these pages, why would you like to set canonical or somehow block these pages from indexing? That should be a candidate for duplicate content penalty I believe.

Could you please share some sample URLs to help me understand the issue you're talking about? I'll try my best to guide to handling this neatly from SEO perspective.

Hi,

I'm working on a website with more than 10 million URLs and hence, couldn't rely on these tools for this. Wrote a script to generate the XML sitemap, it takes a text file with all the URLs and generates the required XML sitemap considering 50,000 URLs per sitemap limit etc. things. Let me know if you have list of URLs with you and interested in generating it at your own (you can ask your development team to integrate this script for auto-generation etc. as well), I'll share the script with you. Cheers!