Specific question about pagination prompted by Adam Audette's Presentation at RKG Summit

[danatanseo]

This question is prompted by something Adam Audette said in this excellent presentation:

http://www.rimmkaufman.com/blog/top-5-seo-conundrums/08062012/

First, I will lay out the issues:

1. All of our paginated pages have the same URL. To view this in action, go here: http://www.ccisolutions.com/StoreFront/category/audio-technica , scroll down to the bottom of the page and click "Next" - look at the URL. The URL is: http://www.ccisolutions.com/StoreFront/IAFDispatcher, and for every page after it, the same URL.

2. All of the paginated pages with non-unique URLs have canonical tags referencing the first page of the paginated series.

3.  http://www.ccisolutions.com/StoreFront/IAFDispatcher has been instructed to be neither crawled nor indexed by Google.

Now, on to what Adam said in his presentation: At about minute 24 Adam begins talking about pagination. At about 27:48 in the video, he is discussing the first of three ways to properly deal with pagination issues. He says [I am somewhat paraphrasing]: "Pages 2-N should have self-referencing canonical tags - Pages 2-N should all have their own unique URLs, titles and meta descriptions...The key is, with this is you want deeper pages to get crawled and all the products on there to get crawled too. The problem that we see a lot is, say you have ten pages, each one using rel canonical pointing back to page 1, and when that happens, the products or items on those deep pages don't get get crawled...because the rel canonical tag is sort of like a 301 and basically says 'Okay, this page is actually that page.' All the items and products on this deeper page don't get the love."

Before I get to my question, I'll just throw out there that we are planning to fix the pagination issue by opting for the "View All" method, which Adam suggests as the second of three options in this video, so that fix is coming.

My question is this: It seems based on what Adam said (and our current abysmal state for pagination) that the products on our paginated pages aren't being crawled or indexed. However, our products are all indexed in Google. Is this because we are submitting a sitemap? Even so, are we missing out on internal linking (authority flow) and Google love because Googlebot is finding way more products in our sitemap that what it is seeing on the site? (or missing out in other ways?)

We experience a lot of volatility in our rankings where we rank extremely well for a set of products for a long time, and then disappear. Then something else will rank well for a while, and disappear. I am wondering if this issue is a major contributing factor.

Oh, and did I mention that our sort feature sorts the products and imposes that new order for all subsequent visitors? it works like this: If I go to that same Audio-Technica page, and sort the 125+ resulting products by price, they will sort by price...but not just for me, for anyone who subsequently visits that page...until someone else re-sorts it some other way. So if we merchandise the order to be XYZ, and a visitor comes and sorts it ZYX and then googlebot crawls, google would potentially see entirely different products on the first page of the series than the default order marketing intended to be presented there....sigh.

Additional thoughts, comments, sympathy cards and flowers most welcome. Thanks all!

[JaneCopland]

Wow, I had never heard of this method of sorting before! Is this something you plan on keeping?

I would not be surprised if this is somewhat related to why Google sometimes finds and ranks some products, only to drop them for others. If the CMS serves it a different canonical version of the website every time it visits, it will index different products at different times.

The subsequent paginated pages are canonicalised back to the first page, but do you have unique product pages still returning 200 OK? I take it you do, due to the wording of your question but just to clarify: To use Amazon as an example, if a page like this is paginated, is a product page it links to still available even if it is listed on a deeper paginated page? If so, Google won't necessarily drop the product page if it doesn't see it linked to by the paginated listing pages, but it might do if it never sees it again due to pagination and canonicalisation. So you might end up with a situation where Googlebot found the Glastonbury coffee mug in January, keeps ranking it well for a few months, has not seen it again by April and drops it. But a white coffee mug set was found a few times in March, so it ranks that instead.

Have a look at this pagination post from one of my former colleagues as well - it has some comprehensive solutions to ecommerce pagination problems.

[danatanseo]

Thank you Jane.

To answer your first question, no, we don't plan on continuing using the current method, however, I am concerned that whatever we decide to change is implemented properly.

To answer the second question, yes, all of our unique product pages return 200 OK status codes. I think the scenario your described is a very plausible scenario and it makes perfect sense to me. Especially you describe something else that I have seen happen, which I didn't even mention in my question. This is, the swapping out of ranking pages. I have seen this many times when one page might rank for a particular term for a while, then drop, and another page will take it's place and may do a lot better or a lot worse, and then in a few months time they will flip flop again.

One thing you didn't speak to was the fact that we have "no crawl, noindex" set up on this URL that is home to all of our paginated series pages. I am wondering what kind of havoc that could be wreaking on our internal linking and authority flow throughout the site? I am thinking it can't possibly be a good thing, no matter how you slice it.

Thank you very much for the link to the Pagination for SEO post. It contained a most excellent flow chart for pagination that I think every SEO should blow up, print out and post on a wall somewhere where developers and IT can see it. I am sharing a small version here:

LbNGood

[JaneCopland]

Hi Dana,

Just to be clear, what I'm seeing is that if I visit a page like http://www.ccisolutions.com/StoreFront/category/led-stage-lights with 107 products, and I click "next", I do not receive the same URL with different products, but rather the URL changes to http://www.ccisolutions.com/StoreFront/IAFDispatcher with the next set of results. I think I'm just being blind, but how did you block that URL from crawling and indexing? I can't see a meta tag or a line in the site's robots.txt file.

This set-up definitely needs a modern pagination solution - glad Ryan's post at Ayima helped! This current set-up is definitely very detrimental to passing authority throughout the site to all products effectively.

[danatanseo]

Thanks so much Jane. I believe that URL is blocked from being crawled by our .htaccess file, although that's something I need to verify with IT. I just know from past discussions that it is blocked from crawling and indexing and it isn't in Google's index.

Would you mind describing, perhaps with a diagram, how this setup is a problem for passing authority internally? I am thinking it breaks the flow kind of like this:

Page 1 of Series -----> passing authority to page 2 --------> authority stops dead in its tracks due to non-unique URL

Instead of looking like this:

Page 1 of Series -----> passing authority to page 2 --------> page 2 passes authority back to home page, page 1 and page 3 of the Series....and so on

Would that be a somewhat accurate description? Thanks so much for responding. It is greatly appreciated!

[JaneCopland]

Hi Dana,

The problem when it comes to passing authority internally is that properly paginated and crawled listing pages can be one of the primary routes via which Google finds and assigns authority to internal pages. Unless those products are linked to elsewhere, they're not going to be found if they cannot be found on a URL like http://www.ccisolutions.com/StoreFront/category/audio-technica?page=2, ?page=3 etc.

The lack of a unique URL with content changed dynamically also means that there never could be a good flow of authority through the site as Google does not have new pages to crawl and new outbound links to index / follow on those pages.

Your diagram is correct - the second option (Page 1 ---authority---> page 2 ----authority---> page 3... ) is what you're looking for with pagination.