Drop in Indexed Page + Organic Traffic

[znotes]

Hey Moz Community,

I've been seeing a steady decrease in search console of pages being indexed by Google for our eCommerce site. This is corresponding to lower impressions and traffic in general this year. We started with around a million pages being indexed in Nov of 2015 down to 18,000 pages this Nov. I realized that since we don't have around 3,000 or so products year round this is mostly likely a good thing.

I've checked to make sure our main landing pages are being indexed which they are and our sitemap was updated several times this year, although we're in the process of updating it again to resubmit. I also checked our robots.txt and there's nothing out of the ordinary. In the last month we've recently gotten rid of some duplicate content issues caused by pagination by using canonical tags but that's all we've done to reduce the number of pages crawled. We have seen some soft 404's and some server errors coming up in our crawl error report that we've either fixed or are trying to fix.

Not really sure where to start looking to find a solution to the problem or if it's even a huge issue, but the drop in traffic is also not great. The drop in traffic corresponded to lose in rankings as well so there could be correlation or none.

Any ideas here?

[LauraSultan]

I suspect the canonical tag implementation is the culprit for the loss in indexed pages, but that's just a strong suspicion. When you implement a canonical tag, Google will fold together all of the pages pointing to the same canonical URL so that they are treated as one URL for the purposes of indexation. This will naturally cause a reduction in the overall number of pages indexed. You won't see 404 errors as a result of this either.

One important point to remember is that a decrease in indexed pages is not necessarily a bad thing that needs to be fixed. In fact, if you were having problems with duplicate content or crawl issues, canonicalizing those URLs should be beneficial if properly implemented.

As for the loss in traffic, it could be related or not. It may just be an adjustment period from the canonicalization, or it could a separate issue. Have you seen a corresponding loss in conversions?

[znotes]

Hi Laura,

Thanks for your response, I know that adding canonical tags can lower our indexed pages count. The only thing I'm worried about is the fact that the drop in indexed pages have been steady since Nov of last year and we just implemented the tags last month.

We have been seeing lower conversions with less traffic some of which has to do with us not ranking as well for a few key-terms as we did last year.

[LauraSultan]

Oops, I missed that part. Have you checked Google Search Console to see if someone set any URL parameters?

The first thing I would do is determine how many pages actually should be indexed to see if there's a large discrepancy between that and the number Google shows. A crawler like Screaming Frog can help with this. If you export the crawl to Excel, you can easily remove duplicates in the canonical URL column and filter out the noindexed pages.

If you find there's no real discrepancy, Google may have simply been cleaning house of some really old links in the index that hadn't been crawled in a while.

Beyond that, if you can pinpoint any specific URLs that have been deindexed, use the "Fetch as Google" tool to help diagnose or post it here so the community can take a look.