Why Are Some Pages On A New Domain Not Being Indexed?
-
Background: A company I am working with recently consolidated content from several existing domains into one new domain. Each of the old domains focused on a vertical and each had a number of product pages and a number of blog pages; these are now in directories on the new domain. For example, what was www.verticaldomainone.com/products/productname is now www.newdomain.com/verticalone/products/product name and the blog posts have moved from www.verticaldomaintwo.com/blog/blogpost to www.newdomain.com/verticaltwo/blog/blogpost. Many of those pages used to rank in the SERPs but they now do not.
Investigation so far: Looking at Search Console's crawl stats most of the product pages and blog posts do not appear to be being indexed. This is confirmed by using the site: search modifier, which only returns a couple of products and a couple of blog posts in each vertical. Those pages are not the same as the pages with backlinks pointing directly at them.
I've investigated the obvious points without success so far:
- There are a couple of issues with 301s that I am working with them to rectify but I have checked all pages on the old site and most redirects are in place and working
- There is currently no HTML or XML sitemap for the new site (this will be put in place soon) but I don't think this is an issue since a few products are being indexed and appearing in SERPs
- Search Console is returning no crawl errors, manual penalties, or anything else adverse
- Every product page is linked to from the /course page for the relevant vertical through a followed link.
- None of the pages have a noindex tag on them and the robots.txt allows all crawlers to access all pages
One thing to note is that the site is build using react.js, so all content is within app.js. However this does not appear to affect pages higher up the navigation trees like the /vertical/products pages or the home page.
So the question is:
"Why might product and blog pages not be indexed on the new domain when they were previously and what can I do about it?"
-
Ben, I'm assuming that you have 301 redirects properly set up for all of those page. Then, make sure you've used the Google Change of Address Tool to properly tell Google that you've moved from one domain to another. If you are consolidating domains, then you need to verify all of those sites in Google Search Console and then use the Change of Address Tool.
You don't mention how long it's been since you've set up those 301 redirects, but it literally can take months before everything is straightened out in Google.
One thing you can also do is to look at the site's log files to see if Google is crawling those pages--it could be that they're not crawling. If they have crawled, then it might just take time before they're indexed.