Welcome to the Q&A Forum

Okay, thanks Alan!

Yes it will continue crawling or yes it will stop the crawl?

Hey Matt,

Thanks for the reply. I'm aware of all the best practice stuff but thanks for sending through. It didn't quite answer my question so let me rephrase...

Will a bot follow a hyperlink (like the example below) on a 404 page or will it stop the crawl on that page (not on the whole site) because the header response code is a 404?

Recent Post Title

I'm currently in the process of redesigning my site's 404 page. I know there's all sorts of best practices from UX standpoint but what about search engines? Since these pages are roadblocks in the crawl process, I was wondering if there's a way to help the search engine continue its crawl.

Does putting links to "recent posts" or something along those lines allow the bot to continue on its way or does the crawl stop at that point because the 404 HTTP status code is thrown in the header response?

That logic seems incorrect since it doesn't account for root domain links that point to the subdomain. This would only apply if all the inbound links were from other root domains.

For example, my blog on the subdomain has 1.4M inbound links, 1.35M of which come from the root domain. I'm guessing this is because it's a footer link. So, the PR6 of the blog seems largely inherited from the root domain, which has a PR of 6.

Were you just trying to oversimplify it?

Hi all,

I'm currently in the process of revitalizing my company's blog. Currently, the blog sits on a subdomain (blog.rootdomain.com). SEO best practice dictates that I should move this (and 301 redirect the old URLs) to rootdomain.com/blog to concentrate link equity and avoid the risk of having search engines treat the subdomain as separate from the root domain.

However, the PageRank Status extension for Chrome is reporting that the PR for the blog on the subdomain and the root domain are the same. Is there any benefit to migrating the subdomain to a subdirectory? Is that data accurate enough to base decisions off of?

Hi all,

The current roadmap for our Eng team has us performing a site migration (redirecting one subfolder to another subfolder) and then a domain migration shortly after. The way I see it, I have 2 scenarios (the 1st involves the site migration THEN the domain migration and the 2nd is the site migration and domain migration being done simultaneously):

1. olddomain.com/subfolder-old to olddomain.com/subfolder-new THEN olddomain.com/subfolder-new to newdomain.com/subfolder-new AND olddomain.com/subfolder-old to newdomain.com/subfolder-new

2. olddomain.com/subfolder-old to newdomain.com/subfolder-new

I also understand that there are two best practices for a domain migration and they are 1) keep everything the same that you can to help Google understand it is the same page, just on a different domain and 2) avoid chain redirects.

As you can imagine, scenario 1 requires more Eng costs than scenario 2. So, my question is, is scenario 2 a perfectly viable option or should I make the push to go for scenario 1?

Any advice is greatly appreciated!

Without poking around your blog, I can't really offer much more advice.

Well that would be why your category page is outranking your homepage for that keyword.

It's not advised to have two different pages optimized for the same keyword. The SEO world calls this keyword cannibalization. Reason being, you're splitting your SEO juice amongst two links instead of funneling it all into one. Having a single page with 10 external links pointing to it is better than having two pages with 5 external links pointing to each.

Based on your username, I'm guessing you're running a blog. Is your homepage just the most recent 4-5 posts that you've put up with some other widgets? If so, you're homepage probably won't rank for much other than generic terms like "Niche Topic Blog" or your brand keywords. Which is okay. Your blog posts and categories are where you should target long-tail keywords.

First, ensure your homepage has been indexed by Google. In Google, do a search for site:yourhomepageURL.com. If nothing comes up, then that would be why your homepage isn't ranking. If that returns search results, then...

Do you use the SEOMoz PRO application? They have a handy report card feature that grades your on-page SEO factors. Perform an on-page audit on those two pages (homepage and category page). That should reveal why the category is outranking the homepage.

Also, you should see which page has more links to it in Open Site Explorer or Google Webmaster Tools and what the anchor text of those links is. If more people are linking to your category page instead of your homepage that could be one reason why.

When you say "competing", do you mean the category page and homepage are close together in the SERPs or just optimized for the same keyword? Did you intentionally optimize the homepage for that keyword?

It all comes down to which page is truly more relevant to the search query. For example, let's pretend you and your competitor sell dog accessories (collars, leashes, food, toys, etc). Someone searching for "dog leashes" would much rather land on your page for "dog leashes" rather than your homepage or your competitor's homepage since it puts them on exactly what they were looking for (rather than having to navigate their way from your homepage to the dog leashes page). Make sense?

Also, assuming your category page is intended to outrank your homepage for that keyword, you wouldn't be competing with your homepage unless it is ranking higher than your category page. Having your homepage ranking for that same keyword is only an issue if it appears before your category page in the SERPs.

Hope that helps!

Hi all,

The current roadmap for our Eng team has us performing a site migration (redirecting one subfolder to another subfolder) and then a domain migration shortly after. The way I see it, I have 2 scenarios (the 1st involves the site migration THEN the domain migration and the 2nd is the site migration and domain migration being done simultaneously):

1. olddomain.com/subfolder-old to olddomain.com/subfolder-new THEN olddomain.com/subfolder-new to newdomain.com/subfolder-new AND olddomain.com/subfolder-old to newdomain.com/subfolder-new

2. olddomain.com/subfolder-old to newdomain.com/subfolder-new

I also understand that there are two best practices for a domain migration and they are 1) keep everything the same that you can to help Google understand it is the same page, just on a different domain and 2) avoid chain redirects.

As you can imagine, scenario 1 requires more Eng costs than scenario 2. So, my question is, is scenario 2 a perfectly viable option or should I make the push to go for scenario 1?

Any advice is greatly appreciated!

That logic seems incorrect since it doesn't account for root domain links that point to the subdomain. This would only apply if all the inbound links were from other root domains.

For example, my blog on the subdomain has 1.4M inbound links, 1.35M of which come from the root domain. I'm guessing this is because it's a footer link. So, the PR6 of the blog seems largely inherited from the root domain, which has a PR of 6.

Were you just trying to oversimplify it?