"/blogroll" causing 404 error

[Baffo]

I'm running a campaign, and the crawling report for my site returned a lot of 4xx errors. When I look at the URLs, they all have a "/blogroll" in the end, like:

mysite.com/post-number-1/blogroll

mysite.com/post-number-2/blogroll

And so on, for pretty much all the pages. The thing is, I removed the blogroll widget completely, so I really wouldn't know what can possibly point to links like that.

Is there anything to fix on the site?

Thanks

[donford]

Hi Andrea

I'm not sure about the issue, but it may help others if you mention what type of software you're running.

I would assume Wordpress since you said widget but could also be Joomla or another CMS.

Good Luck,

Don

[Baffo]

Hei Don,

thanks for the quick help.

Yes, I'm running Wordpress, with the Catalyst framework.

I was using the blogroll widget in the sidebar, but when I started to see the crawling errors I removed it just in case. The crawl is now complete, but even more errors of the same type have come out.

[Adam.Whittles]

Hi Andrea,

If the crawl is returning 404 errors then this means, although you have removed the widget, the pages are still being linked to somewhere on your site.

My advice would be to use the Screaming Frog crawler or if you have access to another crawler then use that. Once you have crawled the site using a crawler, you should be able to find out which pages are still linking to the 404 pages. Once you have found these, you will get a better idea of how to fix the issue.

Remember, a crawler will crawl your entire site, including all links, and if 404s are found then these are being linked to internally.

Hope that helps,

Adam.

[benjaminspak]

Check GA (Google Analytics)

1. Are the 404d pages receiving search traffic?
2. Are the 404d pages ruining your user experience? (Are they accessible via your site links)

If no to both, is this really worth a couple hours of your time?

[Adam.Whittles]

Sorry Ben but I have to disagree with you here. That is very bad practice and also very poor advice. You shouldn't just ignore 404 pages from a site crawl.

Really the only time you should let pages just 404 is when Google has indexed them, there is no relevant page on your site to redirect them to, there are no high value links pointing to them and they are not being linked to from within your site.

However, in this case the 404 pages are being linked to from within the site. This means that value is being passed to these pages from within the site that could otherwise be passed to other pages.

Best practice in this situation is to fix the links that point to the 404 pages and 301 redirect the 404 pages to relevant pages on the site.

P.s. running a quick site crawl and fixing the 404s should only take minutes and not hours to do!

[Baffo]

OK, so, I crawled my site with Screaming Frog and found the same errors. Actually I found out that the "privacy policy" page is causing the same 404 with the same type of URL "mysite.com/post-number-1/privacy-policy" (SEOmoz crawler had detected those as well, I just hadn't noticed).

The privacy policy page is actually published, but I cannot remove it, as I wouldn't be compliant with Google Adsense policy.

A couple of more things though:

• I checked a couple of those 404 pages in Google with the "site:" command, and they're not indexed. I think those pages simply don't exist.

• the blogroll was in the sidebar, and the privacy policy page is in the footer, which means, both of them are site-wide

• I had a site before, then I deleted it and started my current one from scratch, importing all the content from Wordpress to Wordpress. Maybe this transfer has something to do with the issue?

[evolvingSEO]

Hi Andrea

Are you all set with this? The transfer may have had to do with it, but the main importance now is to follow Adam's good advice - find the source of the 404 links and change them on your site. If they're indexed or backlinked to from elsewhere on the web, you need to 301 them to an existing page.

Let us know if you still need help!

-Dan