4xx error - but no broken links founded by Xenu

[Letty]

In my SeoMoz crawl report I get multiple 4XX errors reported and they are all on the same type of links. www.zylom.com/nl/help/contact/9/  and differiate between the number at the end and the language.

But I i look in the source code we nice said:

  <a class="<a class="attribute-value">bigbuttonblue</a>" style="<a class="attribute-value">float:right; margin-left:10px;</a>"
             href="[/nl/help/contact/9/?sid=9&amp;e=login](view-source:http://www.zylom.com/nl/help/contact/9/?sid=9&e=login)" onfocus="<a class="attribute-value">blur()</a>"
             title="<a class="attribute-value">contact</a>"> contact a>

I already tested the little helpfull tool Xenu, but this also doesn't give any broken links for the url's which I found in the 4xx error report.

Could somebody give me a suggestion Why these 4xx errors keep coming? Could it be that the SeoMoz crawlers break the part ?sid=9&e=login' from the URL. Because if you want to enter the link, you first get a pop-up to fill in a login screen.

Thanks for you answers already

[Marcus_Miller]

Hey Letty

It is somewhat hard to say without seeing a link but if Xenu says no broken links then it is likely as you suspect and the moz crawler is having a problem with those links.

Maybe try to confirm one way or the other in Google Analytics. An easy way (from memory) to do this is to search your pages so Content > Site Content > All pages for your custom 404 page (if you have one). Then, you can dig in and see if these are real errors or are just some kind of red herring (most likely).

Hope that helps!
Marcus

[Letty]

Hi Marcus,

A link would be www.zylom.com/nl/help/contact/9/?sid=9&e=login

I think the pop-up screens which requested an login, is causing the 4xx error. because you can't enter without login. Think we should put a no-follow on these types of url's and that could solve the problem. What are your thoughts after seeing the link.

[Marcus_Miller]

Hey Letty

If I request the full link in webbug I get a 200 okay response.

http://www.zylom.com/nl/help/contact/9/?sid=9&e=login

If I request the link less the variables in webbug I get the 404 as reported by the moz crawler.

http://www.zylom.com/nl/help/contact/9/

I would go into analytics, check to see which pages you have 404's for if it is just  '/nl/help/contact/9/' then I don't think you have a problem.

I think nofollow would be a good solution and if we look at part of the nofollow page on google webmaster support:

"Before nofollow was used on individual links, preventing robots from following individual links on a page required a great deal of effort (for example, redirecting the link to a URL blocked in robots.txt). That's why the nofollow attribute value of the rel attribute was created. This gives webmasters more granular control: instead of telling search engines and bots not to follow any links on the page, it lets you easily instruct robots not to crawl a specific link. For example:"

Now, I am not 100% sure if the seomoz crawler respects nofollows (I would imagine it does) but certainly, if other robots are following this link and generating 404's (again, head into your analytics to get a handle on this) then it will certainly help there.

After all, this link is not for robots, it is not to be indexed, it is just a log in form for users so nofollowing that link seems to make perfect sense.

Hope that helps!
Marcus

References:

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=96569