404-like content

[eseyo2]

A site that I look after is having lots of soft 404 responses for pages that are not 404 at all but unique content pages.

the following page is an example:

http://www.professionalindemnitynow.com/medical-malpractice-insurance-clinics

This page returns a 200 response code, has unique content, but is not getting indexed. Any ideas?

To add further information that may well impact your answer, let me explain how this "classic ASP" website performs the SEO Friendly url mapping:

All pages within the custom CMS have a unique ID which are referenced with an ?intID=xx parameter.

The custom 404.asp file receives a request, looks up the ID to find matching content in the CMS, and then server.transfers the visitor to the correct page.

Like I said, the response codes are setup correctly, as far as Firebug can tell me.

any thoughts would be most appreciated.

[YannickVeys]

I'd say: the URL isn't accessible via the menu? Can't find it anywhere? I tried looking under http://www.professionalindemnitynow.com/Medical-malpractice-insurance but couldn't find a link to the page. Is the page only located in your sitemap? That might be why it isn't indexed. Link to it (more!)

The other thing is o/c: high keyword density/spammy usage of the keywords you are targetting

[RyanKent]

Hi Scott.

I am confused why you refer to the link you shared as a soft-404. http://www.professionalindemnitynow.com/medical-malpractice-insurance-clinics. The page title is "Medical Malpractice Insurance for Clinics" which is a perfect match for the URL. The page returns a 200 response header code. By all counts this appears as the proper page which should be returned and not a 404 in any way.

If you have a 404 error log file which shows this page as a 404 error, that issue is completely internal to your site. From the perspective of Google and the rest of the world your site is working perfectly. If the only place the page shows as a 404 is your log file, you want to check with a developer to determine exactly what is triggering the file entry.

With respect to indexing, I support Yannick's findings.

[eseyo2]

The reason I refer to it as a soft 404, is the listing within webmaster tools. See attached image for more examples.

You're right - it is not on the sitemap which I need to address, but still dont see why Google detect this as a 404 when it clearly 200's.

Thanks for your response.

soft-404.jpg

[eseyo2]

Thanks Yannick. Completely agree with the content of the page using the keywords too frequently. This is the site owner claiming to "understand" SEO! I will advise him that he needs to calm down the keyword stuffing.

I'm going to add the page, and other similar landing pages that are used for Adwords, to the public sitemap

[AlanMosley]

You could try using either the global.asx file or a http model to do the rewiring, global.asx would be the easiest.

from memory the begin_request event would be the one to use.

the thing is you need to do the rewriting earlier in the event cycle.

[RyanKent]

The page we are discussing is not listed in the image you shared.

I checked one link which is listed: http://www.professionalindemnitynow.com/business-consultants-quote

The top of the page says "Error - The page you have tried to access cannot be found"

While the page returns a 200 header code, Google is likely seeing the page header text and recognizing it as a "404-like" page as they shared.

[AlanMosley]

This is the point. it should return a 404, but instead returns a 200, this is what is called a soft 404.
See my other comment on how to fix.

[AlanMosley]

i think how google does detects a soft 404 is like this.
http://www.professionalindemnitynow.com/gobblygook should return a 404, but returns a 200, so they now know that you site is prone to soft 404's
but how do they then decide what pages on the site are and what are not s404's is not clear. From reading, my best understanding is that they then look for simularities, to the know s404, such as timings, and other criteria.

[AlanMosley]

Scott, if you fix the problem by using the global.ax fiile, remeber to the make sure that the 404 page does then retuurn a 404.