Identifying why my site has a penalty

[pinnaclecarts]

http://www.webpagetest.org/result/130404_DN_168R/

[pinnaclecarts]

My crawl finished. You also have a bunch of status 418 "I'm a teapot" status codes. IDK what this is so I looked it up.

Per wikipedia:

418 I'm a teapot (RFC 2324)This code was defined in 1998 as one of the traditional IETF April Fools' jokes, in RFC 2324, Hyper Text Coffee Pot Control Protocol, and is not expected to be implemented by actual HTTP servers.

[madegood]

Wow, thanks for all this. It's late now in the UK so I'll check it out tomorrow.

Cheers

p.s. Where are my 418's coming from!?!

[pinnaclecarts]

private messaged you a google doc of the crawl. Looks like pages that no longer exist, they need 301's.

[madegood]

Got your message, thank you. What tool did you completed the crawl with? I'm sort of disappointed this stuff didn't come up in my seomoz weekly scans.

A few questions;

• How do i know where the 301's are being sent from? So in a this chain of events...

Link on a page on my site > routed via a 301 > landing on the desired page

... how do I find the first step in the process? the table you sent me seems to point out only the middle step.

• Yes the 'about us'  and 'contact us' pages are weak. I'm building a new version of the site as we speak and will take care of it then. In the mean time, if i no-index them is that as good as getting rid of them?

I will now sort the 404's and 418's. Without wanting to sound like a broken record; thanks again! Do let me know if there is anything I can do in return once we've got to the bottom of this.

Will

[madegood]

Just checked the 418's and they do seem to be already re-directed with 301's, or are actually in place. What would be the protocol here?

[pinnaclecarts]

Did you change them. The scan I just did doesn't show them.... Maybe your host was getting funky or something lol.

Get this and click the links on your site. You want to link to status code 200, not 301

https://chrome.google.com/webstore/detail/server-status-code-inspec/bmngiaijlojlejaiijgedgejgcdnjnpk

I wouldn't de-index them, I havent found a legitimate reason to de-index anything since 2005, but im a programmer and normally don't need to patch things. You could probably quickly fix them just by adding some content/images.

im going to private msg you another spreadsheet. this should show you source+destination of your 404's and 301's.

btw, the spider im using is Screaming Frog, its the best I've found.

[madegood]

Thanks for the table of links. I'll see to it.

I'll work on the code on the new version of the site, seems pointless to do it now.

I've installed the plugin. How do I change the status code of a page? I don't really understand how it can be anything but 200, as if i'm viewing it it's obviously there! I thought 301's pushed the user to the 200 version of the page and only existed temporarily in the browser? Obviously I'm wrong, perhaps you could explain it for me?

Cheers for the screaming frog tool, looks great.

[pinnaclecarts]

Ok so. If a bit of content resides at /bikes/mountain-bikes/ and the menulink I use is /bikes/mountain-bikes/ I'll get a status code 200. There is no added delay, no page rank lost, 200 == OK. The menu link points directly to the destination content.

Now lets say you've decided to change the location of that content to /bikes/mountain-bikes/index.html.

You do the 301 redirects on from the old url to the new one, THEN you need to update your links to reflect the new location so you're not just pointing at 301 redirects.

[madegood]

Oh okay! That makes sense. Found a few issues with my php rules that automatically write links on a few of my contents pages.

I've learned some valuable tips here, such a fantastic help. I'm going to get the new site up in a week or two and we'll see if things change.

I'll keep you updated!

[pinnaclecarts]

Great. Just audit it, fix problems, audit again, write more great content and give it time. Even if you fix the problem (assuming it was an onsite problem) it may take some time for Google to show the love agian.