Seo moz has only crawled 2 pages of my site. Ive been notified of a 403 error and need an answer as to why my pages are not being crawled?

[nitro-digital]

SEO Moz has only crawled 2 pages of my clients site. I have noticed the following.

• A 403 error message
• screaming frog also cannot crawl the site but IIS can.
• Due to the lack of crawling ability, im getting no feed back on my on page optimization rankings or crawl diagnostics summary, so my competitive analysis and optimization is suffering

Anybody have any idea as to what needs to be done to rectify this issue as access to the coding or cms platform is out of my hands.

Thank you

[Jacobe]

Please share the URL of your website so we can check.

[nitro-digital]

the url is http://www.inoncology.com/

[jesse-landry]

no robots file so it's not that. I do notice that your landing page has an annoying popup that forces a decision and then redirects you upon choosing it... I'm not 100% sure but that might have something to do with it. I can't seem to get past that and onto the site itself honestly. Everytime I click a selection I get to another page with a popup. 
If I were an average internet browser *(which in this instance I am) I would leave your site and not return. Poor user experience hurts ultimately, and that was what I experienced.

Not trying to be rude, just trying to be helpful. Sorry if I came off cross.

G'luck!

[nitro-digital]

Not at all Jesse. Thank you for taking a look

Brief summary - the pharmaceutical industry has to have a disclaimer on their pages for legal reasons. Forcing a selection/disclaimer is required  by law. If you select the first option, it'll take you to the web address intended.

In terms of it being related to the disclaimer, SEO Moz was indexing it with this disclaimer up until a few months ago. There's no robot txt file as you stated, so that also cannot be the reason for this.

hhhhmmm........the mystery continues!!!

[LynnPatchett]

Hi Andy,

Not sure why you are seeing a 403 error (I don't see this in the headers), but you seem to have an issue with your rel=canonical tag on your homepage. It is showing a relative link to "/home" which leads you to this page: http://www.inoncology.com/home/ which is actually showing a 200 OK status, but is obviously not!

Digging into the headers a bit more it seems you might have a circular redirect going on before that, I am seeing /home redirect to /content/internet/pm/inoncology/com_EN which seems to redirect back to /home/.

No 403 in all of those, but maybe the combination of wrong rel canonical tag plus the mixed up redirects is causing the moz crawler and screaming frog to choke?

[nitro-digital]

Hi Lynn

Thank you for taking a look into this issue, your feedback has given me food for thought.

I am going to forward on your feedback to the powers that be on the web development side of things and see if this is an answer to the problem.

I've just looked at http://www.inoncology.com/home/ and from first review, it seems that this is where the crawlers are breaking down, as it comes to a dead end.

Hopefully this is the reason.

Thank you once again

[JoelDay]

Hey Andy,

You've gotten a lot of good info here and in your email with Maura. I just wanted to add in here that the crawl trouble you're having (at least with our system) is with the http://www.inoncology.com/ home page itself.

You've got a 301 redirect from the root to the www subdomain, the www then returned a 403 forbidden to our crawler.

It may have something to do with the verification you require, it may be something more obvious on the server level. It's tough to say.

Either way, you'll want to get this addressed to make sure your site is crawlable.  

I hope that helps.
Cheers,
Joel.

[nitro-digital]

Hi Joel,

I have indeed!

I'll take all this information to the development team and hopefully have this issue resolved asap.

Thanks once again to everybody for their input.

Regards,

Andy