Site not being crawled properly

[Ad-Rank]

This post is deleted!

[DirkC]

I tried crawling your site with Screaming Frog - and it crawled just fine. Personally I never use the tree view - but if you look at your site structure it looks pretty flat - so I doubt it's not following your subnav. It's accessible without js so it should not be a problem to crawl.

Careful when using keyword site:mydomain.com - the ranking displayed is not necessarily the ranking you will get when doing a "normal" search.

Even when you have more optimised pages inside your site - Google sometimes decides to show the homepage instead. Probably related to the importance of each page (I guess your home has more incoming links than the inner pages)

You should check your directives - sometimes you are using rel next/previous to group pages into 1 together with noindex (for 2nd and following pages) - which is confusing for the bots.

You also use canonicals for pagination -example  http://newyoubootcamp.com/testimonials/page/11/ has canonical: http://newyoubootcamp.com/testimonials - for pagination however you either have to use rel next/prev or noindex/follow - canonicals is intended for duplicate content not pagination.

Dirk

[Ad-Rank]

Thanks Dirk!

Where you say "rel next/previous to group pages into 1 together with noindex", could you give the URL? Thanks again, really appreciate this!

[DirkC]

Sorry - already deleted the crawl so can't give you the exact url's. If you do a crawl with Screaming Frog - go to the tab Directives - filter on noindex (or rel next/prev) and then check the Meta robots column for the conflicting directive.

Dirk

[Ad-Rank]

Great, thank you!

[MattRoney]

Hi Ad-Rank! If Dirk answered your question, mind marking one or both of his responses as a "Good Answer?"