Crawl Page Issues

[icrewz]

This post is deleted!

[Hutch42]

With Yoast, it creates different site maps for pages, posts etc.  I have found that indexing the various sitemaps and not just relying off of submitting /sitemap.xml helps with indexing.

[TheeDigital]

A site:icrewz.com shows 2,870 results indexed in Google.

https://www.google.com/search?sourceid=chrome-psyapi2&ion=1&espv=2&ie=UTF-8&q=site%3AiCrewz.com

However, when I try with Screaming Frog, I don't get nearly that many. Looking at the site briefly I think the problem may be that some results or pages are hidden either behind the search or from view due to login/signup.

[icrewz]

This post is deleted!

[TheeDigital]

331 pages returned by Screaming Frog.

Sorry, looks like the search wouldn't be causing it. I do see a lot of your crew pages are throwing a 301 redirect to screaming frog, but I can't duplicate this manually. It doesn't seem to crawl past page 9 though. Also, most of your internal links are redirecting as they include the city name in the url, and then strip the city out for the final page. I can't seem to access your robots.txt to see if something is preventing the crawling of these pages, but that might be where the problem is.

[icrewz]

This post is deleted!

[icrewz]

This post is deleted!