Googlebot stopped crawling

[1akal]

Hi All, One of my website stopped showing in SERP, after analysing in webmaster, found that Googlebot is not able to crawl. However it was working alright few days back. Try to investigate for panelisation, but  no intimation found. I checked robot.txt for no follow etc but all seems to be ok. I resubmitted Sitemap in webmaster again, it crawled 250 pages out of 500 but it still site is not available in SERP (google), in bing it is ok.

Pl suggest the best possible solutions to try.

Thx

[customerparadigm.com]

There may be many technical things going on with your robot.txt file, no-index tags, etc.

But where I would start first is with your website hosting company.

My guess - not having seen your site - is that you may be hosted on a site with a low-cost hosting provider.  And you are experiencing downtime at random times that are affecting Google's ability to crawl your site.

The other clue that points me to your web hosting service is that Google tried to crawl 500 pages, but it was only able to handle 250.

What I would do is first look and see if your site is timing out under heavy loads / lots of visitors.

That's most likely the culprit.

I'd subscribe to a free site monitoring service that will ping the site every 5 minutes or so, and email you if it goes down.

Hope this helps!

-- Jeff

[TakeshiYoung]

Very hard to say without more details. Does your site have unique, high quality content? If it's just duplicate content, Google may crawl it but won't necessarily show it in the SERPs.

Also, what does your backlink profile look like? Google allocates crawl budget based on your PageRank, so if Google isn't crawling all your pages, then you will want to acquire more external backlinks.

[Prospector-Plastics]

This might be a shot in the dark not knowing much about your site - can you check in Google Webmaster to see if you accidentally removed your website using the Remove URLs tool? I know of someone that accidentally did this when copying a pasting a URL, but accidentally only copied their main website address and not the full URL (oops!) and their site dropped out of Google SERPs rather quickly. Just a thought...