Fetching & Rendering a non ranking page in GWT to look for issues

[Dan-Lawrence]

Hi

I have a clients nicely optimised webpage not ranking for its target keyword so just did a fetch & render in GWT to look for probs and  could only do a partial fetch with the below robots.text related messages:

Googlebot couldn't get all resources for this page

Some boiler plate js plugins not found & some js comments reply blocked by robots (file below):

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

As far as i understand it the above is how it should be but just posting here to ask if anyone can confirm whether this could be causing any prrobs or not so i can rule it out or not.

Pages targeting other more competitive keywords are ranking well and are almost identically optimised so cant think why this one is not ranking.

Does fetch and render get Google to re-crawl the page ? so if i do this then press submit to index should know within a few days if still problem or not ?

All Best

Dan

[Toddfoster]

Hello again, Dan,

From what I can tell from your description, you have done what you can to make this work. We would expect JS to be blocked by that robots.txt file.

To answer your questions:

Fetch & render does allow Google to re-crawl the page using GWT. A request of this nature typically takes between 1-3 days to process, so you should know where you stand at that point.

Feel free to put an update here and if there is further information I will see what I can do to help out.

Cheers!

Rob

[Dan-Lawrence]

Hi Robert,

Thanks for your help again !

That's great thanks, but what about 'submit to index' which i did also ? As in did i need to do that or not ?(since GWT says all pages submitted are indexed in sitemap section of GWT, so i take it i didn't need to, but did anyway as a precaution) ?

All Best

Dan

[Toddfoster]

Yes there are a lot of overlaps when it comes to GWT - for the most part if you are making a submission request for crawling, it is indexed simultaneously - I believe the difference lies in some approaches which allow you to crawl as Google as opposed to submitting for official index.

In other words, what you have done is a definitive step in crawling and indexing, as opposed to seeing what Google would find if it were to crawl your site (as a test). "Submit to Index" is normally something I reserve for completed sites (as opposed to Stub content) to avoid accidental de-indexing.

In your circumstances, however, I don't think it will hurt you and it may help you identify any outstanding issues. Just remember to avoid it if you don't want a site indexed before it is ready!

Hope this helps,

Rob

[Dan-Lawrence]

Marvellous !

Many Thanks Robert !

All BEst

Dan

[DonnaDuncan]

Hi Dan,

You might find this Q&A helpful. It offers suggestions for what to do when an unexpected page is ranking for your targeted keyword phrase. I think most, if not all, suggestions apply in your case as well. Good luck!

[Dan-Lawrence]

Thanks for commenting Donna !

And providing the link to the interesting Q&A although this isn't the scenario i'm referring to with my original question.

The page isn't ranking at all although its very well optimised (and not overly so) and the keyword isn't that competitive so i would expect to be somewhere in the first 3-4 pages but its not even in first 100

Very similarly optimised pages (for other target keywords which are more competitive) are ranking well. Hence the fetch and render & submit to index i did, just to double check Googles seeing the page.

Cheers

Dan

[DonnaDuncan]

Can you see if it's cached? Try cutting and pasting the entire URL into the search window, minus the http://. If it's indexed, it should show up in search results. Not the address bar, the search window.

[Dan-Lawrence]

thanks Donna ! yes its all there and cache date is 15 Jan but still thought worthwhile fetching & rendering & submitting again, or does that do nothing more if its already indexed apart from asking G to take another look ?

[DonnaDuncan]

If anything changed between the 15th and today, it'll help ensure it gets updated. But that's all.

[Dan-Lawrence]

ok thanks !

nothing has changed just hoped it might do something