My crawl can't find ANY product pages. The links to product pages aren't links, they're script. :(

[dwayne.jones26]

This post is deleted!

[Joe.Robison]

Looking at the source code for that specific product on the page, I do so the HTML link visible in the source code from the front end, so if you're talking about the link to that one product showing on the page then yes your link is crawlable.

See the attachment for an example. Is this site currently in development mode? If so then you already know that the http://acceptance.madeblinds.com/robots.txt is set to not allow Google to crawl the site. But if the site is supposed to be live, then you have a problem there!

8tthjk

[DanSullivan]

Thanks Joe.

I see your screenshot shows a link clearly in the inspector (which I didn't think to look at), but the source code itself shows script only.  I'm using a crawler from screaming frog and it can't follow the link.  At the moment I can't check in GWT (due to Robots.txt - and yes, it's in dev!)  How do these two view jibe?

Thank you!

2015-10-21_14-22-34.jpeg

[DirkC]

The answer from Joe is not really correct - I you do a "inspect element" it shows the code including the modifications from executed javascript.

"The DOM tree view shows the current state of the tree; it may not match the HTML that was originally loaded for different reasons. For example, you can modify the DOM tree using JavaScript; the browser engine can try to correct invalid author markup and produce an unexpected DOM."

Source: https://developers.google.com/web/tools/chrome-devtools/iterate/inspect-styles/basics#inspect-an-element

Google is able to understand javascript - so it will see the link and crawl. Tools like Screaming Frog and the Moz Crawler won't be able to crawl it.

Hope you know that without javascript - the only thing you see is a blank page. Personally I would rather try to degrade gracefully - meaning that visitors with no javascript enable would still be able to visit your site.

Dirk

[DanSullivan]

Dirk,

Thank you very much for the clear explanation.  I'm a lot more confident that we don't have an indexing issue brewing, but disappointed that I can't use Screaming Frog like normal and still wary that the link is somehow compromised in a way I can't articulate.

Do you think there's a benefit to changing to regular href links, or do I just need to update my thinking?

I understand your note about degrading gracefully also.

Thank you!

[DirkC]

I would try to make sure that even without js your site is crawlable (even if it's only to be able to use Screaming Frog

If you keep it this way - you must make sure that you don't block the js files from indexing. This is a general recommendation from Google but in your case it is a must. If these files are blocked the Google bot will only be seeing the blank page without links.

Dirk

[DanSullivan]

Dirk,

Excellent tip.  Thanks for keeping me out of trouble!

Dan

[Joe.Robison]

Thanks for double checking that Dirk, I didn't know that the inspect element tool and source code would show different code, but it makes sense.