Angular SPA & MOZ Crawl Issues

[gparashar]

Website: https://www.exambazaar.com/

Issue: Domain Authority & Page Authority 1/100

I am using Prerender to cache/render static pages to crawl agents but MOZ is not able to crawl through my website (https://www.exambazaar.com/). Hence I think it has a domain authority of 1/100. I have been in touch with Prerender support to find a fix for the same and have also added dotbot to the list of crawler agents in addition to Prerender default list which includes rogerbot. Do you have any suggestions to fix this?

List: https://github.com/prerender/prerender-node/commit/5e9044e3f5c7a3bad536d86d26666c0d868bdfff

Adding dotbot to Express Server:
prerender.crawlerUserAgents.push('dotbot');

[dave.kudera]

Hey, Dave from the Help Team here!

I just ran your site through our crawler and didn't have any issues crawling your site.  I think the Domain Authority score of 1/100 is actually caused by us not having your site in our Mozscape Index.  To find out more about how we Index the web, you can read about it here

https://moz.com/help/guides/research-tools/open-site-explorer/updates/index-process

Hopefully that helps to clarify, let us know if you need anything else!

[Raptor-crew]

Hi Gaurav

We're also launching a new MEAN SPA, We've gotten our site looking quite good for SEO now, other than the meta description and canonical tags.

However I believe the MOZ crawler still doesn't understand javascript, we use AngularJS or JQEURY to change and add the meat tags for each page, which I am confident is OK with Google but MOZ and SEMRush don't like it.

That could be something to do with the lower score on Moz?

Do you mind me asking how do you set your page meta descriptions and canonical tags?

Dave

[gparashar]

Hi Dave

Thanks for asking. You can set the pageDescription variable (similarly for other meta tags) in each route/state of the SPA which gets cached by Prerender. If you are using prerender you can see the list of default crawler agents supported here: https://github.com/prerender/prerender-node/commit/5e9044e3f5c7a3bad536d86d26666c0d868bdfff

If you wish you can enable prerender for the crawler dotbot too!

[Raptor-crew]

Cheers Gaurav

I will try that technique.  We use something similar for the title but in the Angular app.js file, however have struggled to do the same for the description, canonical tags etc.

Actually right now we add the tags we need in Javascript, using the DOM and prerender and google seems to like it.  We can;t run a crawl any longer in the std Moz package for some reason we can only do one per week now.  Moz doesn't seem to cope too well with the Javascript though.  When we tried other tools it seemed to c heck out fine.

I'm not sure if it is Moz crawlers that are causing us the issue here, anyway!

We're also still using Angular 1.6 so we really need to upgrade to v4 or at least 2.