Drop in number of Pages crawled by Moz crawler
-
What would cause a sudden drop in the number of pages crawled/accessed by the Moz crawler? The site has about 600 pages of content. We have multiple campaigns set up in our Pro account to track different keyword campaigns- but all for the same domain. Some show 600+ pages accessed, while others only access 7 pages for the same domain. What could be causing these issues?
-
AllaO
Do you have some campaigns set to track at the root and some at the sub domain? That would be the only way I could think of when they are essentially being run simultaneously.
Best
-
AllaO,
Sorry about the odd crawl activity. I pulled up your campaigns and it looks like something blocked rogerbot from crawling. This could be new redirects or changes in your robots.txt file. If you don't think one of those is the issue get a hold of our support team: https://seomoz.zendesk.com/forums You can submit a ticket from here.
Best of Luck
-
Hi Nick,
Thank you very much.. no actually we did not make any changes to robot.txt, but we have made some changes to sitemap.xml
Could it create a temporary problem?
-
If that was how our crawler was accessing your pages before, because there aren't links for them else where, then it could be the issue. It does depend there. There are a lot of aspects to how our crawler works so it is really hard to say.