Only 1 Page Being Crawled
-
I have a website I'm tracking www.alhi.com. But my report is saying that only 1 page is being crawled each update. My campaign is set up for the sub domain www.alhi.com, so I'm not sure why I would have this issue. Can you help? Thanks!
-
May be I am not seeing properly but where is your sitemap?
Also, do you add fresh content deeper pages frequently? These intelligent little bots crawl a good deal initially but they tend to 'learn' the frequency of your content updates and update their re-visit schedule accordingly. As busy as these little ones are, you have to feed them something worth devouring and do it so frequently, so their frequent visits to your site is worth their time. If they keep visiting your site and don't find nothing new..they have just spent all their energies for nothing and hence they become lazy for future visits, and this could explain delay to deeper pages. Just my thought... there could be something to else to it too.
-
<META NAME="robots" CONTENT="noindex, nofollow, noarchive, noimageindex"> I would start here -
THANKS SO MUCH! I had my developer add this to our dev site but apparently it was added to both.