Possible bug in Crawl Issues report?
-
Hi all -
My crawl issues report shows 3 pages with missing titles. These are just google verification files and the robot.txt file - shouldn't these be excluded?
Pages with Title Missing or Emptyas of May 11
URL Page Authority Linking Root Domains
https://www.mysite.com/googlea87e28121c071983.html
1 0
https://www.mysite.com/robots.txt
1 0
https://www.mysite.com/google9b9dc57478f61677.html -
Hi Matthew,
I think you will find that those urls are included in your sitemap.xml and that is why they are being crawled. They are also being indexed by google - you can search for the name of the verification files and they show up in the results. Remove them from the sitemap and you should be ok!
-
Wow that is interesting. Thanks for the info and direction Lynn. I will look into that.
Matt