Questions
-
Crawl and Indexation Error - Googlebot can't/doesn't access specific folders on microsites
Hello ImpericMedia, If you can share the site with me (private message is OK) I'll look into it. If you don't want to do that, here are some things I would look at: 1. If you have verified that the Robots.txt file is not blocking the pages you want indexed, and the pages are still not indexed (or indexed with a message about the Robots.txt file) you should check for a Robots Noindex meta tag on the page. If the source code looks strange you may have to use the Chrome Inspect tool to see the fully rendered page. 2. If there are no blocking robots meta tags on the page you should check the HTTP response for an X-Robots header. 3. If there is no X-Robots header, it's probably because of the duplicate content and spammy(seeming) subdomain setup. Sorry about the wait. If you include the site URL it will get other community member's curious enough to check it out next time. I hope this helps. If not, feel free to message me.
Intermediate & Advanced SEO | | Everett0