The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Crawl questions

    Crawl questions

    Intermediate & Advanced SEO
    3 2 856
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • RyanKent
      RyanKent last edited by

      My first website crawl indicating many issues. I corrected the issues, requested another crawl and received the results.  After viewing the excel file I have some questions.

      1. There are many pages with missing Titles and Meta Descriptions in the Excel file. An example is http://www.terapvp.com/threads/help-us-decide-on-terapvp-com-logo.25/page-2

      That page clearly has a meta description and title. It is a forum thread. My forum software does a solid job of always providing those tags. Why would my crawl report not show this information? This occurs on numerous pages.

      2. I believe all my canonical URLs are properly set. My crawl report has 3k+ records, largely due to there being 10 records for many pages. These extra records are various sort orders and style differences for the same page i.e. ?direction=asc.

      My need for a crawl report is to provide actionable data so I can easily make SEO improvements to my site where necessary. These extra records don't provide any benefit.

      IF the crawl report determined there was not a clear canonical URL, then I could understand. But that is not the case. An example is http://www.terapvp.com/forums/news/ If you look at the source you will clearly see

      Where is the benefit to including the 10 other records in the Crawl report which show this same page in various sort orders? Am I missing anything?

      3. My robots.txt appropriately blocks many pages that I do not wish to be crawled. What is the benefit to including these many pages in the crawl report?

      Perhaps I am over analyzing this report. I have read many articles on SEO, but now that I have found SEOmoz, I can see I will need to "unlearn what I have learned". Many things such as setting meta keyword tags are clearly not helpful. I wish to focus my energy and I was looking to the crawl report as my starting point. Either I am missing something, or the report design needs improvement.

      1 Reply Last reply Reply Quote 0
      • Francisco_Meza
        Francisco_Meza last edited by

        Your source code looks different to me, if not .... abnormal.  What's with   doing above the description anyway. I don't know if that code is blocking the crawler from that code to . I'm no code monkey, but all my metas are right next to each other.  Yours are completely separated from the title. Take out that code and see if it happens again. OR better yet, go to GWT and FETCH AS GOOGLEBOT. For sure that will tell you what Google bot is seeing.

        1 Reply Last reply Reply Quote 0
        • RyanKent
          RyanKent last edited by

          Those are commends added to the code. My site has is part of the effort to rid the internet of IE6 browsers. You can read more about it http://www.theie6countdown.com/join-us.html. There is a simple script placed in the site which detects IE 6 & 7 browsers and asks users to update their software.

          TYNT is a SEO tool http://www.tynt.com/. It allows webmasters to track any information which is copied and pasted from their site. It creates links back to the site, and tracks all activity on those links.

          Both of those scripts are working normally and neither should negatively impact crawls. My Google WMT show my site being crawled normally, no errors. I have multiple pages ranked #1. With that said, there are plenty of opportunities for me to improve, which is why I am here.

          The position of the Title and meta tags within the head should not be a factor at all. I did hear Matt Cutts share once that webmasters could move the information to the top to help if there are other issues such as a page where the tag is not properly closed, but that is not really relevant in this case.

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Backlink Question
            gennarojewelers
            gennarojewelers
            0
            2
            75

          • What to do when on your keyword there are no questions ?
            seoanalytics
            seoanalytics
            0
            3
            54

          • Few question about SEO
            Eric_Rohrback
            Eric_Rohrback
            0
            3
            109

          • Schema question
            Paddy_Moogan
            Paddy_Moogan
            0
            3
            256

          • Crawl budget
            BlueprintMarketing
            BlueprintMarketing
            2
            10
            4.8k

          • Subdomain Question
            Francisco_Meza
            Francisco_Meza
            0
            3
            130

          • Shtml question
            UnderRugSwept
            UnderRugSwept
            0
            2
            163

          • Architecture questions.
            CleverPhD
            CleverPhD
            0
            10
            1.1k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy