The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Crawl Errors from URL Parameter

    Crawl Errors from URL Parameter

    Moz Tools
    2 2 275
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • WrightIMC
      WrightIMC last edited by

      Hello,

      I am having this issue within SEOmoz's Crawl Diagnosis report.  There are a lot of crawl errors happening with pages associated with /login.

      I will see site.com/login?r=http://.... and have several duplicate content issues associated with those urls.

      Seeing this, I checked WMT to see if the Google crawler was showing this error as well.  It wasn't.

      So what I ended doing was going to the robots.txt and disallowing rogerbot.

      It looks like this:

      User-agent: rogerbot

      Disallow:/login

      However, SEOmoz has crawled again and it still picking up on those URLs.   Any ideas on how to fix?

      Thanks!

      1 Reply Last reply Reply Quote 0
      • JoelDay
        JoelDay last edited by

        Hi Tony,

        I need more information from you in order to check this out. I'm going to send you a support ticket to reply to.

        Thanks,
        Joel.

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Does anyone know the linking of hashtags on Wix sites does it negatively or postively impact SEO. It is coming up as an error in site crawls 'Pages with 404 errors' Anyone got any experience please?
          Nozzle
          Nozzle
          0
          5
          82

        • Site Crawl 4xx Errors?
          mostcg
          mostcg
          0
          3
          189

        • Could my Crawl Error Report be wrong?
          Brando16
          Brando16
          0
          7
          291

        • Site Crawl Error
          nopsts
          nopsts
          0
          5
          434

        • Since July 1, we've had a HUGE jump in errors on our weekly crawl. We don't think anything has changed on our website. Has MOZ changed something that would account for a large leap in duplicate content and duplicate title errors?
          KristyFord
          KristyFord
          0
          3
          77

        • Dot Net Nuke generating long URL showing up as crawl errors!
          benjaminspak
          benjaminspak
          0
          3
          398

        • Crawl Diagnostics Error Spike
          Brother22
          Brother22
          1
          14
          1.1k

        • Dismiss crawl diagnostics error
          MaratM
          MaratM
          0
          3
          889

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy