The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Crawl Errors from URL Parameter

    Crawl Errors from URL Parameter

    Moz Tools
    2 2 275
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • WrightIMC
      WrightIMC last edited by

      Hello,

      I am having this issue within SEOmoz's Crawl Diagnosis report.  There are a lot of crawl errors happening with pages associated with /login.

      I will see site.com/login?r=http://.... and have several duplicate content issues associated with those urls.

      Seeing this, I checked WMT to see if the Google crawler was showing this error as well.  It wasn't.

      So what I ended doing was going to the robots.txt and disallowing rogerbot.

      It looks like this:

      User-agent: rogerbot

      Disallow:/login

      However, SEOmoz has crawled again and it still picking up on those URLs.   Any ideas on how to fix?

      Thanks!

      1 Reply Last reply Reply Quote 0
      • JoelDay
        JoelDay last edited by

        Hi Tony,

        I need more information from you in order to check this out. I'm going to send you a support ticket to reply to.

        Thanks,
        Joel.

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Does anyone know the linking of hashtags on Wix sites does it negatively or postively impact SEO. It is coming up as an error in site crawls 'Pages with 404 errors' Anyone got any experience please?
          Nozzle
          Nozzle
          0
          5
          82

        • Could my Crawl Error Report be wrong?
          Brando16
          Brando16
          0
          7
          291

        • Moz Crawl Test error
          donford
          donford
          0
          2
          363

        • Site Crawl Error
          nopsts
          nopsts
          0
          5
          434

        • Seo moz has only crawled 2 pages of my site. Ive been notified of a 403 error and need an answer as to why my pages are not being crawled?
          nitro-digital
          nitro-digital
          0
          9
          319

        • SEOmoz showing crawl errors but webmastertools says no errors, need help!
          bobsnowzell
          bobsnowzell
          0
          5
          399

        • SEOmoz crawl error questions
          Malarowski
          Malarowski
          0
          4
          528

        • Errors on my Crawl Diagnostics
          rhutchings
          rhutchings
          0
          2
          645

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy