The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. SEOMoz Crawl Diagnostic indicates duplicate page content for home page?

    SEOMoz Crawl Diagnostic indicates duplicate page content for home page?

    Technical SEO Issues
    3 2 607
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Linesides
      Linesides last edited by

      My first SEOMoz Crawl Diagnostic report for my website indicates duplicate page content for my home page. It lists the home page URL Page Title and URL twice.

      How do I go about diagnosing this?

      Is the problem related to the following code that is in my .htaccess file? (The purpose of the code was to redirect any non "www" backlink referrals to the "www" version of the domain.)

      RewriteCond %{HTTP_HOST} ^whatever.com [NC]
      RewriteRule ^(.*)$ http://www.whatever.com/$1 [L,R=301]

      Should I get rid of the "http" reference in the second line?

      Related to this is a notice in the "Crawl Notices Found" -- "301 Permanent redirect" which shows my home page title as "http://whatever.com" and shows the redirect address as http://http://www.whatever.com/

      I'm guessing this problem is again related to the redirect code I'm using.

      Also...

      The report indicates duplicate content for those links that have different parameters added to the URL i.e. http://www.whatever.com?marker=Blah Blah&markerzoom=13

      If I set up a canonical reference for the page, will this fix this?

      Thank you.

      1 Reply Last reply Reply Quote 0
      • KeriMorgret
        KeriMorgret last edited by

        Hi Perry,

        In your browser, are you seeing things redirect to the double http? I think there's a bug in the crawl tools that's causing some false errors right now. Before you go work on the redirect file, could you send an email to help@seomoz.org to first make sure we're not the ones that messed up?

        Thanks!

        1 Reply Last reply Reply Quote 0
        • Linesides
          Linesides last edited by

          I contacted the help desk as instructed an was told:

          "I took a look at the campaign and it looks like our crawler can't parse the 301 redirect you have in place on the main page. The reason for this is the redirect in place, adds two https when rogerbot tries to crawl through it. Roger can’t parse the redirect as is, but it can identify it (as it did in your notice’s report on the crawl diagnostics page). This isn't a problem for browsers since they are made to ignore redirects of this nature. Crawlers on the other hand have a strict code to follow and can't follow redirects like that. When I load up your site [mywebsite.com] right now, it redirects to www.[mywebsite].com. Try creating a new campaign under the domain you are redirecting to, this should clear any issues up."

          And so I did that and it looks like that worked after the new crawl, however I then set up another campaign for another website I manage being sure to use the "www" in front of the domain and got the same problem again -- the home page appears twice as duplicate content.

          So I'm back to asking my primary question again: What is the definitive redirect code to use to convert a non "www" request to a "www" request? The same redirect code mentioned in my first post is being used on all of my sites.

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Site Crawl -> Duplicate Page Content -> Same pages showing up with duplicates that are not
            davebuts
            davebuts
            0
            2
            72

          • Home Page Blog Snippets - Duplicate Content Help?
            DirkC
            DirkC
            0
            7
            683

          • Crawl Diagnostics and Duplicate Page Title
            iskq
            iskq
            0
            5
            178

          • Container Page/Content Page Duplicate Content
            ProductiveEdge
            ProductiveEdge
            1
            4
            194

          • I am trying to correct error report of duplicate page content. However I am unable to find in over 100 blogs the page which contains similar content to the page SEOmoz reported as having similar content is my only option to just dlete the blog page?
            evolvingSEO
            evolvingSEO
            0
            4
            344

          • How do I fix duplicate content with the home page?
            FrontlineMobility
            FrontlineMobility
            0
            2
            406

          • Duplicate page content errors in SEOmoz
            AlanMosley
            AlanMosley
            0
            4
            612

          • Duplicate Content Home Page
            wissamdandan
            wissamdandan
            0
            10
            1.3k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy