The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Is there an easier way from the server to prevent duplicate page content?

    Is there an easier way from the server to prevent duplicate page content?

    Technical SEO Issues
    38 8 1.8k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • SEOKeith
      SEOKeith last edited by

      Personally I would do the following:

      • Set rel="canonical" as Brent says below
      • 301 redirect the preferred URL, so if you are using www.example.com redirect example.com, that way if anyone points links at example.com "most" of the juice will pass over (this will probably fix the issue you have posted about)
      • Set the referred URL in Google web master tools

      If you are using  CMS like Wordpress rel="canonical" will probably already be taken care of for your website, you can check this by viewing the source or using SEO Moz's on-page keyword optimization tool.

      seosstec 2 Replies Last reply Reply Quote 1
      • brianhughes
        brianhughes last edited by

        Thank you Brent and kjay. Take a look at Ryan's answer, I think that is what I was shooting for. If I can eliminate the problem of an ambiguous URL at the server level then I will not need rel="canonical or 301/302. What do you guys think?

        AdoptionHelp SEOKeith Copstead 3 Replies Last reply Reply Quote 0
        • brianhughes
          brianhughes @AdoptionHelp last edited by

          Thank you Ryan, that is exactly what I expected the problem to be but really couldn't figure out how to address it or solve it. You explained it very well and I appreciate the suggested code to use as well. I should be able to figure it out from here.

          Thank you again!

          1 Reply Last reply Reply Quote 0
          • AdoptionHelp
            AdoptionHelp @brianhughes last edited by

            Rel Canonical is considered a best practice in SEO, so you should just always include it in your pages, even if they're the only copy of the content you know of. It will help prevent any scrapers from stealing your content down the road.

            And re: you're sorta right. Technically speaking, what we're doing with that htaccess code is 301 redirecting every URL, either to the www or non-www version. So say you go with my method anyone going to http://example.com just gets 301'd over to http://www.example.com

            1 Reply Last reply Reply Quote 1
            • SEOKeith
              SEOKeith @brianhughes last edited by

              I always use rel="canonical"

              1 Reply Last reply Reply Quote 1
              • Copstead
                Copstead @brianhughes last edited by

                You should set up the correct Canonicalization rewrites at the server level with IIS or .htaccess.  (Not sure which one you have).  If you know what type of sever you are on, then you can find all the correct rewrites.  (www, non www, lowercase, trailing slash / , etc.)

                For example, here is a great post if you have IIS. http://www.seomoz.org/blog/what-every-seo-should-know-about-iis

                And you should also use rel=canonical tags.

                1 Reply Last reply Reply Quote 1
                • brianhughes
                  brianhughes last edited by

                  Thanks a lot guys this is some great information. Let me get this straight.

                  Is solving this issue with the code below a bad practice?

                  <ifmodule mod_rewrite.c="">RewriteEngine on</ifmodule>

                  RewriteCond %{HTTP_HOST} !^www. [NC]
                  RewriteRule ^ http://www.%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

                  If it's not a bad practice and I implement the code to stop the issue, you are saying I should still use a rel=canonical tag to prevent scraping?

                  KeriMorgret SEOKeith 2 Replies Last reply Reply Quote 0
                  • KeriMorgret
                    KeriMorgret @brianhughes last edited by

                    Generally, if you can fix it with code, that tends to be a bit better than the canonical tag, from my understanding. I've emailed Dr. Pete and asked him to contribute to this thread as well, as he's an expert on canonical tags.

                    brianhughes Dr-Pete 3 Replies Last reply Reply Quote 1
                    • brianhughes
                      brianhughes @KeriMorgret last edited by

                      Thank you Keri, that's what I'm thinking but I want to make sure. Thank you for messaging Dr. Pete, I hope maybe he can expound on this.

                      1 Reply Last reply Reply Quote 0
                      • SEOKeith
                        SEOKeith @brianhughes last edited by

                        The rewrite rule above is not bad practice, it will fix the issue with your URL's

                        However it is good practice to use the rel=canonical tag on your site additionally to prevent any other duplicate content issues.

                        In short the rel=canonical tag tells Google which URL you wish to use, preventing Google from thinking you have duplicate content if multiple URL's exist for the same page.

                        brianhughes SEOKeith 8 Replies Last reply Reply Quote 1
                        • brianhughes
                          brianhughes @SEOKeith last edited by

                          Thank you SEOKeith! I definitely want to make sure I don't use any bad practices to fix this issue and thank you for clarifying that about the code.

                          So if I apply the code and fix the issue from the server by making the URL www.example.com

                          I would then add the rel=canonical tag to prevent scraping.

                          Would this be the correct URL to put in the tag?

                          1 Reply Last reply Reply Quote 0
                          • SEOKeith
                            SEOKeith @SEOKeith last edited by

                            Brian, if you 301 the example.com to www.example.com that will get rid of the duplicate URL issue server side. (this will resolve your current duplicate content issue).

                            Additionally I recommend you add the rel=canonical it will prevent other potential duplicate content issues that may arise and is considered good practice to implement.

                            The tag looks correct, note the space after the domain in quotes:

                            Are you using a CMS ?

                            1 Reply Last reply Reply Quote 1
                            • brianhughes
                              brianhughes @SEOKeith last edited by

                              Thank you SEOKeith, what would be the difference between using a 301 in the .htaccess verses the code Ryan suggested <ifmodule mod_rewrite.c="">?</ifmodule>

                              Also if I use the 301 redirect in the .htaccess would it cover this issue site wide?

                              Okay so the space needs to be there.

                              No, I don't use a CMS?

                              1 Reply Last reply Reply Quote 0
                              • SEOKeith
                                SEOKeith @SEOKeith last edited by

                                Brian, it's the same thing just a different method both 301.

                                No it would not cover the issues site wide only for the home page.

                                CMS = Content Management System (an example would be Wordpress or Drupal).

                                You should still do the rel="canonical" site wide (on each page).

                                All make sense ?

                                1 Reply Last reply Reply Quote 1
                                • brianhughes
                                  brianhughes @SEOKeith last edited by

                                  SEOKeith, the problem is sitewide, all 52 pages. I was hoping to solve the problem in the server and avoid coding each page. But from what I'm gathering is, even if I use the 301 redirect I should still add the rel="canonical" on each page to avoid scraping. This tells the SE that this page is the only page to index and crawl.

                                  Lol, sorry I didn't recognize the acronym. Yes, I have a site that is through Wordpress and one that is through Joomla. The one that I'm having issues with is not through a CMS though.

                                  1 Reply Last reply Reply Quote 0
                                  • SEOKeith
                                    SEOKeith @SEOKeith last edited by

                                    I recommend you update each page, note the rel canonical tag will be different for each page. And 50 pages should take you less than 15 mins 🙂

                                    1 Reply Last reply Reply Quote 1
                                    • brianhughes
                                      brianhughes @SEOKeith last edited by

                                      Thank you again SEOKeith, I understand what has to be done. I just wanted to make sure I was clear on what needed to be done. Yes, the rel canonical tag will reflect whatever the page is I'm adding it. 🙂 Since I didn't get the errors for it I never added it to my other sites; so now I have to it for all of them. Fun...

                                      1 Reply Last reply Reply Quote 0
                                      • Dr-Pete
                                        Dr-Pete last edited by

                                        Just to clarify, the rewrite that Ryan is proposing IS a 301-redirect (see the "R=301") - it's just one way to implement it. Done right, it can be used sitewide.

                                        It's perfectly viable to also use canonicals (and I definitely think they're great to have on the home-page, for example), but I think the 301 is more standard practice here. It's best for search crawlers AND visitors to see your canonical URL (www vs. non-www, whichever you choose). That leads people to link to the "proper" version, bookmark it, promote it on social, etc.

                                        Make sure, too, to use the canonical version internally. It's amazing how often people 301-redirect to "www." but then link to the non-www version internally, or vise-versa. Consistent signals are important.

                                        SEOKeith brianhughes 2 Replies Last reply Reply Quote 3
                                        • Dr-Pete
                                          Dr-Pete @KeriMorgret last edited by

                                          Expounding is what I do 🙂 Other people use different words for it...

                                          1 Reply Last reply Reply Quote 1
                                          • SEOKeith
                                            SEOKeith @SEOKeith last edited by

                                            Sorry I just read this again, the 301 will fix the URL issue site wide.

                                            1 Reply Last reply Reply Quote 1
                                            • 1
                                            • 2
                                            • 1 / 2
                                            • First post
                                              Last post
                                            • Thousands of 404-pages, duplicate content pages, temporary redirect
                                              posthumus
                                              posthumus
                                              0
                                              4
                                              262

                                            • When is Duplicate Content Duplicate Content
                                              AMHC
                                              AMHC
                                              0
                                              6
                                              169

                                            • Issue: Duplicate Page Content > Wordpress Comments Page
                                              DomainUltra
                                              DomainUltra
                                              0
                                              6
                                              1.3k

                                            • Why are pages linked with URL parameters showing up as separate pages with duplicate content?
                                              DeanAndrews
                                              DeanAndrews
                                              0
                                              4
                                              256

                                            • How to prevent duplicate content at a calendar page
                                              Dr-Pete
                                              Dr-Pete
                                              0
                                              8
                                              1.1k

                                            • How can I prevent duplicate content between www.page.com/ and www.page.com
                                              onlineexpression
                                              onlineexpression
                                              0
                                              4
                                              536

                                            • Duplicate Page Content Lists the same page twice?
                                              RyanKent
                                              RyanKent
                                              0
                                              2
                                              564

                                            • Duplicate content, canonical and seasonally changing of home page content
                                              sferrino
                                              sferrino
                                              0
                                              6
                                              784

                                            Get started with Moz Pro!

                                            Unlock the power of advanced SEO tools and data-driven insights.

                                            Start my free trial
                                            Products
                                            • Moz Pro
                                            • Moz Local
                                            • Moz API
                                            • Moz Data
                                            • STAT
                                            • Product Updates
                                            Moz Solutions
                                            • SMB Solutions
                                            • Agency Solutions
                                            • Enterprise Solutions
                                            • Digital Marketers
                                            Free SEO Tools
                                            • Domain Authority Checker
                                            • Link Explorer
                                            • Keyword Explorer
                                            • Competitive Research
                                            • Brand Authority Checker
                                            • Local Citation Checker
                                            • MozBar Extension
                                            • MozCast
                                            Resources
                                            • Blog
                                            • SEO Learning Center
                                            • Help Hub
                                            • Beginner's Guide to SEO
                                            • How-to Guides
                                            • Moz Academy
                                            • API Docs
                                            About Moz
                                            • About
                                            • Team
                                            • Careers
                                            • Contact
                                            Why Moz
                                            • Case Studies
                                            • Testimonials
                                            Get Involved
                                            • Become an Affiliate
                                            • MozCon
                                            • Webinars
                                            • Practical Marketer Series
                                            • MozPod
                                            Connect with us

                                            Contact the Help team

                                            Join our newsletter
                                            Moz logo
                                            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                                            • Accessibility
                                            • Terms of Use
                                            • Privacy