The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Duplicate Content From Indexing of non- File Extension Page

    Duplicate Content From Indexing of non- File Extension Page

    Intermediate & Advanced SEO
    22 4 1.3k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • WebbyNabler
      WebbyNabler @NakulGoyal last edited by

      No it's not a wordpress, it was created with Dreamweaver.  I didn't make sample and sample.html same page, but google is treating it that way....  I have implemented the 301, so I guess I just have to wait for a crawl

      Mike.Goracke 1 Reply Last reply Reply Quote 0
      • Mike.Goracke
        Mike.Goracke @WebbyNabler last edited by

        "I accidentally manually submitted the url to google and manually in submitted it to index and that when this issue began...."

        It sounds like you accidently added this URL to the index. You can follow the procedure outlined below to request Google remove the specific URL from the index:

        https://support.google.com/webmasters/bin/answer.py?hl=en&answer=59819

        I checked your site's structure using Screaming Frog and it does not appear that you are linking to any non-.html versions. If I perform a scan using one of your non-.html pages, it appears that it only links to itself.

        Since you have the 301 redirect in place, you can choose to wait it out and Google should correct things eventually; otherwise, requesting Google remove the URL is a faster... PERMANENT process.

        Good luck.

        Mike

        WebbyNabler 1 Reply Last reply Reply Quote 1
        • WebbyNabler
          WebbyNabler @Mike.Goracke last edited by

          Thanks Mike, you are awesome!  I actually was thinking to do that, but I was concerned that it might have some larger implications?

          I also just resubmitted a sitemap so hopefully that "might" speed up the crawl process...

          Thanks again!

          1 Reply Last reply Reply Quote 0
          • TakeshiYoung
            TakeshiYoung @WebbyNabler last edited by

            You repeat this code a few times, maybe that's the problem? Pretty sure you only need it once:

            RewriteEngine On
            Options +FollowSymlinks
            RewriteBase /

            The line:

            RewriteEngine On

            Also only needs to be included once in an htaccess file. You may want to remove all the other instances.

            Try adding this code at the very top, after the first "RewriteEngine On":

            RewriteCond %{REQUEST_URI} ! .html$
            RewriteCond %{REQUEST_URI} ! /$
            RewriteRule ^(.*)$ $1.html

            WebbyNabler 1 Reply Last reply Reply Quote 0
            • WebbyNabler
              WebbyNabler @TakeshiYoung last edited by

              I get a server error when I do this?  Sooo confused...  Here is the htaccess changes I made.  FYI...I have removed the code you told me to put in there temporarily so the site's not down.  I attached the server error screenshot too...

              Options +FollowSymlinks
              RewriteEngine On

              RewriteCond %{REQUEST_URI} ! .html$
              RewriteCond %{REQUEST_URI} ! /$
              RewriteRule ^(.*)$ $1.html

              RewriteCond %{HTTP_HOST} ^hanneganconstructionllc.com [NC]
              RewriteRule ^(.*)$ http://hanneganremodeling.com/$1 [L,R=301]

              RewriteCond %{HTTP_HOST} ^www.hanneganconstructionllc.com [NC]
              RewriteRule ^(.*)$ http://hanneganremodeling.com/$1 [L,R=301]

              RewriteCond %{HTTP_HOST} ^hremodeling.com [NC]
              RewriteRule ^(.*)$ http://hanneganremodeling.com/$1 [L,R=301]

              RewriteCond %{HTTP_HOST} ^www.hremodeling.com [NC]
              RewriteRule ^(.*)$ http://hanneganremodeling.com/$1 [L,R=301]

              RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index.html\ HTTP/
              RewriteRule ^index.html$ http://www.hanneganremodeling.com/ [R=301,L]

              RewriteBase /
              RewriteCond %{HTTP_HOST} ^hanneganremodeling.com$ [NC]
              RewriteRule ^(.*)$ http://www.hanneganremodeling.com/$1 [R=301,L]

              m9Kq8yJ

              Mike.Goracke 1 Reply Last reply Reply Quote 0
              • Mike.Goracke
                Mike.Goracke @WebbyNabler last edited by

                Maybe give this a whirl:

                If URL does not contain a period or end with a slash

                RewriteCond %{REQUEST_URI} !(.|/$)

                append .html to requested URL

                RewriteRule (.*) /$1.html [L]

                WebbyNabler 2 Replies Last reply Reply Quote 0
                • WebbyNabler
                  WebbyNabler @Mike.Goracke last edited by

                  Nope.  Didn't work.  I am seriously about to lose my mind with this....

                  TakeshiYoung 1 Reply Last reply Reply Quote 0
                  • WebbyNabler
                    WebbyNabler @Mike.Goracke last edited by

                    Oh, and my site auditor is seeing it as a directory with a file in it???  Ugghhh....

                    1 Reply Last reply Reply Quote 0
                    • TakeshiYoung
                      TakeshiYoung @WebbyNabler last edited by

                      Try my code without all the other redirects, and see if it works. If it does, then add back the other redirects one by one until everything works.

                      WebbyNabler 1 Reply Last reply Reply Quote 0
                      • WebbyNabler
                        WebbyNabler @TakeshiYoung last edited by

                        That caused the same "500 Internal Server Error" .......   😞

                        TakeshiYoung 1 Reply Last reply Reply Quote 0
                        • TakeshiYoung
                          TakeshiYoung @WebbyNabler last edited by

                          Try:

                          RewriteCond %{REQUEST_FILENAME} !-d
                          RewriteRule ^([^.]*[^./])$ /$1.html [R=301,L]

                          Mike.Goracke WebbyNabler 2 Replies Last reply Reply Quote 0
                          • Mike.Goracke
                            Mike.Goracke @TakeshiYoung last edited by

                            If that does not work, give this a whirl:

                            RewriteCond %{REQUEST_URI} !\.[a-zA-Z0-9]{3,4}

                            RewriteCond %{REQUEST_URI} !/$

                            RewriteRule ^(.*)$ $1.html

                            WebbyNabler 1 Reply Last reply Reply Quote 0
                            • WebbyNabler
                              WebbyNabler @Mike.Goracke last edited by

                              Nope. 😞

                              1 Reply Last reply Reply Quote 0
                              • WebbyNabler
                                WebbyNabler @TakeshiYoung last edited by

                                Nope. .....good lord....

                                Mike.Goracke 1 Reply Last reply Reply Quote 0
                                • Mike.Goracke
                                  Mike.Goracke @WebbyNabler last edited by

                                  LAME! You may just want to let the 301 redirect you have in place take its course or remove the URL from Google's index since it was added by mistake anyway.

                                  Mike

                                  WebbyNabler 1 Reply Last reply Reply Quote 0
                                  • WebbyNabler
                                    WebbyNabler @Mike.Goracke last edited by

                                    Yeah I looked further into the URL removal, but I guess technically I did not meet the criteria....and honestly I am fearful other potential implications of removal....I guess I will just have to wait for the 301 to ick in.  I just cant believe there is not a simple .htaccess code to cause all URL's to show the .html extension.  I mean it is a simple thing to implement the reverse and have the extension dropped...I mean....good lord...

                                    Thanks for all your help though Mike, I truly appreciate the efforts!

                                    1 Reply Last reply Reply Quote 0
                                    • 1
                                    • 2
                                    • 1 / 2
                                    • First post
                                      Last post
                                    • Glossary index and individual pages create duplicate content. How much might this hurt me?
                                      LeadSEOlogist
                                      LeadSEOlogist
                                      0
                                      9
                                      199

                                    • Our client's web property recently switched over to secure pages (https) however there non secure pages (http) are still being indexed in Google. Should we request in GWMT to have the non secure pages deindexed?
                                      N1ghteyes
                                      N1ghteyes
                                      0
                                      3
                                      128

                                    • Duplicate content - how to diagnose duplicate content from another domain before publishing pages?
                                      Chemometec
                                      Chemometec
                                      0
                                      7
                                      141

                                    • Duplicate Content: Is a product feed/page rolled out across subdomains deemed duplicate content?
                                      danwebman
                                      danwebman
                                      0
                                      4
                                      146

                                    • Indexing non-indexed content and Google crawlers
                                      CleverPhD
                                      CleverPhD
                                      0
                                      8
                                      508

                                    • Duplicate page content and duplicate pate title
                                      RyanKent
                                      RyanKent
                                      0
                                      3
                                      555

                                    • Duplicate content on index.htm page
                                      Fatfreddy
                                      Fatfreddy
                                      0
                                      5
                                      765

                                    • Pop Up Pages Being Indexed, Seen As Duplicate Content
                                      Tompt
                                      Tompt
                                      0
                                      2
                                      1.0k

                                    Get started with Moz Pro!

                                    Unlock the power of advanced SEO tools and data-driven insights.

                                    Start my free trial
                                    Products
                                    • Moz Pro
                                    • Moz Local
                                    • Moz API
                                    • Moz Data
                                    • STAT
                                    • Product Updates
                                    Moz Solutions
                                    • SMB Solutions
                                    • Agency Solutions
                                    • Enterprise Solutions
                                    • Digital Marketers
                                    Free SEO Tools
                                    • Domain Authority Checker
                                    • Link Explorer
                                    • Keyword Explorer
                                    • Competitive Research
                                    • Brand Authority Checker
                                    • Local Citation Checker
                                    • MozBar Extension
                                    • MozCast
                                    Resources
                                    • Blog
                                    • SEO Learning Center
                                    • Help Hub
                                    • Beginner's Guide to SEO
                                    • How-to Guides
                                    • Moz Academy
                                    • API Docs
                                    About Moz
                                    • About
                                    • Team
                                    • Careers
                                    • Contact
                                    Why Moz
                                    • Case Studies
                                    • Testimonials
                                    Get Involved
                                    • Become an Affiliate
                                    • MozCon
                                    • Webinars
                                    • Practical Marketer Series
                                    • MozPod
                                    Connect with us

                                    Contact the Help team

                                    Join our newsletter
                                    Moz logo
                                    © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                                    • Accessibility
                                    • Terms of Use
                                    • Privacy