The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Can Search engines crawl this page

    Can Search engines crawl this page

    Technical SEO Issues
    4 3 972
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • imrubbish
      imrubbish last edited by

      Hi guys,

      To put a long story short we have had to make a copy of our site and put it on another domain so in essence there are 2 copies of our site on the web.What we have done is put a username and password on the homepage - http://www.ughhwiki.co.uk/ now i just want to be 100% sure that the search engines cannot crawl this?

      Thank you

      Jon

      1 Reply Last reply Reply Quote 0
      • CPU
        CPU last edited by

        Hi Jon

        I wasn't able to access the page. Is the password just on the homepage though? If I was you I would test some other pages as you wouldn't want some one coming into the site via a search engine result if you are trying to keep everyone locked out at the moment.

        Hope this helps.

        imrubbish 1 Reply Last reply Reply Quote 0
        • imrubbish
          imrubbish @CPU last edited by

          Hi Lewis,

          Thanks for the reply. We are trying to prevent this site from being able to be crawled by anyone, search engines and visitors. The login page is hopefully able to prevent google from seeing any of our content while we edit it and then put it on our actual live url. Does that make sense?

          1 Reply Last reply Reply Quote 0
          • AdamThompson
            AdamThompson last edited by

            Google should not be able to crawl that page as long as it is password protected.

            However, Google already has it indexed and password protecting it may not induce Google to de-index the page.

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • My pages are being crawled, but not indexed according to Search Console
              DonnaDuncan
              DonnaDuncan
              0
              4
              64

            • Search Console Indexed Page Count vs Site:Search Operator page count
              benesmartin
              benesmartin
              0
              2
              78

            • Can Googlebot crawl the content on this page?
              danatanseo
              danatanseo
              0
              7
              156

            • What can i do to move my site up the search engines
              ClaireH-184886
              ClaireH-184886
              0
              3
              219

            • Hello every one please give me idea about offpage techniques for keyword ranking for first page on top 5 all search engine ,and it must be durable for long time. If i will not use on page optimization?
              MarkScully
              MarkScully
              0
              9
              499

            • How can the search engines can crawl my java script generated web pages
              randfish
              randfish
              0
              2
              1.3k

            • Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
              Nobody1560986989723
              Nobody1560986989723
              0
              5
              422

            • Advice on display this content on my page for search engines
              EricaMcGillivray
              EricaMcGillivray
              0
              6
              537

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy