The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Best way for Google and Bing not to crawl my /en default english pages

    Best way for Google and Bing not to crawl my /en default english pages

    Intermediate & Advanced SEO
    1 1 126
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Shawn124
      Shawn124 last edited by

      Hi Guys,

      I just transferred my old site to a new one and now have sub folder TLD's.  My default pages from the front end and sitemap don't show /en after www.mysite.com.  The only translation i have is in spanish where Google will crawl www.mysite.com/es (spanish).

      1. On the SERPS of Google and Bing, every url that is crawled, shows the extra "/en" in my TLD.  I find that very weird considering there is no physical /en in my urls.  When i select the link it automatically redirects to it's default and natural page (no /en).  All canonical tags do not show /en either, ONLY the SERPS. Should robots.txt be updated to "disallow /en"?

      2. While i did a site transfer, we have altered some of the category url's in our domain.  So we've had a lot of 301 redirects, but while searching specific keywords in the SERPS, the #1 ranked url shows up as our old url that redirects to a 404 page, and our newly created url shows up as #2 that goes to the correct page.  Is there anyway to tell Google to stop showing our old url's in the SERP's?  And would the "Fetch as Google" option in GWT be a great option to upload all of my url's so Google bots can crawl the right pages only?

      Direct Message me if you want real examples.  THank you so much!

      1 Reply Last reply Reply Quote 0
      • 1 / 1
      • First post
        Last post
      • Glossary/Terms Page - What is the best way?
        0
        1
        33

      • How to 301 Redirect /page.php to /page, after a RewriteRule has already made /page.php accessible by /page (Getting errors)
        DirkC
        DirkC
        0
        2
        435

      • What's the best way to check Google search results for all pages NOT linking to a domain?
        Cyrus-Shepard
        Cyrus-Shepard
        0
        7
        163

      • Does Google make continued attempts to crawl an old page one it has followed a 301 to the new page?
        CleverPhD
        CleverPhD
        0
        2
        126

      • Are links to on-page content crawled / have any effect on page rank?
        Vahe.Arabian
        Vahe.Arabian
        0
        4
        320

      • Is there any delay between crawling a page by google and displaying of the ratings in rich snippet of the results in google?
        NEWCRAFT
        NEWCRAFT
        0
        3
        545

      • Best way to de-index content from Google and not Bing?
        ShaMenz
        ShaMenz
        0
        5
        1.4k

      • What are the different tactics for getting ranked/ included in Google finance searches such as http://www.google.com/finance/company_news?q=NASDAQ:ADBE
        joemascaro
        joemascaro
        0
        7
        1.2k

      Get started with Moz Pro!

      Unlock the power of advanced SEO tools and data-driven insights.

      Start my free trial
      Products
      • Moz Pro
      • Moz Local
      • Moz API
      • Moz Data
      • STAT
      • Product Updates
      Moz Solutions
      • SMB Solutions
      • Agency Solutions
      • Enterprise Solutions
      • Digital Marketers
      Free SEO Tools
      • Domain Authority Checker
      • Link Explorer
      • Keyword Explorer
      • Competitive Research
      • Brand Authority Checker
      • Local Citation Checker
      • MozBar Extension
      • MozCast
      Resources
      • Blog
      • SEO Learning Center
      • Help Hub
      • Beginner's Guide to SEO
      • How-to Guides
      • Moz Academy
      • API Docs
      About Moz
      • About
      • Team
      • Careers
      • Contact
      Why Moz
      • Case Studies
      • Testimonials
      Get Involved
      • Become an Affiliate
      • MozCon
      • Webinars
      • Practical Marketer Series
      • MozPod
      Connect with us

      Contact the Help team

      Moz logo
      © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
      • Accessibility
      • Terms of Use
      • Privacy