The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Best way for Google and Bing not to crawl my /en default english pages

    Best way for Google and Bing not to crawl my /en default english pages

    Intermediate & Advanced SEO
    1 1 126
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Shawn124
      Shawn124 last edited by

      Hi Guys,

      I just transferred my old site to a new one and now have sub folder TLD's.  My default pages from the front end and sitemap don't show /en after www.mysite.com.  The only translation i have is in spanish where Google will crawl www.mysite.com/es (spanish).

      1. On the SERPS of Google and Bing, every url that is crawled, shows the extra "/en" in my TLD.  I find that very weird considering there is no physical /en in my urls.  When i select the link it automatically redirects to it's default and natural page (no /en).  All canonical tags do not show /en either, ONLY the SERPS. Should robots.txt be updated to "disallow /en"?

      2. While i did a site transfer, we have altered some of the category url's in our domain.  So we've had a lot of 301 redirects, but while searching specific keywords in the SERPS, the #1 ranked url shows up as our old url that redirects to a 404 page, and our newly created url shows up as #2 that goes to the correct page.  Is there anyway to tell Google to stop showing our old url's in the SERP's?  And would the "Fetch as Google" option in GWT be a great option to upload all of my url's so Google bots can crawl the right pages only?

      Direct Message me if you want real examples.  THank you so much!

      1 Reply Last reply Reply Quote 0
      • 1 / 1
      • First post
        Last post
      • Will google be able to crawl all of the pages given that the pages displayed or the info on a page varies according to the city of a user?
        0
        3
        17

      • Google crawling 200 page site thousands of times/day. Why?
        brettmandoes
        brettmandoes
        0
        7
        90

      • Does Google give weight to the default measurement units (metric / imperial) on pages?
        katemorris
        katemorris
        0
        5
        496

      • We are switching our CMS local pages from a subdomain approach to a subfolder approach. What's the best way to handle this? Should we redirect every local subdomain page to its new subfolder page?
        Ray-pp
        Ray-pp
        0
        4
        123

      • Does Google make continued attempts to crawl an old page one it has followed a 301 to the new page?
        CleverPhD
        CleverPhD
        0
        2
        126

      • What is the best way to optimize/setup a teaser "coming soon" page for a new product launch?
        KeriMorgret
        KeriMorgret
        0
        7
        743

      • Our site is recieving traffic for both .com/page and .com/page/ with the trailing slash.
        Profero
        Profero
        0
        5
        670

      • What are the different tactics for getting ranked/ included in Google finance searches such as http://www.google.com/finance/company_news?q=NASDAQ:ADBE
        joemascaro
        joemascaro
        0
        7
        1.2k

      Get started with Moz Pro!

      Unlock the power of advanced SEO tools and data-driven insights.

      Start my free trial
      Products
      • Moz Pro
      • Moz Local
      • Moz API
      • Moz Data
      • STAT
      • Product Updates
      Moz Solutions
      • SMB Solutions
      • Agency Solutions
      • Enterprise Solutions
      • Digital Marketers
      Free SEO Tools
      • Domain Authority Checker
      • Link Explorer
      • Keyword Explorer
      • Competitive Research
      • Brand Authority Checker
      • Local Citation Checker
      • MozBar Extension
      • MozCast
      Resources
      • Blog
      • SEO Learning Center
      • Help Hub
      • Beginner's Guide to SEO
      • How-to Guides
      • Moz Academy
      • API Docs
      About Moz
      • About
      • Team
      • Careers
      • Contact
      Why Moz
      • Case Studies
      • Testimonials
      Get Involved
      • Become an Affiliate
      • MozCon
      • Webinars
      • Practical Marketer Series
      • MozPod
      Connect with us

      Contact the Help team

      Moz logo
      © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
      • Accessibility
      • Terms of Use
      • Privacy