The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Best way for Google and Bing not to crawl my /en default english pages

    Best way for Google and Bing not to crawl my /en default english pages

    Intermediate & Advanced SEO
    1 1 126
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Shawn124
      Shawn124 last edited by

      Hi Guys,

      I just transferred my old site to a new one and now have sub folder TLD's.  My default pages from the front end and sitemap don't show /en after www.mysite.com.  The only translation i have is in spanish where Google will crawl www.mysite.com/es (spanish).

      1. On the SERPS of Google and Bing, every url that is crawled, shows the extra "/en" in my TLD.  I find that very weird considering there is no physical /en in my urls.  When i select the link it automatically redirects to it's default and natural page (no /en).  All canonical tags do not show /en either, ONLY the SERPS. Should robots.txt be updated to "disallow /en"?

      2. While i did a site transfer, we have altered some of the category url's in our domain.  So we've had a lot of 301 redirects, but while searching specific keywords in the SERPS, the #1 ranked url shows up as our old url that redirects to a 404 page, and our newly created url shows up as #2 that goes to the correct page.  Is there anyway to tell Google to stop showing our old url's in the SERP's?  And would the "Fetch as Google" option in GWT be a great option to upload all of my url's so Google bots can crawl the right pages only?

      Direct Message me if you want real examples.  THank you so much!

      1 Reply Last reply Reply Quote 0
      • 1 / 1
      • First post
        Last post
      • Best way to link to 1000 city landing pages from index page in a way that google follows/crawls these links (without building country pages)?
        lcourse
        lcourse
        0
        7
        54

      • Prioritise a page in Google/why is a well-optimised page not ranking
        ChrisAshton
        ChrisAshton
        0
        2
        88

      • We are switching our CMS local pages from a subdomain approach to a subfolder approach. What's the best way to handle this? Should we redirect every local subdomain page to its new subfolder page?
        Ray-pp
        Ray-pp
        0
        4
        123

      • What's the best way to check Google search results for all pages NOT linking to a domain?
        Cyrus-Shepard
        Cyrus-Shepard
        0
        7
        163

      • Are links to on-page content crawled / have any effect on page rank?
        Vahe.Arabian
        Vahe.Arabian
        0
        4
        320

      • Is there any delay between crawling a page by google and displaying of the ratings in rich snippet of the results in google?
        NEWCRAFT
        NEWCRAFT
        0
        3
        545

      • Best way to de-index content from Google and not Bing?
        ShaMenz
        ShaMenz
        0
        5
        1.4k

      • What are the different tactics for getting ranked/ included in Google finance searches such as http://www.google.com/finance/company_news?q=NASDAQ:ADBE
        joemascaro
        joemascaro
        0
        7
        1.2k

      Get started with Moz Pro!

      Unlock the power of advanced SEO tools and data-driven insights.

      Start my free trial
      Products
      • Moz Pro
      • Moz Local
      • Moz API
      • Moz Data
      • STAT
      • Product Updates
      Moz Solutions
      • SMB Solutions
      • Agency Solutions
      • Enterprise Solutions
      • Digital Marketers
      Free SEO Tools
      • Domain Authority Checker
      • Link Explorer
      • Keyword Explorer
      • Competitive Research
      • Brand Authority Checker
      • Local Citation Checker
      • MozBar Extension
      • MozCast
      Resources
      • Blog
      • SEO Learning Center
      • Help Hub
      • Beginner's Guide to SEO
      • How-to Guides
      • Moz Academy
      • API Docs
      About Moz
      • About
      • Team
      • Careers
      • Contact
      Why Moz
      • Case Studies
      • Testimonials
      Get Involved
      • Become an Affiliate
      • MozCon
      • Webinars
      • Practical Marketer Series
      • MozPod
      Connect with us

      Contact the Help team

      Join our newsletter
      Moz logo
      © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
      • Accessibility
      • Terms of Use
      • Privacy