The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. How to Remove /feed URLs from Google's Index

    How to Remove /feed URLs from Google's Index

    Technical SEO Issues
    3 1 5.2k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M_D_Golden_Peak
      M_D_Golden_Peak last edited by

      Hey everyone, I have an issue with RSS /feed URLs being indexed by Google for some of our Wordpress sites. Have a look at this Google query, and click to show omitted search results. You'll see we have 500+ /feed URLs indexed by Google, for our many category pages/etc. Here is one of the example URLs: http://www.howdesign.com/design-creativity/fonts-typography/letterforms/attachment/gilhelveticatrade/feed/. Based on this content/code of the XML page, it looks like Wordpress is generating these:

      <generator>http://wordpress.org/?v=3.5.2</generator>

      Any idea how to get them out of Google's index without 301 redirecting them? We need the Wordpress-generated RSS feeds to work for various uses.

      My first two thoughts are trying to work with our Development team to see if we can get a "noindex" meta robots tag on the pages, by they are dynamically-generated pages...so I'm not sure if that will be possible. Or, perhaps we can add a "feed" paramater to GWT "URL Parameters" section...but I don't want to limit Google from crawling these again...I figure I need Google to crawl them and see some code that says to get the pages out of their index...and THEN not crawl the pages anymore.

      I don't think the "Remove URL" feature in GWT will work, since that tool only removes URLs from the search results, not the actual Google index.

      FWIW, this site is using the Yoast plugin. We set every page type to "noindex" except for the homepage, Posts, Pages and Categories. We have other sites on Yoast that do not have any /feed URLs indexed by Google at all.

      Side note, the /robots.txt file was previously blocking crawling of the /feed URLs on this site, which is why you'll see that note in the Google SERPs when you click on the query link given in the first paragraph.

      1 Reply Last reply Reply Quote 0
      • M_D_Golden_Peak
        M_D_Golden_Peak last edited by

        I believe I found the solution: implement an x-robots-tag into the HTTP header of the various feed URLs. But, I need some help with creating the code to place in my .htaccess file. Any takers?

        1 Reply Last reply Reply Quote 0
        • M_D_Golden_Peak
          M_D_Golden_Peak last edited by

          I tried many different htaccess file codings (such as recommended here), but they didn't work. Had to succumb to using the outdated Meta Robots plugin by Yoast, which can add the "noindex" code to the http header of /feed/ URLs. But, at least it's a solution: http://wordpress.org/plugins/robots-meta/. Hopefully this helps someone else.

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Strange URL's for client's site
            everestagency
            everestagency
            0
            3
            456

          • Sitemap url's not being indexed
            GreenStone
            GreenStone
            0
            7
            158

          • If Google's index contains multiple URLs for my homepage, does that mean the canonical tag is not working?
            Dr-Pete
            Dr-Pete
            0
            4
            217

          • Google indexing less url's then containded in my sitemap.xml
            Juist
            Juist
            0
            7
            3.4k

          • Can JavaScrip affect Google's index/ranking?
            echo1
            echo1
            0
            7
            1.7k

          • How to fix and test Google's indexing / caching problem
            ShaMenz
            ShaMenz
            0
            2
            972

          • Removing a site from Google's index
            issuebasedmedia
            issuebasedmedia
            0
            13
            976

          • Why is this url showing as "not crawled" on opensiteexplorer, but still showing up in Google's index?
            KeriMorgret
            KeriMorgret
            0
            2
            617

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy