Robots.txt Question

[davidangotti]

In the past, I had blocked a section of my site (i.e. domain.com/store/) by placing the following in my robots.txt file: "Disallow: /store/" Now, I would like the store to be indexed and included in the search results. I have removed the "Disallow: /store/" from the robots.txt file, but approximately one week later a Google search for the URL produces the following meta description in the search results: "A description for this result is not available because of this site's robots.txt – learn more"

Is there anything else I need to do to speed up the process of getting this section of the site indexed?

[seohive-222720]

David, If I am correct this is an old message sitting in the index.  Give it another week or so and I am sure this message will vanish.  I had this with one of my sites that I went live with but forget to allow in the robots.txt file.

shivun

[davidangotti]

Thanks. I figured this was the case, but was not sure if I was missing any "best practices" about getting the previously blocked URL included faster.

[ThompsonPaul]

To help speed up the process of getting re-included, use the "Fetch as Googlebot" and "Fetch as Bingbot" tools in Webmaster Tools for a page in the blocked section - this significantly helps jumpstart indexing of pages.Once you see a successful Fetch status, click Submit to Index, and then specify  to submit URL and all linked pages.

In addition

• make certain your new pages are listed in your sitmap.xml file, and then resubmit the sitemap to the search engines using Google and Bing Webmaster Tools
• make sure your own internal pages (especially a few strong ones) link to the newly unblocked content
• see if you can get a couple good new incoming links to some of the pages in the new section - even if they're no-follow, they can help guide the crawlers to the newly available pages

Essentially you're trying to give the SEs as many hints as possible that there are new pages to crawl and hopefully index.

Paul

[edited for additional clarity]

[ThompsonPaul]

Thanks for the "Good Answer" flag, David! I reformatted & added a little extra info to make the process a little clearer.

Paul