Negative impact on crawling after upload robots.txt file on HTTPS pages

[CommercePundit]

I experienced negative impact on crawling after upload robots.txt file on HTTPS pages. You can find out both URLs as follow.

Robots.txt File for HTTP: http://www.vistastores.com/robots.txt

Robots.txt File for HTTPS: https://www.vistastores.com/robots.txt

I have disallowed all crawlers for HTTPS pages with following syntax.

User-agent: *
Disallow: /

Does it matter for that? If I have done any thing wrong so give me more idea to fix this issue.

[ShaMenz]

Hi CP,

If you wish to use robots.txt to block crawlers, then your two robots.txt files should be as follows:

For your http protocol (http://vistastores.com/robots.txt

User-agent: *
Allow: /

For the https protocol (https://vistastores.com/robots.txt

User-agent: *
Disallow: /

Personally, I prefer to use the noindex meta tag for page blocking because it is a more reliable way of ensuring that the pages are not indexed.
(Never try to use both at once)

This link explains the difference between the two:
[Google Webmaster Tools Help.](http://www.google.com/support/webmasters/bin/answer.py?answer=35302 "Robots blocking crawlers")  

Hope that helps,

Sha

```You can use a robots.txt file to request that search engines remove your site and prevent robots from crawling it in the future. (It's important to note that if a robot discovers your site by other means - for example, by following a link to your URL from another site - your content may still appear in our index and our search results. To entirely prevent a page from being added to the Google index even if other sites link to it, use a [noindex meta tag](http://www.google.com/support/webmasters/bin/answer.py?answer=61050).)