BEST Wordpress Robots.txt Sitemap Practice??
-
Alright, my question comes directly from this article by SEOmoz
http://www.seomoz.org/learn-seo/robotstxt
Yes, I have submitted the sitemap to google, bing's webmaster tools and
and I want to add the location of our site's sitemaps and does it mean that I erase everything in the robots.txt right now and replace it with?
<code>User-agent: * Disallow: Sitemap: http://www.example.com/none-standard-location/sitemap.xml</code><code>???</code>
because Wordpress comes with some default disallows like wp-admin, trackback, plugins. I have also read other questions. but was wondering if this is the correct way to add sitemap on Wordpress Robots.txt http://www.seomoz.org/q/robots-txt-question-2 http://www.seomoz.org/q/quick-robots-txt-check. http://www.seomoz.org/q/xml-sitemap-instruction-in-robots-txt-worth-doing I am using Multisite with Yoast plugin so I have more than one sitemap.xml to submit Do I erase everything in Robots.txt and replace it with how SEOmoz recommended? hmm that sounds not right.User-agent: *
Disallow:
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-login.php
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /comments**ERASE EVERYTHING??? and changed it to**<code>
<code>
<code>User-agent: *
Disallow: </code>Sitemap: http://www.example.com/sitemap_index.xml</code><code>``` Sitemap: http://www.example.com/sub/sitemap_index.xml ```</code><code>?????????</code> ```</code> -
This is a duplicate question and was answered in another thread.
Thanks!