IIS 7.5 - Duplicate Content and Totally Wrong robot.txt

[TiffenyPapuc]

Well here goes!  My very first post to SEOmoz.

I have two clients that are hosted by the same hosting company.  Both sites have  major duplicate content issues and appear to have no internal links.  I have checked this both here with our awesome SEOmoz Tools and with the IIS SEO Tool Kit.

After much waiting I have heard back from the hosting company and they say that they have "implemented redirects in IIS7.5 to avoid duplicate content" based on the following article: http://blog.whitesites.com/How-to-setup-301-Redirects-in-IIS-7-for-good-SEO__634569104292703828_blog.htm.

In my mind this article covers things better: www.seomoz.org/blog/what-every-seo-should-know-about-iis.  What do you guys think?

Next issue, both clients (as well as other sites hosted by this company) have a robot.txt file that is not their own.  It appears that they have taken one client's robot.txt file and used it as a template for other client sites.  I could be wrong but I believe this is causing the internal links to not be indexed.  There is also a site map, again not for each client, but rather for the client that the original robot.txt file was created for.  Again any input on this would be great.  I have asked that the files just be deleted but that has not occurred yet.

Sorry for the messy post...I'm at the hospital waiting to pick up my bro and could be called to get him any minute.

Thanks so much,

Tiff

[djlaidler]

Hey Tiff,

I don't have a complete answer for you cos I am doing some insomnia q&a (it's 4.30am in Oz) but I really hope you are using robots.txt over robot.txt? This may be the reason is not working.

Can you tell us what the contents of the robot/s.txt is?

Dan

[jesse-landry]

yeah i think to answer we'd definitely need to see the robots content.

have you determined those pages are truly not indexed at all? like, have you googled specific sentences from those pages and returned zero results?

if not, do so and make a list of all the pages that aren't being crawled by Google and compare that to your robots.txt...

Or just make a new robots.txt file for each domain. Shouldn't take more than a minute or two