Why does the moz crawl test lists page twice?

[tpushpathadam]

Hi,

I'm running into an issue where some crawlers list my pages twice, once with a trailing slash, once without. I first saw it on a few pages with screaming frog, then saw it happen on all my pages with the moz crawler.

The site is www.kidsandart.org and its on Squarespace. I grepped the sitemap.xml I submitted to google webmaster and got 167 distinct pages, all of them without a trailing slash.

Any insights on why this is happening, and how to regard moz crawler results would be appreciated.

thanks

Tom

[iSTORM-New-Media]

This is a problem I have encountered with SquareSpace. SS does not automatically redirect non-trailing slash pages to the trailing slash version. This creates duplicate content issues and reporting issues, as:

www.kidsandart.org/artists

and

www.kidsandart.org/artists/

will be recorded as two separate pageviews on your reporting.

If you want to know more about why it's recorded as two separate pages, read this blog from the Official Google Webmaster Central Blog: http://googlewebmastercentral.blogspot.ca/2010/04/to-slash-or-not-to-slash.html

Unfortunately this is just a problem with SS and since you can't use htaccess to redirect pages or use the built in Redirect Tool to do this, you are kind of just stuck until SS improves their system.

[iSTORM-New-Media]

One thing to note that I forgot to mention is that SquareSpace does use rel=canonical tags on their pages. So while your reporting may be off, link juice will not be split between / and no-/ pages. This also takes care of duplicate content problems, as the tag defines the source content.