Questions
-
Moz Crawler URL paramaters & duplicate content
Happy to help! We crawled roughly 49k pages because there were that many links on the site that we could find. 50k is also the new standard crawl limit for campaigns in Standard and Medium subscriptions. Adding a rel=canonical to a page doesn't mean it won't get crawled by our campaign crawler, only that the crawler is to refer to the canonicalized link for reporting purposes. Without going into too specific of URL details, these pages are considered duplicates because their canonical tags point to different URLs. For example, DOMAIN.COM/charters/search/mx/QR?booking_date=&booking_days=&booking_persons=search_location%25252525253Dcabo-san-lucasbooking_date=&booking_days=&booking_persons=search_location%25252525253Dcabo-san-lucas&limit=20&offset=20 is considered a duplicate of DOMAIN.COM/charters/search/mx/QR?booking_date=&booking_days=&booking_persons=limit%252525253D20 because the canonical tag for the first page is DOMAIN.COM/charters/search/mx/QR?offset=20 while the canonical for the second URL is DOMAIN.COM/charters/search/mx/QR Since the canonical tags point to different pages it is assumed that DOMAIN.COM/charters/search/mx/QR?offset=20 and DOMAIN.COM/charters/search/mx/QR are likely to be duplicates themselves. Here is how our system interprets duplicate content vs. rel=canonical: Assuming A, B, C, and D are all duplicates, If A references B as the canonical, then they are not considered duplicates If A and B both reference C as canonical, A and B are not considered duplicates of each other If A references C as a canonical, A and B are considered duplicated If A references C as canonical, B references D, then A and B are considered duplicates The above example from your campaign actually falls into the fourth example I've listed above. Hope this helps clear things up
Other Research Tools | | SamWeber0